$$(\sqrt{5} - \sqrt{12})^2 + \sqrt{(3-\sqrt{12})^2} = 5 - 2\sqrt{5}\sqrt{12} + 12 + |3-\sqrt{12}| = 17 - 2\sqrt{60} + |3 - \sqrt{12}|$$
Т.к. $$3 = \sqrt{9} < \sqrt{12}$$, то $$|3-\sqrt{12}| = \sqrt{12} - 3$$
$$17 - 2\sqrt{60} + \sqrt{12} - 3 = 14 - 2\sqrt{4 \cdot 15} + \sqrt{4 \cdot 3} = 14 - 4\sqrt{15} + 2\sqrt{3}$$
Ответ: $$14 - 4\sqrt{15} + 2\sqrt{3}$$