863.
a) $$(-3xy+a)(3xy+a) = (a-3xy)(a+3xy) = a^2 - (3xy)^2 = a^2 - 9x^2y^2$$.
б) $$(-1-2a^2b)(1-2a^2b) = (-(1+2a^2b))(1-2a^2b) = -(1+2a^2b)(1-2a^2b) = -(1 - (2a^2b)^2) = -(1-4a^4b^2) = -1 + 4a^4b^2 = 4a^4b^2 - 1$$.
Ответ: a) $$a^2 - 9x^2y^2$$; б) $$4a^4b^2 - 1$$.