5) 1 3/4 a²b( 8/7 ab + 16/21 a³);

5) $$1 \frac{3}{4} a^2b(\frac{8}{7} ab + \frac{16}{21} a^3) = \frac{7}{4}a^2b(\frac{8}{7}ab + \frac{16}{21}a^3) = \frac{7}{4}a^2b \cdot \frac{8}{7}ab + \frac{7}{4}a^2b \cdot \frac{16}{21}a^3 = 2a^3b^2 + \frac{4}{3}a^5b$$.

Ответ: $$2a^3b^2 + \frac{4}{3}a^5b$$