The correct preposition to use in this sentence is "to".
So, the correct sentence is: We must go to the policlinic.
Translation: Мы должны пойти в поликлинику.