What ___ you ___ at 5 o'clock yesterday?

The sentence is in the past continuous tense, indicated by "yesterday at 5 o'clock". The past continuous is formed with "was/were + verb-ing". Since the subject is "you" (plural), we use "were". Therefore, the correct option is "were...doing".

Answer: were...doing