I've forgotten my bag, so I need to return / remove home to get it.

The word 'remove' is incorrect in this context. The correct word to use is 'go'.
Corrected sentence: I've forgotten my bag, so I need to go home to get it.