Don't give ... trying and your marks will improve. (stop trying)

1. The parenthetical note indicates the meaning is to stop trying.

2. The phrasal verb 'give up' means to stop trying.

3. Thus, the missing word is 'up'.