I downvote bad memes, overly repeated reposts, memes asking for upvotes, memes attacking another user, memes created by trolls, and fnaf and undertale memes.
I use votes as steering (especially when looking at the Hot section).
Upvotes - towards the front page. Downvotes - away from the front page. Keep scrolling if I don't care.
Other users don't lose points when you downvote them (unless you're changing your mind about a previous upvote), so it doesn't hurt them if you downvote. And I think it's important to make sure that the first few pages of Hot have the best memes possible.