Downvotes will lower a user's score, but you would need a lot of downvotes to make a difference. And if your meme gets 2 downvotes and no upvotes, it goes back to submitted. There was actually an upvote/downvote bar that you could actually see, but sometime last December they did away with that, but the downvotes do still work. The easiest way to see this is when a bad troll comment gets downvoted and it hides the comment.