AFAIK you keep the icons you've earned and the only way to loose points is if someone takes away a image or comment upvote that's less than a month old.
HOLD UP... CAN YOU "GRIND DOWN"? APPARENTLY YOU LOSE POINTS IF YOUR POST OR COMMENT GETS DOWNVOTED SO IS IT POSSIBLE TO GRIND ALL THE WAY DOWN BACK TO 0? AND WILL YOU STILL HAVE THE ICONS?