You can get rid of them only if you take of their wand or being at an age that they can't control their powers. But some wizards are what we call Animagi and in 20th century they were only nine of them (and some others that the Ministry of Magic didn't knew, such as Rita Skeeter, Wormtail, Padfoot and Prongs that cept it secret) if they are small enough, they can escape and nobody will understand them. If they're big, they can just attack them and they'll escape too
