I know it seems counter-intuitive but in order to enable Free Speech you do have to moderate speech.
Specifically, when you open up a space for hate, hate's gonna fill it up. Nazi's gonna Nazi, and they'll always fall back on "so much for a tolerant society!" Free Speech requires protection from Hate Speech.
Just like how you don't get to shout fire in a crowded theater.