What you're advocating for isn't tolerance or free speech. It's consequence-free speech.
You want to be able to say whatever it is you want to say whenever it is you want to say it and not suffer any consequences for it.
You don't get to do that.
You'll agree with me that a tolerant society is one that promotes equality. That equality means that all of our perspectives should be given the same weight, right?
Hate speech, however, does the exact opposite. It promotes inequality. Hate speech says that the listener is less than the speaker- usually, that the subject of the hate speech is less than human.
A tolerante society doesn't have to put up with that kind of bullshit. A tolerant society can tell the speaker to GTFO with their nonsense.
That's the apparent paradox you're just approaching with your meme- that a tolerant society isn't very tolerant (to hate speech).
And, yeah, it doesn't have to be. So, if you say racist, sexist, or any other kind of nonsense expect to be called on it and expect to have consequences.