Over the years, I've blocked easily 1,000 people on Facebook and Twitter (when I had a Twitter account) and maybe 100 on YouTube and it was always funny when someone being blocked would complain that I was blocking their free speech. I had to point out to them that their free speech had not disappeared at all, but having free speech doesn't mean they have the right to constantly harass me or have my full attention, my eyes, or my ears.
No, I have the right to ignore people particularly when it's obvious that no common ground can be found and we're clearly never going to agree on anything.
Their free speech is still intact as is mine (and others),but we have the right to ignore people with whom we disagree. We don't always have to engage them in an argument or play their games even though debate is certainly very useful to hone one's communication skills and to craft one's counter arguments. :)