Considering they make a game out of raping women, have pedo huts in Afghanistan, and other nasty stuff... I'd say they rank in the top 10 worst kinds of humans.
The peaceful people are fine, the violent ones can f**k off a cliff for all I care.
Most people do acknowledge that peaceful Muslims aren't the problem. They simply address the fact that moderates (ones who want the extremists to kill us infidels) and extremists make up the majority. You don't see Catholics and other Christians lighting people one fire for being gay or a witch these days. Islam has a long way to go in modernizing itself or it will perish.