Probably the same number of people who think aliens are using mind control on them and only tin foil hats can save them. There will always be wackos.
The point is, expressing legitimate concern based on fact does not make it a phobia. Saying it is only because they are different when that is not the underlying reason is disingenuous.
Most people don't fear radical Islamist because they are Muslim. They fear them because they are evil and are dangerous to society.
A Pew Research Study found that 8% of the Muslims in Turkey had a favorable view of ISIS. Though that percentage seems low, it still represents 6.5 million Turkish people. The percentage is higher in places like Pakistan, Nigeria, Iran, and Egypt.
According to Pew, most Muslims have an unfavorable view of ISIS, however, the numbers suggest there are, at a minimum, a hundred million ISIS supporters worldwide.
https://www.pewresearch.org/fact-tank/2015/11/17/in-nations-with-significant-muslim-populations-much-disdain-for-isis/