Speaking as someone who fits both criteria, this is true. I've also had people have the gall to correct me with "people of color" instead of "colored people" (never mind that there's practically no difference between the two terms, and the fact that I'm a colored person (with some white ancestry)), or call me racist for not liking Michael Jackson's music (never mind that one of my colored heritages is black, and you can dislike someone or something without their ethnicity being a factor).