I agree with you. If you're not interested in black women, you're a racist. If you ARE interested in black women, you'll still have to prove that you're not a racist. You might get accused of fetishizing them, or in Bill Burr's case - having a "minority sex slave".
Damned if we do, damned if we don't.
It's the same thing with transwomen - if you're not interested in them, you're transphobic. I personally would love to have a cute transgender girlfriend, but as I have recently experienced, those of us men who ARE interested are often immediately bashed as "chasers" or "fetishists".