Being labeled a white supremacist with zero evidence is slander/libel and grounds for lawsuits. He may wish to forget about it and not pursue it, or he might pursue it but not win, but a blanket statement that it's "not going to happen" is ridiculous.