Difference being bigotry is a hateful mental outlook that can be reformed, whereas being female or gay or black or disabled or from a different country (or all of the above) isnβt.
Which is why shunning bigots is appropriate and beneficial, but discriminating against those who were born a different way isnβt.