And the thing is that 1.4% of the white population owned slaves, and even less of the other races, but Hispanics and blacks had slaves. Slaves were a sign of wealth, not race. If you were rich, you had slaves, unless you were disgusted by the practice.
And nobody likes to talk about this - but the people who sold slaves to the "white man" were literally the African slave traders who stole children from their parents to make money off them. The "oppressed" and "oppressors" were not simply cut down to race. It was cut down to character and morality. Martin Luther King said he had a dream that his children would be judged by the content of their character and not their skin color. 60 years later, we look at history and people now all through the lens of race. It's just in the name of progress.