They used the same argument that is still effective today for getting the masses to do their dirty work for them. Pushing social issues to hide their real motives.
Yes it was a small percentage that actually owned slaves, but they also held the political power and made sure to use it.
And here we are again. Claiming state's rights when it comes to social issues while the super rich plunder the nation. Try ignoring the simple answers and flow the money. It is a better guide to who is screwing you and why.