He is dog-whistling all the time to the racists. From his spurious attacks on candidate and then President Obama's citizenship, to his describing protesting people of color as "thugs", to his “…They’re bringing drugs, they’re bringing crime, they’re rapists." If you voted for him, you voted for a man who unashamedly tries to normalize and promulgate racist ideas.
Perhaps you're white and it didn't seem to affect you, but it's there. He never hid it.