Native Americans are STILL barely a blip, Jews aren't much higher in number either, and contrary to rumor, only a very few rich owned slaves, making their descendents only 5% of the population till fairly recently. They only hit the 10% mark a decade or so ago.
Catholics only INVENTED Christianity, although its runaway rebel child, Protestentism, was, and still is, the largest Christian denomination in the USA.
Things weren't always so pleasant for minorities, but the standard of living in general was relatively high for the times.
Crime was a lot lower than today. Goodness, that's even including the fact that some of today's everyday crude behavior was considered criminal or close to it. So even with a broader definition of crime an/or unseemly behaviour, it was still significantly safer than today.
Or did ya really think granny and grandpa were lying to ya?