That statue wasn't bothering anyone. It wasn't walking up to people and tapping them on the shoulder and preaching about how great slavery was.
The implied offense is... guess what? Implied. Here's what's not implied; A bunch of SJWs urinating on a piece of bronze because it's shaped to look like someone they disagree with. That's reality.
...and just for sh**ts and giggles, let's have a peek at the big bronze statue of Vladimir Lenin in Seattle, WA and tell me how this blood-soaked communist who isn't even an American, much less an arguably "evil" American like the SJWs claim Robert E. Lee was, has a statue and that's perfectly okay. But honor a General who won many battles that he had no business winning logistically, because he was a brilliant military tactician, and was incidentally given a blanket pardon along with every other Southerner by President Lincoln, now that is an affront to everything America stands for.
Please. Please. Please. Can we make far-reaching decisions with our heads instead of our hearts? Because the human heart is a big, whiny, entitled douche that should be mostly ignored. Think, don't feel.