Technically there is nothing wrong with the flag itself. It's just a flag primary used for that what flags are for in general. To be a symbol of the nation. Nothing more. Of course, now that I see the flag most of all shown out of motivations that are not exactly "clean", it has become a symbol for racism and slavery.
Of course, there are more symbols that have very controversial feelings now, and are quite often also used in that manner. Banning symbols is not always easy. Especially not in a free nation. "Freedom of expression" is often used as a reason. But another message is that burying the past will not erase it. And the confederate flag has become a part of America's history. And if you use the flag as a reminder not to repeat the mistakes of the past, then the flag suddenly gets a different meaning. There are many symbols used for "evil" in the past (I hate the word "evil" as it's always a manner of where you stand, but by lack of a better word), now being used as reminder not to repeat the "evil" of the past.
Now the confederate flag may not cause this dilemma, but another controversial symbol, the swastika, is originally a holy symbol. A religious mark. For many people it stands symbol for fascism, due to Hitler adopting the mark as the symbol for Nazi-Germany, yet banning it would be a violation of freedom of religion, and the religious meanings do predate Nazi-Germany. Many constitutions stand in the way of banning that symbol for this reason. Germany itself has banned the symbol but had to leave a clause that makes an exception for when the symbol is used in its original religious meaning. When it comes to the flag, some reason for resistance is not just the the racist symbol people take if for, but it did use to be a national symbol, and thus part of people's identity. And when you get to attack people's identity, then people get onto the fray. I see this in other countries too were discussions are being held about things that some people associate with racism and slavery and plead for removal, only to get people to get insulted and jump onto the fray.
It may therefore not just be the flag itself.... It's rather that it does not only symbolize racism and slavery, but for some people also their identity. Never underestimate how much value people put into that. If it's worth to care about... I dunno... I only get the feeling the more people try to ban the flag the more they'll try to defend it. Oh well...