If you go back far enough in history, most every square inch of earth has been conquered and re-conquered and conquered again. Human history is shaped by conquest. Islam wasn’t the first or the last to do so.
Example: The modern-day map of Europe is in many ways descended from the legacy of Roman conquests, and indeed the influence of Christianity as a state-backed Roman religion from the time of Constantine onward.
Why isn’t that legacy just as much of a problem?
Do you think it was all justified as the Romans were “civilizing” barbarians?
Well: I guarantee you the Islamic conquerors saw it that way, too. And they weren’t exactly wrong, either. Middle-Age Islamic art and science was cutting-edge and bolstered by the networks of common culture and trade that the caliphates established.
And let’s not forget how European Christians conquered and colonized a gigantic chunk of the world not to mention the entire New World. All of that activity in the aggregate was outrageously more destructive (or, to use more neutral language, “impactful”) on human history than the wave of Islamic conquests in the 600s-700s.