Maybe it has something to do with the fact that there's a difference between migrating and colonizing. When Europeans went to Africa and Asia and India and the Americas, they were not there to assimilate they were there to colonize and take over. If a group of people move to another country with the plan to take over that country and not assimilate, then that would make them colonizers.
