It all started when people started growing food and living in one spot all the time, when it became more profitable to use prisoners captured in battles for free labour, as opposed to simply killing them.
And, economically speaking, it is pretty profitable to have someone work for you without having to pay them anything else than the most basic necessities they need to stay alive, despite it being immoral.
And the reason why it took so long before people started to perceive slavery as immoral is that it was so normalised. In antiquity, almost every home had like, about 1-4 slaves. And when something is normalised, most people who aren't negatively affected by it don't really question it, since it benefits them and they aren't the ones who are harmed by it.
Most people don't really care about [insert oppressive social structure here] before the said social structure affects them or their loved ones negatively, or the people who are negatively affected by it start protesting against the said social structure.
Caption this Meme 