That's funny because my family has an average income and two rental properties because of past relatives. When they died, my parents took ownership of the house, and we decided to rent it to other people.
So, your "meme" is not true. My family is not rich. We are, however, landlords. My mom works part-time because of a disability, and my dad works a full-time blue-collar job.
So yes, you are stealing from me and my family if you support this idea.