When the land increases in value and the rent goes up just because the incomes in the area rose, it's certainly not the landowner's labor that determines their success.
Did any landowner create the land that they own? When the land goes up in value making the landowner richer, is it because the landowner has contributed something to the rest of society?
No, quite the contrary, the landowner is reserving a scarce piece of land from the rest of society. They get richer by depriving the rest of us of that value. What gives them the right to exclusively claim it, denying it to the rest of us?