He did not create the land or make it valuable. Also, he does not keep it valuable. The land was already here, its value was created by nature nd its value is maintained by the community, especially the title deed system, the justice system and the military.
And if someone lose their ownership of land, its merely a possession. Also, its a responsibility and a privilege to own land.
But, in the case of labor, the worker creates the labor and makes it valuable. But, if he doesn't pay a tax, he goes to jail. That's like slavery. Paying land value tax is fair though.
Yes, we went down this road, but how is going to jail for not paying for income tax more fair than not being allowed eternal, protected land ownership free of charge?