Property taxes are 2 taxes, a tax on land and a tax on buildings, the latter being taxed the most out of the 2. A land value tax would be a tax on the unimproved value of the land.
The value of the land is created by the community and the infrastructure around it, not by one person's labor.