It's impossible to be taxed on something unless you own it. You're confusing land tax with rent. When I pay income tax, am I renting my body? When I pay sales tax, am I renting my business?
Land value is not created by land owners. So taxing it, UNLIKE EVERY OTHER TAX, is fair.
Zero land tax will result in the nation being owned by a few investors and the rest of society paying them rent. The only way to allow us equal access to land is to abolish every other tax except on land. This what the "laissez faire" economists were advocating. But we never hear about classical economics (the difference between land and labor) because both democrats are republicans are funded by land price investors who love cheap labor and people who are desperate for money.