Property ownership is part of the social contract, some other cultures will have no ethical concerns and robbing you, beating, you enslaving you, and killing you as human rights are a social construct. To keep them from coming in and overtaking society, changing norms to allow this legally you'll have to erected a barrier of some sort to prevent just anyone coming in and screen those that you allow in. And that is is addition to the criminal element in cultures where these thigs are immoral, but will do so anyway.
In other words you'll need a boarder and immigration policy. I find it likely to adsorbed the idea of no boarders though cultural osmosis and never thought through the rights you stand to lose if it was actually implemented, because you've only been exposed to those that with Judeo-Christian values.
