I avoid people because they suck.
Take organized religion for example. That is possibly the best, and most widespread example of voluntaryism. Aside from children, nobody is compelled to participate (middle eastern countries excluded). In my experience, the highest concentration of thieves, liars, and child molesters - participate in organized religion.
I say this because I have direct experience. The reason is simple, association with the group diminishes their personal transgressions and provides a place for them to hide with people who will defend them against attack from outsiders.
Good luck with your pipe dream. I hope it attracts a lower percentage of garbage humans than religion does.