And in some countries they may have unisex bathrooms, but here, with the larger number of violent criminals, it can be quite the risk. Heck, you travel a bit. A lot of these fellas in certain areas are creepy as shit.
Now I don't want to refer to movies for overused tropes, but there is a reason the image doesn't work in more populated areas, other than for the occasional hoodlum in more packed areas, who, of course, are just as dangerous.