They do it in Britain, Australia, and parts of China and Japan. They are only clothes mate. Nothing to be ashamed of.
Society is so conditioned these days. We accept people walking around in very short mini skirts, including men. We also accept idiots with ridiculous hairstyles, or men wearing women's clothes. Bur as soon as a person steps out of the house with pyjamas, all Hell breaks loose.