Yup. Of I say mate or matey, I still think your my friend. I'll call ye a baka as a joke though. I'd never actually call somebody an idiot. Unless if they are. Then I'd say it in my native language of Polish.
Truth is.... There is no gender neutral word for idiot in Polish, just the masculine or feminine. Just like many other words. Phew.... That's a lot of words.
