We are our minds, what gives you the ability to judge whether or not someone is wrong about their gender?
It isn't what you described, the issue is that their gender does not match their biological sex.
It's an established fact that gender and sex are not the same thing.
Historically, it was used interchangeably with sex but like everything, language evolves. It has changed as our understanding has changed, we are always moving forwards. I appreciate that that might be difficult for people who base their morality and understanding of the world on an ancient system rather than current experience. If you can adapt to the Earth not being flat or at the centre of the Universe, surely you can adjust to this?
gender (n.)
c. 1300, "kind, sort, class, a class or kind of persons or things sharing certain traits," from Old French gendre, genre "kind, species; character; gender" (12c., Modern French genre), from stem of Latin genus (genitive generis) "race, stock, family; kind, rank, order; species," also "(male or female) sex," from PIE root *gene- "give birth, beget," with derivatives referring to procreation and familial and tribal groups.
Also used in Latin to translate Aristotle's Greek grammatical term genos. The grammatical sense is attested in English from late 14c. The unetymological -d- is a phonetic accretion in Old French (compare sound (n.1)).
The "male-or-female sex" sense is attested in English from early 15c. As sex (n.) took on erotic qualities in 20c., gender came to be the usual English word for "sex of a human being," in which use it was at first regarded as colloquial or humorous. Later often in feminist writing with reference to social attributes as much as biological qualities; this sense first attested 1963. Gender-bender is from 1977, popularized from 1980, with reference to pop star David Bowie.