There is a statistical correlation between men's and women's brain patterns, but if you accept that as a defining characteristic, and that it is integral to gender, you have also made an argument for treating women an men differently legally. If men and women are inherently and always different psychologically (and this is even more important than physical presentation?), it makes sense to treat them differently. Women shouldn't be in the military because they are meek and not naturally aggressive. Men should not be babysitters or school teachers because they are not naturally nurturing.
Just because a male presents feminine personality traits does not make him, in fact, a woman. He is merely a feminine man, and that's okay. If he want's to live as woman, I'm okay with that, as it does not affect me. But that does not make it factually true.
We allow many false beliefs in our society, because they are comforting, and in some cases essential for us continue through life without having a psychological breakdown. If another persons comforting lie does not adversely affect society, there is no need to take it away. The psychological damage from insisting on the truth can be greater than allowing the comforting lie.