So is race, species, phylum - even life, for that matter, which is really nothing more than a self-replicating chemical reaction with mutate capacity. Other species get by fine without the cultural accoutrements we call language and labels. Male and female body parts and functions are essentially the same, with some differences in size and use - other than the uterus and all that goes with that which only females have.
Bottom line is, aside from what this year's dictionary says, gender has and still is based on sexual identity - both of them. These so-called "other genders" - much like "intersex" which you referred to - are described in terms of aspects of either F or M and differ only in mix or gradients between them.
People do make too much of a fuss about these supposedly huge differences. Me, I've always seen that and all we attach to them as minor. Trying to redefine them and squeeze in a plethora of confusion is a waste of time, and quite frankly, since I first heard about this a few years ago, haven't been able to shake the feeling that it's a con made by haters to make those more prone to favoring such to step their way into an unfavoring light.