That's because there's more than one way to define sex. As you said, through genitalia, or through endocrinology, genetics, gamete sizes, etc. And even then, any biologist worth their weight in student loans would tell you that we don't make classifications to be the definitive end-stop on anatomical definitions. Rather, that "we" try to create definitions that catch as much of the characteristics as possible so laymen like us can talk casually about it and speak in broad general terms.
That said,
Gender is what we use to socially describe the attributes of one's sex. So, for example, a classic western-human (Had to specify to locale as social definitions vary by culture) male won't be found wearing light colors like pinks or yellows or baby blues or floral wear like you would find with a western-human female. Men have been recognized in the past as the "Breadwinner" or the "provider" of the household.
For some reason, the right wing can't get the difference between sex and gender.
Because of this error, they often conflate transgender with transsexual.
Truly, it boggles the mind.