Short men ARE discriminated against, especially in the workplace, and are much more likely to be victims of violence, for obvious physical reasons.
"Also, a woman rejecting a short man goes back to the whole physical attraction factor, as opposed to saying all short men are inferior to taller men as a whole." Except that's precisely how many women think of short men - inherently inferior to taller men.