Using the phrase "non-men" seems to actually a weird attempt to re-introduce a gender binary. There's men, then there's everything else.
Scientifically, all fetuses start off female, then differentiate during development, so non-female would actually be more scientifically accurate.
The probable reason for using non-male (rather than non-female), is there is almost no backlash against trans males trying to use male only spaces, but there is a huge backlash against trans females trying to enter female only spaces (sports, locker rooms, identifying as lesbian, but being mad real lesbians aren't interested, etc.). They are trying to redefine female only spaces as merely non-male spaces. Very few would argue that a trans female is a man. But many would argue that they aren't real women/females, and thus don't belong in places that are exclusively for females.