I have to go to the bathroom once a school day, generally in a class right next to a non-gendered bathroom (if it’s not I just walk to the right building).
Personally as a fellow enby, I agree.
Never felt comfortable to enter the girls bathroom as a kid for some reason although back then I didn't consider myself as non-binary just queer.