Being on the ace or aro spectrum is just as valid as being gay, lesbian, bi, pan, etc. There are a lot of close-minded people out there who have a hard time accepting things they can't personally relate to and people who might not realize LGBTQ is much more than just the gay people. This seems especially true with older generations (Gen X, boomers, and maybe also some millennials even though it hurts to admit that. I like to think millennials aren't as bad as the other two though.) who may have either grew up with a phobic societal view of LGBTQ or a very limited understanding of LGBTQ.
It doesn't make what they did right of course. What they did was phobic. I’d suggest ignoring people like that because it likely came from a place of ignorance unless you want to put in the effort to try to explain what ace is.
When I explain to people like that, I try to keep it simple. You know how some people like guys, some people girls, and some people like both? Well, there are also some people who have no interest or very little interest in anyone. These people don't feel the same urges other people do. They experience attraction differently than others do and that's more than ok.
If they are still phobic after an explanation, then I’d say it's genuine phobic behavior. I know it's annoying and tiring having to explain yourself after a while though and sometimes you might not have the energy or be in the mood to do so, and that's ok.