ok. having thought about it as deeply as i can, i have some ideas. maybe you won't hate them all ( :
first of all, some critters can see light in the infrared frequency range and some can see in the ultraviolet, neither of which humans can do at present. do those critters see 'extra' colors beyond ROY G BIV, or do they only see ROY G BIV shifted in their direction or maybe even spread out over a wider range of frequencies compared to us? i don't see how to answer this question by any experiment, but the idea itself raises the possibility that what we call orange or purple or whatever is not strictly associated with a specific frequency range, which i find interesting as an idea itself.
it also raises the possibility that colors can only be described relative to each other. for example, green is the color that is simultaneously most like yellow and blue. orange is the color that is simultaneously most like yellow and red. of course it doesn't answer the original question absolutely.
why we see red as red (if indeed we all do, certainly an open question) may have everything or nothing to do with how brains came to work over millions of years. red is a perception of the lowest-energy electronic transitions, while violet relates to the highest-energy electronic transitions that we can see. we might then expect these two colors to be the most different from each other, but that's not how they look to me. yellow and green look pretty different from each other, at least to me, and they correspond to very similar-energy transitions. i know what red looks like only because someone told me as a child which color is red, as with all the other colors.
the brain is made mostly of water and protein, and yours and mine have very much the same basic structure, so if 'redness' is only a way of perceiving the lowest-energy transitions, it wouldn't be surprising if it turns out that we all see it the same. and if we do all see it the same, then redness may be the result of a shared perceptual phenomenon that links physics to whatever consciousness is. unfortunately, we'd probably need to understand consciousness and perception a lot better than we do in order to test that idea.
and if we don't all see it the same, that would be even more interesting.