Also you forget Haldane's Rule, which, incidentally, you always used to bring up.
To date no Neanderthal Y-DNA markers have been found in modern humans. Doesn't mean male Neanderthals didn't mate with Sapiens females, but that we only have evidence of Neanderthal females with Sapien males.
You used to also mention that Sapiens had larger communities while Neanderthals were antisocial and practically solitary. Hard to kidnap, carry away, and rape when you're solo against a big group.
You're offering conjecture that if true, it would have been the reverse, species-wise, which, given what we know about modern humans' propensity towards rape, unfortunately was very likely the case, and gang-wise, as well. Kill the males, rape the females. En masse. And thus they went extinct as a species, the women leaving behind hybrids now known as Caucasoids.