The Neanderthals never accepted to share their food, so war was declared. The Neanderthals were a lot stronger (physically) but our ancestors were a lot more innovative and had a higher rate of reproduction. This lead to 100,000 years of ongoing global conflicts between the two species.
Just because some individuals of both species mated doesnt change the fact that both species went to war and competed eith each other. Not to mention the matings where not always consensual as many times Neanderthal males violated female humans, and Neanderthals themselves where promiscous and mated with several partners.