I know the feeling. Whenever I used to be into writing it was a common occurance, but don't do what I did and take the cheap, undignifying route of including erotic content.
I once wrote what I thought was generally a wholesome story that happened to include one "scene" in it. The only feedback I got? "I didn't read the rest of it. I just wanted to see the $e# part, and I liked it."
Keep doing it because you like it. Don't give up on yourself or stoop to the level of someone who doesn't enjoy your stuff as-is.