But that's exactly where the danger lies. Because it's fictional, it makes you let your guard down. Even if you consciously reject what it's saying, it can still worm it's way into your mind, and affect your thinking. And it's a slippery slope from there. The more you read Hairy Potter, the more it numbs you to the revolting evil that it's protagonists practice, until it gets to the point where it's no great mental jump to think, "This isn't bad. This is just some people having innocent fun, and occasionally using supernatural power to fight against the antagonist." Now, your situation might not get so bad as that, but nonetheless, you still don't need to be filling your mind with that.
But hey, don't do something just because some guy on the internet told you to. Take it to God's Word, and see what He has to say there. But no matter what your decision is from there, I still wish you a great day!