That would encourage people to create fake accounts. By displaying only the positive metric, the benefit to down voting is only personal points - you can't actually hurt anyone with them.
I would like that but then people would complain all the time they have so many downvotes, but it would make me happy to see people who are upvote begging
And the downvote begging people like me-