They could if they could prove it but they can't. An actual criminal electoral fraudster is never going to stuff ballots of both candidates just to keep it impartial, are they?
The problem isn't that, the problem is that they're trying to strike down votes that are legally protected as being valid until they can prove that they're not and they can't.