This isn't a good idea. The reason review audits work (though they aren't perfect as you pointed out) is because all actions that you can take in the review queue are also possible to take outside of the review queue. So if you fail an incorrect audit, you can (and are encouraged to) take the opposite action outside of the review queue, which is why review audits only stop robo-reviewers and not people who are really sure about what they're doing.
Voting is different, since you already vote directly on the post and there is no way to vote without clicking on the vote button. If you fail a bad review audit, you can open the post in a new tab and take the correct action from there. But what do you do if you fail a bad voting audit? The system already stopped you from voting on the post correctly, and there is no way you can get around this to vote on the post correctly without voting. So you're stuck being forced to think that a good post is a bad one or that a bad post is a good one.
There are a lot of bad answers out there which have a positive score and on which downvotes should be allowed.
For example, this answer is accepted and has a score of 7 even though it doesn't answer the question (all it does is it suggests a workaround when the other answers actually provide an answer).
Even worse, this answer is completely wrong. Now it has a score of -2, which is good, but when I saw it the first time, it had 2 upvotes and no downvotes; I was the first one to downvote it. After I downvoted it, 3 other users also did, which is good. But if there were voting audits as you suggest, this answer could have been chosen as a voting audit at the time it had 2 upvotes and no downvotes, telling me and the 3 other users who downvoted it that we're wrong, when it actually is the answer that's wrong. That would lead to a wrong answer having 2 upvotes, and nobody would have the possibility to downvote it. You don't want that, do you?
Besides, what good would voting audits do? You mention users voting incorrectly. I can think of several reasons why they would vote incorrectly, but none of them would be solved by voting audits:
I would interpret the way you wrote your question as that you think people are intentionally voting incorrectly to sabotage the system. That sounds far-fetched to me. There might be one user who does that on one post, but the problem is clearly not big enough for there to be any need to do anything about it (if there even is a problem, which I doubt).
Users can also vote incorrectly because they're serial voting. But there are already ways to handle this. There is a script that reverses serial voting, and users who do it too often can get suspended for voting irregularities. This stops serial voting a lot better than voting audits would.
Another reason why people could vote incorrectly (though I've seen no evidence of this) would be robo-voting: like for robo-reviewing, people carelessly vote on everything they see just for the badges. I've never seen any evidence of this happening, but even if it were a problem, voting audits wouldn't be a solution.
If there were voting audits, people who could see the score would just upvote everything that has a positive score and downvote everything having a negative score in order not to be caught by audits, and the audits would be useless.
Of course, the score could be faked like it is in review audits, but I would find it very frustrating if I'm looking for an answer to a programming problem and the best answer with everything I need is all the way on the bottom just for an audit's sake.
Besides, if people go around robo-voting just for the badges, they would probably do it from the home page or the Questions page where most posts were posted just a few minutes ago and have 0 score and therefore wouldn't be selected as audits.
For these reasons, I think that voting audits would be harmful and not do any good.
If a user consistently sabotages this, and enough users do that, it can cause harm.
That sounds like an imagineered problem. There is zero incentive to behave like this. There's nothing to be gained from it and no viable cause for a group of like-minded users to unite under. – Pekka Nov 18 '17 at 8:24