To add to Louis's excellent answer, I'll provide some statistics:
There are a total of 131 disputed-review-audits, including those deleted. They span 281 days, putting this at roughly 1 disputed audit every 2 days.
During this same time period, there have been 283,545 audits and 18,697 failures, meaning we get roughly 1 audit disputed out of every 2,164 shown, or one out of every 143 audits failed.
What does this mean for your proposal? Nothing yet. There's a huge variation in how many reviews (and hence audits) individual users do. A total of 17,424 users were audited during this period, with an average of 16 audits per user and a maximum of 2,935 audits for a single user. In theory then, we could get away with generating 10 or so unique audits per day, as long as we were careful to track who got them and never re-show them. There are a few confounding factors though...
We probably shouldn't be showing you audit tasks based on posts you've already interacted with. This doesn't always happen, but we do try to avoid this now. Of course, this means that we shouldn't audit anyone approving the audits with the audits they've approved.
We'd have to be a bit cautious about changes made to questions after they've been picked as audits. We can use a snapshot of the audit post at the time it was approved, but we should still avoid using known-bad questions if, for instance, they get undeleted / reopened, or known-good questions that get deleted.
If we run out of audits, we're gonna have to fall back on the automated systems. Which means we can't really slack on those either, since all it takes is one person every two days to not like one of 'em and we're back where we are now.
Folks disagree with moderators and 20K users all the time. Just because the system trusts them doesn't mean the folks failing audits are going to. So we'll still get some number of disputed audits.
But let's assume the best case: 10 audits per day. That's at least 5 20K reviewers, reviewing at least 10 potential audits every. single. day. To save answering one meta question every 2 days.
This is why Brad's proposal is so appealing: if we're going to have folks reviewing audits, might as well focus on audits that folks think are actually problematic. I'd probably turn that around though: only review audits that've been reported a couple times and only invalidate them after review.