Review audits are a hard problem. It's difficult to programmatically generate hard audits with a high enough degree of confidence to act on them. This results in a lose-lose situation at the moment - people (rightly) complain about the poor ones that are generated, but they also usually feel formulaic and easy to game if you spot the inconsistencies in how they get generated. Fixes to this system of audit generation are just tweaking the sensitivity of a ROC curve.
On the flip side of the coin, the moderator tools for handing out review bans are clunky. It shouldn't be a big deal to say "this wasn't right", and let the system take it from there. (As an example, there are often three users involved in letting spam slip through, to message and ban each takes two copy and paste operations per user, plus digging around to determine appropriate lengths).
This is a feature request for a "convert to audit" tool for moderators.
When we see a post that clearly should have been picked up during the review process, https://stackoverflow.com/review/first-posts/XXXXXX is the starting page. I'd propose that the UI for this feature is run off that page and includes:
A "what should have happened" information capture.
On suggested edits that's a reject reason, for other queues it's probably pretty much the standard flagging dialog. This can be used as a learning point later.
A "who was clearly wrong" information capture.
This is probably a checkbox next to each reviewer. Pre-populated based on #1.
An if/how long to ban incorrect reviewers option.
This should be pre-filled automatically. There really isn't any sense in making humans guestimate how this person has reviewed historically when there's already data that's a great starting point.
A "promote to audit" checkbox.
This would push the review item, anonymous but otherwise unchanged, into a pool of manually picked audits. The new pool is likely to be small so it would not replace, but rather augment the current mechanism with better audits that are both broader and deeper in coverage than anything automatically generated can do.
Items that slip through are (or should be) edge cases - exactly where moderators are expected to be acting. Please can we have a tool that can help fix three issues in the current setup simultaneously.