Thank you for the detailed feature request. It's a refreshing break from the usual "X is broken, fix it!" requests that we usually get. This being said, I'm skeptical.
Highly likely benefits:
- Less moderator time spent on debating the validity of X specific audit.
Probably less but is it going to be lower enough to counter the work needed in the new queue? I am quite certain that people are still going to complain about audits, seeing as many complaints are illegitimate. (e.g. Someone letting through what is clearly spam, failing the audit and then complaining.)
Even when the complaint is legitimate, moderators do not need to be involved in the resolution (for instance). I realize this does not entail that moderators did not read the complaint or did not perform any moderation action on the complaint (e.g. deleting comments).
Looking at disputed-review-audits
, it seems to me that 1 dispute a day is an overestimate of the rate at which audit disputes come up on Meta. I'm thinking it is enough of an overestimate to account for deleted questions but I could be wrong. Only someone who can access deleted questions would be able to quickly get statistics that include these. 1 dispute a day is not much.
- Considerably fewer meta posts from normally-competent reviewers getting banned or wanting an audit disabled.
I don't think normally-competent reviewers get banned. To get banned one has to fail audits to the point of no longer being "normally-competent".
Potential benefits:
- Slightly fewer bad reviewers, if the algorithm is relaxed to allow more subtleties in and mods/20kers pick good hard audits that are still unambiguous.
No thanks. The audit system is designed to quickly weed out robo-reviewers. It is not designed to be a bar exam. If you start adding subtleties to the system because you are trying to teach people how to review, what you are going to end up with is more people complaining. "Ah, yes you clicked 'leave open' but if you look at the 34th sentence in the 3rd paragraph, you'll clearly see the OP is asking for an external resource. Gotcha!" And they'll complain even more stridently than they do now because they audits will have been hand-picked. They'll feel like someone is out to get them. Right now we can tell them that the algorithm is stupid.
Then it seems to me that the biggest benefits this proposal would provide would also for the most part be provided by Brad Larson's proposal. His proposal seems simpler to implement and would require less overall human intervention.