Prologue
Since late 2019, I have been investigating why posts are ending up in the Help & Improvement (H&I) review queue when it should have been closed in Triage. Unfortunately, due to the poor choice of wording of the Triage action Requires Editing button, more work is created for everyone:
- new questions don't get closed fast enough
- poor questions flood the H&I review, and users are not able to edit them, so it's basically a skipfest
- I have to develop my own tools to manually handle such H&I reviews, by being able to manually close the question in H&I
- I have to visit each question's post timeline to fetch the Triage review that pushed the question into H&I (happens when 3 users vote for Requires Editing)
- I have to apply manual bans and pray hard that the users chance upon the review ban message, WHICH IS NOT ALREADY APPARENT AND THEY GO ON TO DO THE SAME THING AGAIN AND AGAIN.
Apparently, Triage has been this way since March 2016:
- Triage Review Queue - Inconsistent guidelines for "Requires Editing"?
- How to fix triage! (?)
- Adopt a consistent policy on what "Requires Editing" means in the review queues
- The incorrect usage of "Requires Editing" is still causing problems. Can we do something about it?
- Can we have more clear instructions for review queues?
- and many more: https://meta.stackoverflow.com/search?tab=newest&q=%5Btriage%5D+%22requires+editing%22+is%3Aq
Which has been made worse with the granting of review privileges to at least 53,000 new users recently, an understatement. (60 answers, score of -501 on MSO)
Current Status
Currently on Stack Overflow there are 2894 banned reviewers. (this is not the norm if you are wondering - just a year ago it averaged 200 users at any one time)
Out of which only 118 are automated bans for failing review audits (~5%)
2118 review banned users are manually banned for clicking Requires Editing in Triage, when the question should have been closed instead
Out of 2894 currently banned reviewers, 2767 are banned from Triage for some reason (combined audits and manual) 96%!!!
I am currently adding manual bans every day at an average rate of 200 per day, and even more bad reviews are NOT being caught as I am the only moderator who is handling this issue
Also, at this rate we are going to have 4000 banned reviewers around April 1st, and probably 6-7 thousand by end-April
See live Google spreadsheet for full stats and trend graphs.
Quick Wins?
This Triage issue, if unable to be fixed without further analysis and investigation, could still be alleviated by implementing the following:
1. Make it more obvious that you're review banned
- so that users can be alerted to the reason and avoid making the same mistake
- perhaps push a notification to the user's global inbox?
- (and 15 linked questions)
2. Retain a link in the top bar review queue menu for users in review suspension
- same as above, currently the only way to read the review ban message is to go to stackoverflow.com/review, but there is no link to this page if you're currently review-banned.
3. Raise the review privilege from 500 to 1000
- due to the free rep users got from the doubling of question rep from 5 to 10, we have at least 53,000 new users who suddenly flood the review queues without knowing how to review
4. Implement a compulsory tutorial/guide when users gain the privilege to perform reviews
- e.g.: like the new close votes privilege: New close/reopen privilege explanation for users who earn it for the first time
5. Change the text of the action buttons from "Requires Editing" and "Unsalvageable" to avoid confusion
- to "Needs an editor" and "Should be closed..." respectively
Further steps
Join us in the public discussion room where users are getting banned live, for daily updates (stats/graphs), and escalation plans on the situation:
>> Bad Stack Overflow Reviews