Given the number of bad reviews in the Suggested Edits review queue (the problem that has been plaguing the queue since its inception, worsened these days by the queue being close to the threshold of 500 items every day) and the implications of the state of affairs (review suspensions and subsequent complaints), it is overdue that we did something about properly onboarding the new reviewers.
Understanding that an interactive onboarding process (e.g., with test review items with known outcomes) is unlikely to be implemented in a timely fashion (or, honestly, ever, for that matter), the following proposal requests an update that would require only minimal (if any) developer time from the company.
After the recent onboarding overhaul, new reviewers started to get a popup upon entering the queue for the first time (which can also be opened any time by clicking the "Learn more" button under the short queue description). Here is what the current guidance looks like:
However, the guidance is either blatantly incorrect, incomplete, or contrary to established community guidelines for editing. Specifically:
- the first bullet point is actively misleading: upon choosing either the "reject and edit" or "improve" option, a single reviewer is enough to handle the review item. Additionally, if there is a disagreement among reviewers, it can take more than 2 to handle a given item;
- the second bullet point lacks either an explicit explanation of what preserving the author's intent means (the wording from the Help Center article can be used verbatim: "changes a post to say the opposite, or something very different from what the original post read");
- the third bullet point needs to at least link to the FAQ on how to make a good edit, which details common reasons why suggested edits can be rejected. Also, it omits to mention the "no improvement whatsoever" common rejection reason (present in the Help Center), opting to focus on the "clearly worsen the post" part, which leads to a lot of confused approvals — good edits have to improve the post in the first place, not just make it "not worse". What's worse, it also glances over another important rejection reason — adding irrelevant tags (or removing the relevant ones). If the intention was that the list is not exhaustive, it should be explicitly indicated as such;
- the fourth bullet point contains an unnecessary emphasis on the happy path: "Even small changes can be good edits". This focus often leads to a misguided belief amongst reviewers that any small change (even if it misses critical problems with the post) is a good one, but this is contrary to the editing guidelines. Ideally, the prefix should be removed, leaving only the pragmatic advice reviewers should be looking for: "Choose Improve edit if a post could use more changes".
Another part of the problem is that the onboarding popup lacks any guidance for reviewers who were granted the privilege to review tag wiki and excerpt suggested edits, leading to the two most common reasons for bad reviews there: paying no attention to tag excerpts requiring usage guidance (instead of just reiterating what the tech is) and not being on the lookout for copied content.
Both of the abovementioned guidelines are tucked away in the rejection modal and, given there is no indication that the "reject" button opens a modal with a list of reasons instead of submitting the review, is detrimental to onboarding new reviewers to the workflow of the queue.