I've just gained 500 reputation and am thinking about doing my part in the First Questions, First Answers, and Late Answers queues. What are these queues for? What do I need to keep in mind to do a good job and avoid wasting my own or others' time? And what about those audits that I've heard about?
1 Answer
Statistically, there's a greatly elevated chance that a new user's first post(s) (especially one answering an old question) will be generally garbage. Maybe it's spam; maybe it's a misplaced comment, vote, or even question or answer; maybe it's something else. Given this, there are three review queues that exist to separate those posts from the decent/ great ones and handle them appropriately: First Questions (FQ), First Answers (FA), and Late Answers (LA).
Queue Structure
All three queues share this layout:
Review Actions
- Looks OK
- The post requires no action, even after considering all guidance provided throughout this post.
- Edit
- Use the "Edit" review action if, and only if, after your edit the question will be an on-topic question which should remain open and on the site, or the answer will actually answer the question and should remain undeleted on the site.
- The post can be edited by you to resolve all reasons the post is in this review queue.
- Do not use this review action for improvements to the post which do not fully resolve the reasons the post is in this review queue. If you want to make an edit which does not resolve the fundamental issues of the post, then either go to the question page to do so and/or use the small "edit" button in the post-menu below the post, not the "Edit" review action.
- Selecting this option will open the edit pane to either suggest or directly make an edit, depending on your reputation level.
- Recommend deletion ("Delete" for trusted users) (Late Answers only)
- The answer is irredeemable and should be deleted.
- Selecting this option will open a modal allowing you to optionally select a canned comment to be posted under the answer as the "Community" user.
- Share feedback (First Questions/ Answers only)
- The post could be improved by the poster adding/ removing some information from the post.
- Selecting this option will open a modal allowing you to select a canned comment to be posted under the post as the "Community" user.
- Other action
- This option starts out disabled, and will be enabled and automatically selected if you take some other action on the post (vote on it, flag it, comment on it, etc.).
- Note that all other options are still available after this one is enabled, and that picking this option is not required when doing some other action on the post.
Finishing a Review Task
Every single review, no matter what action you take, will end with either selecting an option in the "Actions" list and clicking Submit, or by clicking Skip to skip the review. Even if you've taken some action on the post, the review will not be recorded unless you submit or skip it.
If the chosen action opens a modal (eg. to select a canned comment to leave on the post), this modal must also be submitted to fully complete the review.
Queue Strategies
Whenever reviewing a post, start out by taking some time to consider each of several possible flaws it might have; First Questions (FQ), First Answers (FA), and Late Answers (LA), unfortunately, cover an unusually broad spectrum for review queues, so you have to keep a flexible focus and not simply select Looks OK just because it passes a couple of the smell tests.
Below is a rather thorough list of potential flaws, which you'll want to run through in approximately the order given to avoid wasting time or missing something important. (Actually, it's deliberately rather over-written; don't be too put off by the length. Once you've gotten into the rhythm with some good reviews, you shouldn't need to refer to this very often at all. Still, skimming it every once in a while may be a good reminder.)
Keep in mind that whenever the guidance mentions taking some action on a post (eg. flagging), after taking that action, you'll need to select an option in the right-hand menu and actually submit the review. In FQ/ FA queues, this will often mean selecting the Other action option, unless you also want to take one of the other existing review actions. In LA specifically, if an answer isn't worth keeping around, you should select Recommend deletion instead (this will also prompt you to optionally select a canned comment to be left on the answer as the "Community" user, which you may do if appropriate).
The easy stuff
In any of the 3 queues, if you see a post that:
- Looks like spam:
- It probably is. Flag as such. Don't comment and don't edit — while editing out the spammy content might seem useful, it is actively harmful: It makes recognizing the post as spam harder, invalidates currently active flags, and allows rolling back subsequent active flags, delaying proper final disposition.
- Is rude, profane, whatever:
- Flag as such. Don't bother commenting in most cases.
- Isn't written in English:
- Flag as Very Low Quality (VLQ) and comment, since SO, and indeed most of the SE network, is English-only by policy.
- Is unreadable gibberish:
- Flag as rude or abusive. Don't comment.
- Is something you have no idea:
- Skip this sucker!
- Is something that will take you way too long to work through:
- Skip it!
- Is something that makes you feel too lazy to even try:
- Skip it!
In any of the 3 queues, if you see an answer that:
- Is a misplaced comment or vote (anything that just asks the OP if they still have the problem, thanks someone for helping, confirms that a posted solution worked, is nothing more than a diagnostic to help get to the actual problem for an actual answer to solve, or similar):
- Flag as Not An Answer (NAA); if possible, comment to tell the user not to do that again.
- Is a misplaced question or edit (asking for more answers or ones that fit some slightly different situation or even a completely different situation):
- Flag as NAA and comment if possible.
- Is fundamentally inapplicable to the question (a LINQ answer on a Java question or a C++ answer on a Haskell question):
- Downvote. Flag as VLQ and perhaps comment. (In practice, because VLQ and NAA mostly feed into the same queue now, they're largely interchangeable, but this may not always be the case; the strictly correct thing to do with a post that's legitimately an answer but in the wrong place is to VLQ-flag it, not NAA.)
- Is link-only (consists in essence of nothing more than a URL of the actual answer, without paraphrase, amendment, or tips included):
- Check whether the question asks for only a link. If so,
- Flag the question as off-topic (asking for a offsite resource) — no matter how old it is — and
- Skip the review
- A signpost to the real answer is Not An Answer, and should be flagged accordingly and commented if possible. (Be careful: if the answer text still answers the question, the NAA flag does not apply. Check below for whether to downvote instead.)
- Links to tool downloads generally don't qualify as such unless they lack an explanation of how they apply, but may be spam; check more carefully than usual. Otherwise, you might comment that more information around the bare tool link would be helpful.
- Check whether the question asks for only a link. If so,
In FQ specifically, if you see a question that:
- Has no code but needs some (problem with a website that just links to the site itself without any code, vague description of an API call that's not behaving as expected, bare stack trace, etc):
- Flag to close as needs debugging details because it has no MCVE, then comment to suggest specifically what to include. Do your best to make sure the OP realizes that they should edit, not just reply to your comment.
- Asks which of several possibilities is the best:
- Flag to close as opinion-based. Usually, don't comment.
- Asks for a complete implementation of an entire program:
- Flag to close as too broad. Comment if practical to explain how much they're asking for.
- Asks for a good library for X, a tutorial for Y, or the like:
- Flag to close as off-topic because it's asking for off-site resource links. (Often these can be fixed to be a regular question, but that's part of the close/reopen process. If you can do it easily enough, go for it, but often it's more involved.)
- Isn't actually about programming:
- Flag to close as off-topic for whichever of the migrate reasons is most applicable: SU, SF, blatantly off-topic, or other site.
- Lacks any discernible question, as such:
- Flag to close as unclear what you're asking.
The subtle stuff
In any of the 3 queues, if you see a post that:
- Looks mostly OK, but has a link to a site that's not well-known, or that allows user content (like GitHub, Blogspot, YouTube, etc):
- Check the link carefully; if blatant spam:
- Flag as such. Don't comment.
- Otherwise, check the author's profile and see if they're self-promoting a lot or without disclaiming; if it's fairly plain:
- Flag as spam. Don't comment.
- Otherwise flag as needs moderator intervention and explain that it's subtle spam. Don't comment unless it's the sort of self-promotion that might be a legitimate mistake.
- Check the link carefully; if blatant spam:
In any of the 3 queues, if you see an answer that:
- Looks pretty good, maybe even a little too good, especially on a question with an accepted answer already:
- Open the answer link in a new tab and scroll up and down from the answer position to see if it's plagiarism (usually from the accepted answer); if so:
- Flag as needs moderator intervention, and explain what it's plagiarized from. Comment if it seems like it could be an honest mistake.
- Otherwise, check again for spam and handle accordingly.
- Open the answer link in a new tab and scroll up and down from the answer position to see if it's plagiarism (usually from the accepted answer); if so:
- Seems relatively trite and not particularly thorough, especially if there are a number of other answers and the question is not new:
- Open the answer link in a new tab and scroll up and down from the answer position to see if a substantially earlier answer already said everything this does; if so:
- Downvote; comment if practical.
- Open the answer link in a new tab and scroll up and down from the answer position to see if a substantially earlier answer already said everything this does; if so:
- Is wrong, generally a lousy hack, barely addresses the question, or otherwise well below par:
- Downvote; comment if practical.
In FQ, if you see a question that:
- Looks pretty common:
- Do a quick hunt for duplicates (0..2 keyword searches in the dupe-finder box) and flag if you find one. If you want, you can edit the auto-comment that it leaves.
- Reminds you of some question which you can't quite pin down:
- If you can think of a phrase XYZ that might help someone else find it, comment to mention that it seems like a duplicate of something like XYZ.
- Otherwise just Skip!
- Could be answered with a quick LMGTFY:
- Downvote. Optionally comment to explain the sort of search query that would help; do not link to LMGTFY or the like.
- Is only there because of a typo or other silly mistake that the OP would normally not have made:
- Flag to close as off-topic because it's just a typo. Comment to point out the issue.
- Is based on some utter misunderstanding of one or more fundamentals of programming or the general technology in use, such that explaining would require an indeterminate amount of personal tutoring:
- Flag to close as needs details or clarity and possibly downvote. Comment if you can keep your bafflement from spilling into rudeness.
- Doesn't quite have enough information to reliably answer:
- Flag to close as needs debugging details because it's lacking an MCVE and comment to explain what's needed.
- Is seriously messy in a way that you're not sure you, or indeed anyone, can really fix:
- Flag as VLQ. Comment if you have some specific critique.
- Seems to be a dead end, or asking about some bizarre and pointless corner case:
- Downvote if you're fairly sure no one will ever benefit from the answers to this question. Comment if practical to politely explain why this isn't worth asking.
Spit and polish
In FQ, if you see a question that:
- Has a screenshot linked because the OP doesn't have enough rep to post directly:
- Edit to bring it inline after checking to be sure it's not something horrible or irrelevant to the question, and fix anything else while you're at it. Do not use the "Edit" review action unless your edit results in an on-topic question which should remain open.
In any of the 3 queues, if you see a post that:
- Has something you can only partly fix or address (do not use the "Edit" review action for partial fixes):
- Edit or comment, then
- Skip! (You may be noticing a pattern here. Skipping is always a correct response. It may not be as useful as it could be, since someone still has to review it, but it's far better than giving a bogus review, of any sort.)
- Is poorly spelled or formatted or tagged or whatever, but in a way that you can fix yourself:
- Edit to fix it. Only use the the "Edit" review action if your edit results in an on-topic question which should remain open or an answer which answers the question and should remain undeleted on the site.
- Includes well-meaning noise (a signature, greetings, best wishes, worry that it's too late to post something, a note that the poster had the same problem [assuming the post also contains the solution they used; otherwise it's NAA, see above], a vague declaration that searches were made or days were spent tearing out hair, or similar).
- Edit to remove; fix other problems you spot at the same time. Don't comment, but do make sure to note in the edit summary. Do not use the "Edit" review action unless your edit results in an on-topic question which should remain open or an answer which answers the question and which should remain undeleted on the site.
- Has no (real) code, just a link to any of the numerous code-snippet sites:
- Edit to bring in the snippet if practical and if the snippet is licensed as CC By-SA 3.0; if they're just too long or unlicensed, comment accordingly. Only use the the "Edit" review action if your edit results in an on-topic question which should remain open or an answer which answers the question and should remain undeleted on the site.
- Could use some changes by the OP to make it better:
- Comment.
- Is quite good already (and isn't plagiarized or otherwise problematic):
- Upvote of course. Comment if there's something in particular to call attention to.
- Doesn't manage to meet even a single one of the above categories:
- Looks OK, at last. Remember: don't hit Looks OK if you should really be skipping! It's better to make someone else do the work than to wrongly mark it as OK.
There's a good deal of guidance around proper flagging for various categories, and you should look into that for a more complete understanding. But if you misflag occasionally, you'll generally get feedback, albeit rather tersely, and can then figure out what you got wrong. Ask here on Meta if you really get confused.
Up- and down-votes are a rather odd part of these queues, and strictly speaking are never necessary. However, given the opportunity, it can be handy at times to vote on posts you review.
You might notice after a while that the comments on a lot of these get pretty repetitive. If so, consider digging up a prefab comment list/ tool, perhaps one such as Nathan Tuggy's list here, but handle with care: dumping thoughtless autocomments where they only partly apply can be rude and not particularly helpful.
Audits
A final note: from time to time you will run into review audits. Some, you will fail, because audits are imperfect and so are you; even moderators fail audits. Most of the time this is not a problem, but there are a few known catches that can make you fail audits more often than is really needed.
Audits will fake post details.
An audit will not show the accurate score, or any comments, and may change the displayed username and reputation; answers will always display as though not accepted even if the original was, and questions will display as though asked very recently and hardly viewed even if they were asked weeks ago and had a thousand or more views. So if you're checking for plagiarism as you should, but aren't paying attention to which part of the page you landed on, you might look at the accepted answer and see that it looks just like the audit copy but has a different author. But if it was scrolled to when the page loaded, it's still the same exact post that you're reviewing — not a plagiarized copy.Audits select occasional lemons to throw at you.
The audit selection algorithms are fairly complex, but the gist is that posts have to be flagged, negatively scored, and deleted to be considered known-bad; and posts must have a good highish score with very few downvotes to be considered known-good. So far so good, but occasionally you find a post that was wrongly flagged, but then deleted for some other reason: perhaps the poster deleted it because it was wrong, or perhaps a moderator deleted it because it was redundant or plagiarized from who knows where.
The only real recourses are to just swallow the occasional bad audit (reporting especially bad cases here on Meta) or to open the post in a new tab first to see what happened to it. The latter works quite well once you get the hang of spotting audits, and the attention required overlaps fairly well with that needed to review accurately, so it's not really wasted effort.Some actions will auto-fail audits that probably shouldn't.
In particular, posts that have since been deleted in reality, if selected for audits, have in the past been impossible to flag. Don't think that means there's nothing to do. Looks OK is only applicable if the post is fine, not if it's been marked to be handled later. If you can't get Other Action to un-disable up by downvoting, flagging, or commenting, hit Skip instead.
This answer was revised and substantially expanded from the original Nathan Tuggy wrote, as the question did not seem entirely suitable for a FAQ.
-
19When a user first gets access to review queues, they should have a link to this guide presented to them. I've recently been suspended from reviews because of fundamentally understanding a few things that are explained VERY clearly here, but aren't intuitive.– thelrCommented Jun 1, 2020 at 20:42
-
1I had exaclty the same issue. Any chance this could pop-up as a link? Commented Oct 19, 2021 at 12:57