291

If you're reviewing low quality posts, I'd like you to read this. All of it. Not skim it, not just vote up/down with everyone else. In exchange, I'll keep it short. Or if you are too busy, here you go:

TL;DR: Review carefully. Don't delete everything. Deletion is for things that actually aren't answers, not for low-quality answers or "wrong" answers. Flag incorrect comments from review as "not constructive".

Recently there was a movement to "burn down" the low quality posts queue. This is a fine proposition, the queue is big and it does need to be reviewed.

I've been reviewing low quality posts in earnest for several months now, and here's what I've noticed: People tend to just delete everything. Sure, most posts in the queue are crap. I get that. And reviewers have been doing a good job of executing justice on these junk answers...in the same way that courts would if they just sentenced everyone to 10 years in jail unless they could really really prove they were innocent and happened to have some good treats for the jury.

Everyone reviewing LQP needs to review Shog9's awesome answer post. Right now. Go on, I'll wait.

I've shamelessly stolen Shog's image, and added some things to it. Specifically, what we should and shouldn't be deleting in review:

six things that might be apples, but aren't entirely: a nice apple, an orange, a mostly-eaten apple, a sign with a picture of an apple pointing somewhere else, a half-gone worm-eaten apple, and a green apple among red ones

What I'd like to focus on here is low quality and "wrong" answers. This is where I see the biggest issue.

  • Low quality answers. Yes, I know the queue is named "low quality posts", but not every minor problem needs to be deleted. Consider first if you can edit - or leave a comment for the author asking for more details, a better explanation, etc. Only if you can't plausibly imagine anyone putting in the work to fix the post should you opt to delete these kinds of answers.

    For example, an answer might not have description for why the code works, but it still shouldn't be deleted; just leave a comment asking the author for an explanation, and move on.

  • "Wrong" answers: This is what really gets me:

    a review-deleted answer consisting of, "Is the code to delete the body missing a check to ensure the body hasn't already been destroyed? That shouldn't cause your game to crash the first time the body is destroyed but I could see hardware/OS platforms behaving that way after that first destruction."

    This does attempt to answer the question. It may be plain wrong — but that's something for downvotes and comments to decide. Not deletion. This is why we have the voting system — if something is wrong, it should float to the bottom below all the other not-wrong things. This is the system working.

How you can help: I understand that it's fun to delete stuff. It's fun to feel like you're making a difference. But there's a better way:

There will always be people reviewing that do this. They leave auto comments from review on answers that don't really need them. When I review, I delete the things that need it — link-only answers and (real) non-answers. But more importantly, I click 'looks OK' on the things that shouldn't be deleted.

There's another problem here — that group of people that will always leave those comments there can't really be stopped. But we can remove them as we go:

flag those comments as "no longer needed"

I've asked before if this is what we should do. The answer then, and now, is a resounding yes. When you see a review comment that simple doesn't fit a post, please flag it for moderator attention with "not constructive." They're easy flags to review, and they make a difference in the site.

One possible solution to the incorrect review problem is to change the UX of the review interface - ManishEarth sums up the proposal quite nicely.

24
  • 34
    Yes, the auto-comments are really annoying. I feel that people use them just to use them, not because they fit a particular purpose.
    – user3920237
    Commented Mar 6, 2015 at 23:36
  • 2
    Great post. I was thinking exactly the same thing when reviewing LQP queue today - something like half of my votes (or even more) was "looks ok", which I find quite unusual for this queue.
    – BartoszKP
    Commented Mar 6, 2015 at 23:52
  • 7
    You need to address the SO users that vote VLQ inappropriately. That solves a lot more problems. Commented Mar 7, 2015 at 1:25
  • @HansPassant That'd be a much different post. We do have an issue with VLQ being used inappropriately on both questions & answers. When mods see it that's when we try to step in to adjust the behavior of specific users.
    – Taryn Mod
    Commented Mar 7, 2015 at 15:04
  • 123
    Maybe the "Looks OK" button should be renamed, then? It's not intuitive that we should click "Looks OK" for bad posts that we want to downvote.
    – ruakh
    Commented Mar 7, 2015 at 17:42
  • 6
  • 7
    Eh, VLQ's should be deleted. How do you fix a post comment/edit/downvote-ing that is totally garbage?
    – Braiam
    Commented Mar 8, 2015 at 4:16
  • 3
    @Braiam: Well, yes. But the problem here is with people's definition of VLQ.
    – BoltClock Mod
    Commented Mar 8, 2015 at 4:28
  • 8
    You think link-only answers are delete-worthy but not unexplained code? That seems like a double standard, to me.
    – TylerH
    Commented Mar 8, 2015 at 5:24
  • 15
    @TylerH: Link-only answers have no content whatsoever; they're essentially a waste of time for those who aren't willing to click on links for whatever reason, or once the links are broken. Unexplained code at least contains code that can be judged on its own should the reader be so charitable.
    – BoltClock Mod
    Commented Mar 8, 2015 at 6:17
  • 9
    @BoltClock Link only answers contain content in the form of a link to an answer. It's bad, but so is unexplained code. Both are VLQ.
    – TylerH
    Commented Mar 8, 2015 at 6:58
  • 1
    Good tip on flagging inappropriate Deletion Reason comments
    – APC
    Commented Nov 4, 2015 at 7:54
  • 1
    Related: Improve help text for the low-quality-posts queue
    – Wrzlprmft
    Commented Dec 10, 2015 at 19:04
  • 1
    Let's say I reviewed an answer and found it in category number (Picture # ) 5 or 6 i.e. low quality answer or a wrong answer. Let's say I put some comment and downvote it. After that I have to go past that review item to do my next review. What should I do at this stage? If I click "Looks OK" then that's not the case as the post is still not ok. It will become ok after the answerer makes some changes based on comment. If I press "Skip" then I loose my review count even though I did put my efforts to review the item and give an appropriate feedback.
    – RBT
    Commented Dec 15, 2016 at 1:46
  • 10
    @RBT "Looks OK" means "it shouldn't be deleted right now". It's poor wording, mostly.
    – Undo Mod
    Commented Dec 15, 2016 at 1:47

4 Answers 4

80

I agree with this in principle. If an answer is just flat-out wrong, that does not in and of itself qualify it for deletion. An incorrect answer does not make it low quality. There can be some very high-quality answers that are simply factually incorrect.

Plenty of answers exist in a grey area—answers that are low-quality but not blatantly irrelevant. All of the following should be deleted without question:

  • New questions posted as an answer.
  • Attempts to reply to existing answers.
  • Answers that are nothing but links with no explanation.
  • Answers that clearly have nothing at all to do with the question.

Nobody really disputes any of the above. The real problematic cases are the following:

  • Answers that could really be better-suited as comments.

    These are often short one-liners that technically attempt to answer the question, but are really nowhere near enough to actually help anyone solve the problem at hand.

  • Answers that might answer the question but don't have any explanation attached to them.

    Code-only answers are a common example of this, but they can also be answers that just say “do this thing” without explaining why or what that even means.

Let me address these one at a time.


Should answers that should be comments be deleted?

This might be somewhat controversial, but I'm of the opinion that yes, these should not be allowed as “answers” to questions on Stack Overflow.

Now, obviously, it's important to use your own judgement when considering answers like these. Sometimes, even though an answer could, theoretically, be a comment, it does answer the question, and that's okay. (Usually that's indicative of a poor question, but that does not usually warrant punishing the answer.)

See, this is an answer:

Is it a good answer? No. But it is an answer, and there's no reason to delete it.

On the other hand, this can be deleted:

What's the difference? In the former, the answer directly addresses the problem posed by the asker of the question. It's a poor question, but that's irrelevant. In the latter, the asker actually included a relatively specific problem as well as some code that requires a much more substantive answer. That answer should be a comment.

I think in these cases, what qualifies as an answer actually can be context-sensitive. I'm perfectly willing to click-through to the question to take a look at what's actually being asked.


Should answers with no explanation be deleted?

This one is a little less clear-cut.

This isn't helpful at all:

  • There is absolutely no explanation about what that code does, how it addresses the problem, or why it's the correct solution. It's not even really clear why that code is at all relevant to the question.
  • Furthermore, there already exist other answers that are far superior to that one, those that actually answer the question. Keeping that poor answer around isn't going to help anyone.

On the other hand, this one's actually okay:

Is this a stellar answer? No. Very much no. But it explains why the code posted is an answer to the question. Just by saying “try the null coalescing operator”, the answer is given enough context so that a specific piece of information can be extracted from the code snipped and applied elsewhere.

In the previous, poor-quality answer, there was no indicator how that answer could be used any where else because it wasn't clear what was the solution and what was just boilerplate code.

Similarly, this sucks, but it's still an answer:

Again, it's clear what's changed, and it's clear what the hell the answerer is actually attempting to convey.


Oh, and this is important...

Learn to love “No Comment Needed”.

See that? You can vote to delete questions without leaving comments! Especially if something should not blatantly be deleted, don't pollute the world by clicking one of the comment reasons just because you can.

Those comment reasons let you leave comments without having to think about typing out comments. They're tempting. But with great power comes great responsibility—please don't abuse the feature.


One more thing—that image Shog posted a while back? That was about Not An Answer flags. “Very Low Quality” and “Not An Answer” flags are two very different things. Just because something's technically an answer doesn't mean it should be kept around.

That said, the point you made in your question is quite sound, and I agree—more discretion is needed in the Low Quality Posts queue. I just don't believe that the absolute you proposed in your question is wholly accurate.

12
  • You might want to add answers which don't attempt to answer the question but instead offer lots of advice on side-issues (no reation to "XY-problem" or "Don't do it"). Or is that something you avoided on purpose? Commented Mar 7, 2015 at 0:27
  • 3
    @Deduplicator I personally find that those tend to fall into the "should be comments" group of answers. It's rare to find a well-written answer of that sort, and when I do, it's usually because the question was unclear to begin with. Commented Mar 7, 2015 at 0:29
  • 1
    I think it is worth pointing out the text of the very low quality flag: This answer has severe formatting or content problems. This answer is unlikely to be salvageable through editing, and might need to be removed. Basically absolute and total garbage. Personally I'm surprised we have the flag at all. None of the posts mentioned in this answer nor in the question above deserve that flag, so I feel it is rather a mute point..
    – Seth
    Commented Mar 7, 2015 at 0:30
  • 1
    @Seth That heavily depends on your definition of "severe" and "salvageable." I agree the description is pretty bad, but I think the flag is valuable for what it's currently used as. Commented Mar 7, 2015 at 0:32
  • @AlexisKing Which is?
    – Seth
    Commented Mar 7, 2015 at 0:33
  • @Seth Mostly, all of these. Answers that serve no value don't need to be kept around. Commented Mar 7, 2015 at 0:35
  • 1
    @AlexisKing Every single one of those qualifies as NAA..
    – Seth
    Commented Mar 7, 2015 at 0:41
  • 1
    @Seth In theory, yes. In practice, no. The VLQ has always been a little “squishier” in what's permitted as opposed to the NAA flag, mostly because there's a review queue for it. One could argue about how the flags are defined and their overlap extensively, but that's a conversation for another meta post. Commented Mar 7, 2015 at 0:46
  • 18
    @AlexisKing Both VLQ and NAA flags send posts to the same review queue, the Low Quality review. As a reviewer, one doesn't even get told which flag was raised.
    – user3717023
    Commented Mar 7, 2015 at 2:21
  • 2
    Re "no comment needed": for a long time I always left a comment because "You can't flag/vote to close without a reason, so surely I have to choose a reason when recommending deletion!" I have chosen "no comment" a few times now, but I always feel like if I can't fit it in one of the existing comments, I shouldn't be choosing to delete -- I feel I'm substituting my judgement about what deserves deletion for the community's. Commented Mar 7, 2015 at 16:04
  • 6
    @Jeffrey But as a matter of fact, delete votes are not categorized by reason - only close votes are. The comment templates are just that - some comments you may wish to add to the post. And the option 'no comment' is there on purpose.
    – user3717023
    Commented Mar 8, 2015 at 5:18
  • ... as is the option to add your own free-form comment, and then choosing delete and none needed, or keep. Commented Jan 11, 2016 at 17:22
31

The guidance you refer is helpful and authoritative, but please note that there is another guidance, no less helpful and no less authoritative that suggests a different approach, to dive deep, check thoroughly and cut mercilessly:

...answers that are strong candidates for deletion:

  • belong to low rep or anon users with no real commitment to the community

  • are provably duplicate, that is, were added well after (30+ mins later) other answers that contained the same exact information

  • are short in length

  • do not explain much of anything

"But wait!" - I hear you saying - "these two seem to contradict each other, which one I pick?"

Not really. These simply complement each other and are meant to be used in different context.

  • Guidance you refer lays out approach to evaluate delete-worthy posts quickly and safely and ensure that mistakes in doing this are minimized. If you are (are you?) a diamond moderator acting on one of (hundreds!) routine flags you get every day, you better stick with this guidance as firmly as you can - because if you don't, there is a real risk of accidentally deleting valuable content.

  • Another guidance is for the cases when one can (should) invest substantial effort into studying and curating reviewed content. If, again, you are a diamond moderator, this guidance applies to rare, "special cases" where you are expected to go beyond routine quick evaluation.

You see, guidance for diamonds moderators is essentially to act depending on how much time and effort they are expected to put into content review. I think that a regular user reviewing LQ posts can follow this approach as well.


Worth noting that at 20 LQ reviews a day limit, one has plenty time to do more thorough check of reviewed posts... Not to mention that there is no shame in using “Skip” if it feels too much work.

3
  • ...I personally prefer approach covered by quick and safe guidance - because, well, I prefer to spend less time on review (guess that makes me skip over many "known good" audits). But I can totally relate to those who prefer thoroughly curating content, especially in their favorite tags
    – gnat
    Commented Mar 12, 2015 at 21:03
  • 1
    The Jeff Atwood’s post you quote starts with: I think there are a general set of guidelines for question maintenance that should apply to any extremely popular question: … This shifts the meaning. I would certainly refrain from applying this advice on posts with one or two answers, where even a very incomplete answer can be useful.
    – Palec
    Commented Mar 12, 2015 at 23:19
  • 4
    @Palec this guidance was originally written for diamond moderators; they can give so much attention only to extremely limited amount of posts. This was long before LQ queue was introduced for tens thousands 2K users
    – gnat
    Commented Mar 13, 2015 at 5:10
20

I agree with this. One thing I want to add: When I come across a code-only answer in the queue, what I do is leave a comment like this (using AutoReviewComments), and click "Looks OK":

Could you please [edit] in an explanation of why this code answers the question? Code-only answers are discouraged, because they don't teach the solution.

This actually works. I have had numerous people who have then gone and edited their answer to include an explanation of the code!

0
-8

I'll say for myself why I am (ab)using the "low quality" queue instead of votes. Because downvotes take a toll on my reputation while the flags are "free".

The rationale is: my hard-earned rep doesn't deserve to suffer because someone posted crap. That's because, by Pareto principle, there's orders of magnitude more crap than good stuff, so I cannot possibly downvote everything that deserves it and retain a positive score.

I'm not saying this is the right thing to do. I'm saying that I'm working around a flaw in the system that would otherwise rip me off.

I can only speak for myself. But I do imply that most probably, I'm far from being the only one, and the situation that led to the current topic is a direct result. Remember that the vast majority of users with the privilege do not have so many rep points and steady income that they no longer care about a few (dozen).

24
  • 3
    This might deserve to have its own question - but it doesn't really belong in an answer here.
    – Undo Mod
    Commented Nov 23, 2015 at 22:06
  • 1
    You cannot downvote everything deserving it either way, there's a vote-limit. And just because an answer deserves a downvote doesn't mean deleting it is a good idea, people can learn trying to figure out why it is bad, which might even be spelled out in the comments. Anyway, that presupposes that your judgement is sound in every single case, which I doubt: Mine sure isn't. As undo already commented, removing the investment on downvoting is a different question and should be its own feature-request (probably a dupe, but not sure). Commented Nov 23, 2015 at 22:06
  • 17
    I wonder how many others are avoiding downvotes for this reason.
    – Kevin B
    Commented Nov 23, 2015 at 22:07
  • 3
    @Undo sure it does. But since this directly relates to the sutiation in the VLQ queue, this belongs here as well. Commented Nov 23, 2015 at 22:08
  • 2
    @KevinB I can only speak for myself. But I do imply that most probably, I'm far from being the only one. Commented Nov 23, 2015 at 22:09
  • 3
    reputation is meant to be spent. Um, well, one can spend it. Or you can get kicks out of looking at a number. But you can't buy cheeseburgers with it. Show decide what they are for, for you.
    – Drew
    Commented Nov 23, 2015 at 22:14
  • 2
    @Drew but I can buy "rep points" in the eyes of an employer, for example (and I did it once already). Of all things, rep is a testament of time and effort and some level of expertize and passion in the field. Commented Nov 23, 2015 at 22:20
  • 12
    Flags aren't "free", either. You have a limited number of them per day, and if they are misused, you can be banned from flagging. For example, using "very low quality" flags for things that should simply be downvoted and that don't require immediate deletion will most likely lead to those flags being declined. If enough of them are declined, you'll be banned from flagging. I recommend only using the right tools for the job.
    – Brad Larson Mod
    Commented Nov 23, 2015 at 22:23
  • 4
    I think This link to you is all you need. Those are probably some pretty good answers. And ya know what happens to 2/3 or more of downvote points lost? You get em back the next day when the atrocity was deleted. But no. Few downvote. Go figure. The community is left plodding through stuff that shouldn't be here.
    – Drew
    Commented Nov 23, 2015 at 22:23
  • 3
    @Drew "when the atrocity was deleted". So why not work towards deleting it in the first place rather than lose points and hope for the best? Commented Nov 23, 2015 at 22:26
  • 1
    And I am not "hoping for the best" even if it is not deleted. I do that for what I consider absolutely horrible Answers that need a message left. Why, because someone may waste a good deal of time going down that route. Why, cuz I feel like it and think when I do it is fair and balanced.
    – Drew
    Commented Nov 23, 2015 at 22:30
  • 5
    @Drew "reputation is meant to be spent" - oh, really? I thought is was meant to be accumulated to measure my contribution to the site and earn me privileges as acknowledgement for it. If my downvote makes the site better - why should I lose reputation for it? Commented Nov 24, 2015 at 20:55
  • 3
    @Drew 1) how do I get it back? Earn for other things? That's not getting it back, that's earning it anew. 2) Investment? Maybe the site wins from this, but I lose. The fact is, the system as well as ppl don't give a f**k about such things. All they see and judge by is the net number. So I strive to maintain it to further my goals while staying true to my values (which are "no cheap rep"). Commented Nov 24, 2015 at 21:18
  • 2
    Just use downvoting as a motivation for posting good answers. If you post a good answer for every downvote you make everyone's winning.
    – nico
    Commented Nov 26, 2015 at 0:03
  • 2
    @nico I consider this a good answer and downvotes as upvotes because it's designed to incur repulse, being so negative, soaked with opportunism and universal distrust. That's because that's what I feel: everything and everyone is against me, I'm all alone, no help is forthcoming, so I have to cheat, deceive, eel my way around this, survive in any way possible - since the system's designers knowingly made it hopeless for me the normal way. And the post's intention is to convey these feelings as strongly as possible. Commented Nov 26, 2015 at 5:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .