38

Spurred on by this question and subsequent discussion, I decided to do a bit of research into what would happen if we let the Roomba loose on poorly received answers.

The results are interesting, to say the least.

Negatively scored answers on Stack Overflow - aggregation by last edit date

The rule being followed for now: the answer has been edited in the last X days, where X is 30, 60, 90, 180, or 365 days. I can edit the query to incorporate other rules if we want to see that.

The main thing to call out is that I doubt we'd be losing any substantial answers with a move like this. Notice that after a year of no edits and a score of -3 or below, there are lots of answers (roughly 33K) that could be culled because they could be really poor.

Of course, this doesn't take into account the fact that the question may be on a low-traffic tag or that the question itself was particularly heinous (although that's even more impetus to make the question and answer pair go away), but it does offer something.

I'm interested in something like this since it seems that there's a bit of tension as to when we should use our delete votes, and whether or not using them on posts that really don't answer the question (but are attempts at an answer, as is a very heated topic in these parts) is a valid action. My gut tells me that the system should be able to cull crappy answers from the site instead of relying on a gang of 20K+ users who have a ton of free time on their hands.

I've also put together a follow-up query to answer some other questions:

  • What about accepted answers?
  • What about a breakout between upvotes and downvotes?

That query is here:

Negatively scored answers on Stack Overflow - upvote, downvote, and acceptance breakouts

Determining if an answer is "tightly" scored is entirely too tedious to do, and the key point we want to discover is the average between the upvotes and downvotes across all answers scored at a specific value. I've provided that, and the trend is what I'd expect: as the score total gets lower, the likelihood of the answer being upvoted increases, which may not have any ultimate bearing on its quality at all.

Also, the breakout on accepted answers indicates to me that the most likely scenario you'll see an answer with a negative score that's accepted is one that has a score of -1. The likelihood of occurrence is practically cut in half with any other scores. That said, the fact that an answer may be accepted doesn't factor in a lot, but I'm not sure that the script should be allowed to delete an answer that's accepted without really considering more edge cases.

Thoughts? Concerns? Issues with the query?

48
  • 39
    Would it be possible to see the break down between up/down? A score of -3 is very different if it's +0/-3 vs +44/-47.
    – Andy Mod
    Commented Feb 5, 2016 at 6:29
  • 2
    @Andy: I can work on that a bit later. That would be interesting to see.
    – Makoto
    Commented Feb 5, 2016 at 7:18
  • 14
    Showing how not to solve a problem, especially when juxtaposed against a correct solution, has value. Bonus points if it is cargo cult. Most visible when just about everybody gets their underwear in a bundle because a poster deletes his answer and scrubs its content. Legitimizing a bot to do what a post owner is not allowed to do is the most obvious problem with this proposal. Commented Feb 5, 2016 at 8:34
  • 2
    There is another big advantage to consistently cleaning up a lot of those loose ends: More people will downvote where appropriate, as they are more likely to recoup their investment in the short- to mid-term. Commented Feb 5, 2016 at 13:31
  • 6
    The flip side of that is we'd essentially be giving low-rep users 30 untraceable delete votes per day, @Deduplicator. Right now deletion (whether via review or via vote) is pretty traceable; you can see who is voting or reviewing and access to both is tightly controlled. Downvotes are anonymous, readily available, and should probably stay that way.
    – Shog9
    Commented Feb 5, 2016 at 19:29
  • 2
    Why, @Magisch? Is there really so much that isn't being deleted right now? Very few people with delete votes even use them. We're erring on the side of not giving people who haven't written much the ability to remove others' writing willy-nilly... I tend to think this makes sense.
    – Shog9
    Commented Feb 5, 2016 at 19:44
  • 3
    @Shog9 Thats because almost nobody has them. 10k is a big mark to get. I'd use all of them every day just like I do close votes if I had them. The problem is not that not enough people are able to delete/close, its that you guys heavily rate limit those who do have them and deter alot of people who enjoy moderation
    – Magisch
    Commented Feb 5, 2016 at 19:45
  • 6
    Whether or not you "farm rep" is your own choice, @Magisch. But you haven't put even 200 answers onto the site yet; barely 100 that've been upvoted. Heck, if you wanted more downvoted answers to be deleted, you could start with your own - and yet you're lamenting the lack of power over others' instead? Invest more of your own work into the system before you gripe that your share of ownership is too small.
    – Shog9
    Commented Feb 5, 2016 at 19:50
  • 2
    @Shog9 answers for me are more of a means to an end. If you checked, you probably saw my answers are mostly for easy c questions. Thats because im not a professional coder yet (im an apprentice). My enjoyment of this site comes mainly from keeping it clean.
    – Magisch
    Commented Feb 5, 2016 at 19:52
  • 7
    Trivia: the roughly 8800 users with 10K on Stack Overflow have cast roughly 31000 delete votes in the past 30 days. That's not quite 4 delete votes a month, and includes deletions of their own posts. You start out with 5 delete votes per day at 10K, and can earn as many as 30... There exists a tremendous potential here that isn't being used... Perhaps start by asking the folks who can delete why they aren't deleting the stuff you think should be gone @Magisch?
    – Shog9
    Commented Feb 5, 2016 at 19:55
  • 2
    @Shog9 Could be that some people who really enjoy answering questions don't enjoy moderating so much, and vice versa. Could also be because moderation is currently its own reward only (save a few badges) and answering comes with a shiny incrementing number.
    – Magisch
    Commented Feb 5, 2016 at 19:56
  • 3
    @Shog9 fair point, but I believe there is currently an imbalance tilted towards not enough moderation on So, not too much.
    – Magisch
    Commented Feb 5, 2016 at 19:57
  • 3
    You can believe whatever you want, but you might wanna start by earning the privilege to see deleted posts, to view the tools that list recently-deleted stuff, close stats, etc. Make informed observations, y'know. Anyway, this is probably quite distracting to Makoto, so if you wish to continue this conversation join me in The Tavern.
    – Shog9
    Commented Feb 5, 2016 at 19:59
  • 3
    You can sweep a dirt floor, @Tiny. You just stop when you hit packed dirt. Next time I'll choose a more familiar analogy.
    – Shog9
    Commented Feb 5, 2016 at 23:11
  • 4
    @Shog9 - have been down this rabbithole trying to get discussion on why high rep people do not cast downvotes EVER, much less DELETE votes. It is a waste of time. The # of people that can vs do is not going to change unless you can receive rep from casting downvotes some how.
    – user177800
    Commented Feb 6, 2016 at 15:09

5 Answers 5

58

I salute the idea. One possible concern is that it would kill "involuntary cautionary tale" answers, i.e. answers that present a prima facie plausible solution that would seem obvious to many, but which, upon closer inspection by other users, is highly inadvisable. Such answers typically end up heavily downvoted, often with a trail of cautionary comments. They then serve as a useful warning sign of what NOT to do. Vacuuming them could remove this useful information.

Typically there will also be a great answer that shows the right way to do it. But it's arguably useful to also point out the wrong way. Sure, you can see the good answer, but you might think, "No, I know, there's an easier way!" And then proceed to implement the bad solution you just came up with, unaware that there are reasons not to do this (and that you're not the first one to come up with this bad solution).

I'm posting this because this happened to me, and I was glad that someone else's mistake was pointed out and dissected, so that I wouldn't attempt to repeat it. I wish I had kept the bookmark. (Had I done so, I would have posted it here: Request for good examples of heavily downvoted answers that are worth keeping on the site)


Addendum: Some have argued that such answers shouldn't be left lying around, but rather rewritten as direct "don't do this because XYZ" answers, or merged into other, better answers, such that the valuable information they contain is better presented. That's a point worth discussing, but it is orthogonal to the question being discussed here i.e. it doesn't depend on auto-deletion. Also note that once the original answer is auto-deleted, it's harder to do those things; and in the meantime most users can't see the cautionary remarks.

17
  • 8
    Do you have an example of this? I can't imagine an answer that has a cautionary tale which has been downvoted in the extreme...
    – Makoto
    Commented Feb 5, 2016 at 8:38
  • 4
    What is the point in showing someone what not to do when there are lots of information (in most of the cases) on what is best to do...?
    – T J
    Commented Feb 5, 2016 at 8:40
  • 16
    I can't think of a heavily downvoted answer (with no voting irregularities) that would be worth keeping on this site tbh.
    – Magisch
    Commented Feb 5, 2016 at 8:42
  • 5
    @TJ: Because sometimes you can see the suggestion of what to do, but think, "No, I know, there's an easier way!" And then proceed to implement the bad solution you just came up with, unaware that there are reasons not to do this (because that involuntary cautionary tale answer got deleted). Commented Feb 5, 2016 at 9:01
  • 4
    @Jean-FrançoisCorbett You know, I end up searching for something in stackoverflow when I have no such thing as "an easier way" in my mind. If I had one, I already screwed up with that and that is why I'm here looking for better answers, not worse ones. I come here trying to solve the problem with what I thought was the easier way. I already have a problem. You seem to be targeting those who never try anything but starts their job copy pasting from stackoverflow..? Do we really need to keep crap in favor of them..?
    – T J
    Commented Feb 5, 2016 at 9:05
  • 1
    @Makoto: I've seen this every now and then, but I haven't bookmarked any, no. Also, I clarified what I meant by "cautionary tale answer". I don't mean an answer that says: "Here's an apparent solution, but don't do this, it's dangerous." I mean an answer that says: "Here's a solution, do this!" with many comments that then say "No! Don't do that! Because XYZ." Commented Feb 5, 2016 at 9:06
  • 1
    @TJ: I'm not talking about "crap". I'm talking about seemingly plausible solutions, with subsequent valuable warnings / information why not to use them. I'm not targeting silly users; I'm targeting people like my past self. Commented Feb 5, 2016 at 9:10
  • 2
    @Jean-FrançoisCorbett I don't really know what "seemingly plausible solutions, with subsequent valuable warnings / information why not to use them" means. My understanding of it is: The thing that you shouldn't do. As per my understanding, a developer visits a question for 2 reasons: 1) he has no clue how to go about doing something. Solution: We show him the right way to do it. 2) He screwed up with what seemed plausible solutions Solution: We show him the right way to do it. The key thing to note is: If I have a seemingly solution thingy... In most cases I give it a try first.
    – T J
    Commented Feb 5, 2016 at 9:18
  • 1
    I can't think of a situation when I google like "Will there be any issues if I try X" (my seemingly plausible solution). In case if I do, and if there is a question and an answer for that in stackoverflow - The answer to that will be a good positive answer explaining the issues with my approach and alternative solutions. How come a horribly downvoted answer is going to help me there..? If it was good for something, it wouldn't be in that state
    – T J
    Commented Feb 5, 2016 at 9:18
  • 3
    So you, @TJ, have never needed this personally. Good for you. I'm just posting this because something like this has helped me in the past, and I'm assuming I'm not the only one. But maybe I'm wrong. Commented Feb 5, 2016 at 9:23
  • 1
    @Jean-FrançoisCorbett This isn't about you or me, This is my last attempt at making you understand why what you're saying doesn't make sense to me. Scenario 1) Someone is looking for the best way to do "X". We show him solutions that are highly accepted by the community, everyone is happy. Scenario 2) someone is having an issue with x, or not sure whether his approach "x" is best, in which case he searches "is 'x'the best way to do 'y'". The point is, the solution that explains why x is not a good way to do y is going to have a good feedback from the community.
    – T J
    Commented Feb 5, 2016 at 9:28
  • 4
    This is probably why Magisch said he can't think of such a downvoted answer. If it is downvoted it's most probably posted in the wrong place (as an answer to "what is the best way to do x"). There is no point in presenting the crappy ways to someone who's looking for the best way.
    – T J
    Commented Feb 5, 2016 at 9:31
  • 4
    This is the classic use case, but I have to admit... leaving "never ever do this" to the subtleties of -10 score with, of all things, explanatory comments lingering around seems a little bit silly. It's worth considering writing the cautionary tale up as such in one or more of the other answers. Whether the downvoted answer should be auto-deleted, deleted routinely by humans, or what, I'm not sure... but given that we currently incentivize self-deletion, it seems like deleting more is consonant with that. Commented Feb 5, 2016 at 20:32
  • 1
    This is a great point. When starting a project with a new technology, I often follow "related question" trails to get a deeper understanding of what I'm likely to mess up.
    – Josiah
    Commented Feb 6, 2016 at 15:39
  • 1
    Every time I saw an incorrect answer (no need to invent new terms), its gist could be non-taxingly incorporated into the correct one like "way X is flawed: Y". And I never saw heavily downvoted answers at all - probably because they are more likely to be deleted as VLQ. Commented Feb 7, 2016 at 2:03
50

Automatic deletion for questions works because there are multiple signals that can be used to determine abandoned questions... One of the best being the presence of answers. If no one could find a way to provide a helpful answer to a question, then there's a really good bet it isn't a useful question; votes (down or close) help to reinforce that.

Answers don't have that. Answers are much easier to get rid of without automation though - votes from high-rep users and flags from everyone both lead to speedy deletion in cases where that's appropriate.

I think leaving answer deletion in the hands of actual humans is a good idea.

15
  • 1
    This still leaves two open questions: how can an answer be considered helpful if it's been downvoted, and what does this imply for the deletion privilege - that we're really meant to be using our delete powers not just for dangerous content, but also low quality content?
    – Makoto
    Commented Feb 5, 2016 at 18:51
  • Is there ever a case where an answer can be considered useful if it is heavily downvoted?
    – user4639281
    Commented Feb 5, 2016 at 18:52
  • @Makoto on sites where I have 20k (only AU) I vtd lossy answers to lossy questions all the time, mainly to allow roomba do its job.
    – Braiam
    Commented Feb 5, 2016 at 18:53
  • Yes, @Tiny. I think Jean sums up this scenario well
    – Shog9
    Commented Feb 5, 2016 at 18:53
  • 3
    Well, while that is an interesting sentiment, I haven't actually seen an example that is worth keeping around. If anything the answers usually end us as "Don't do it this way:", in which case they are usually upvoted to draw attention to something you shouldn't do. I would love to see actual examples of heavily downvoted content that is worth keeping on the site, which couldn't possibly exist in the form of "Don't do it this way".
    – user4639281
    Commented Feb 5, 2016 at 18:55
  • 1
    Well, post a discussion asking for it then @Tiny. I mean, if you think this is a good idea, then do your research first. I'm sure I'm not the only one who has come across usefully-wrong answers before, but I don't exactly keep a list of them - so put up a request & let folks help you as they're able.
    – Shog9
    Commented Feb 5, 2016 at 18:58
  • 3
    @TinyGiant I'm going to take a guess that some of the downvoted posts that might need to stick around would be obsolete answers as technology evolved. They might have been correct at one point, but now they maybe be wrong for new tech. These need to stick around for people stuck with older technology.
    – Taryn StaffMod
    Commented Feb 5, 2016 at 19:02
  • @Shog9 Like this?
    – user4639281
    Commented Feb 5, 2016 at 19:14
  • 2
    @bluefeet I agree that there are downvoted answers that are useful as "what to avoid" examples. However, I've not found the community to generally downvote obsolete answers, especially if someone edits them with version information. I'm not saying downvotes on obsolete answers don't happen but I don't recall ever running into an answer that got into the negative merely because it had been made obsolete. Ironically, I do have an answer that was downvoted for being obsolete. The downvoter commented with his reason and made it clear that the downvote was his. I just don't think it is common.
    – Louis
    Commented Feb 5, 2016 at 19:18
  • @bluefeet That sounds like it should be it's own answer, as it is a different situation than the one Jean pointed out, to which Shog was referring. Again, I would love actual. examples
    – user4639281
    Commented Feb 5, 2016 at 19:18
  • @Louis It's probably not common, but it does happen. I did a lot of research into "obsolete"/negatively scored answers for this and this. The problem is, it can be very difficult to identify these by querying the DB. I've looked at voting history - more DV's than UpV's over the 2nd half of the life of the answer, etc. It's tough to find them, but they exist.
    – Taryn StaffMod
    Commented Feb 5, 2016 at 19:24
  • @bluefeet Er... I know they exist. As I said, I do have one answer that was downvoted for being obsolete. It still has a positive score though.
    – Louis
    Commented Feb 5, 2016 at 19:28
  • 2
    @TinyGiant This answer is a good example of a mistake that people would be likely to make. However, it so happens that the upvoted answer on that question does mention the very issue that the bad answer runs into so the answer does not need to stay. Ideally, it seems to me that's what should happen: if someone posts a bad solution illustrating a mistake people are likely to make, then an answer that presents a good solution should also explain why the bad answer is bad.
    – Louis
    Commented Feb 5, 2016 at 19:29
  • @Louis could you post that over here?
    – user4639281
    Commented Feb 5, 2016 at 19:32
  • 2
    That's an interesting example, @Louis, because until recently there was another answer to that question which described the same technique and in fact it was this answer which motivated the correct answer... IOW, it's possible that the removal of the previous (incorrect) answer resulted in the posting of a new one. This is exactly the sort of answer I'd like to see remain as a warning to folks that their "clever solution" is neither novel nor a good idea.
    – Shog9
    Commented Feb 5, 2016 at 19:34
26

I, like Shog, don't think this is a good idea. It's possible that negatively-scored answers are helpful to users.

I looked extensively at obsolete answers and negatively scored accepted answers to see if there was something that could be improved in how we handle them. Many of these negatively scored answers, may have been great once when the technology was brand new (5+ years ago). But now, the technology has evolved and the answer isn't good for the current tech, so it gets downvoted. These answers exist on the site and are helpful to those stuck using old tech.

Answers specific to aren't wrong for those stuck still using it, but the answer isn't very helpful to someone using . We don't want to delete an answer that might be negatively scored because it's possible someone still needs it.

I'd prefer leaving this to users to vote to delete, so there is some sanity check on the answer before leaving it to a bot to delete based on score.

5
  • 1
    I'm fine with this sentiment, but I feel that there may be things that we could do to suss out and identify answers based in old tech and answers which are completely terrible. Right now it does feel like there's no clear answer on if a 20K user should spend their delete votes on answers which are terrible in addition to deleting content which is harmful.
    – Makoto
    Commented Feb 5, 2016 at 19:42
  • @Makoto I've tried/am still trying to figure out ways to identify them. I've got some ideas but nothing has come to fruition yet... I am still looking and trying to get buy-in from others.
    – Taryn StaffMod
    Commented Feb 5, 2016 at 19:45
  • 7
    Well, there's lots of discussion and feature-requests around ways to mark outdated answers as outdated / best prior version x.y . Current options: Downvoting (likely ineffective due to inertia, and actively harmful if old version still in use), editing (probably rejected as against authors intent, whether it updated the post to the new best or just added a warning about applicable versions), writing a new answer (see inertia), closing the old question as a dupe of a new one to get rid of the old answers (very shady, and reversed fast), and of course the classic one, despair. Any others? Commented Feb 5, 2016 at 20:01
  • 3
    @Deduplicator Lots of things I've contemplated and tested internally during the months that I've looked at this problem. Why do you think my feet turned blue?
    – Taryn StaffMod
    Commented Feb 5, 2016 at 20:08
  • I also have an answer, a right answer of course! :), which has +7,-9. I like to watch the fight between good and evil over the years... At the moment evil leads by -2
    – hek2mgl
    Commented Feb 6, 2016 at 5:33
11

It's not clear to me that this is a problem in need of a solution.

We already (by default) sort answers by vote/acceptance. So if a highly negatively voted answer exists alongside positive answers, most people won't even see it.

Using the Roomba to delete old and worthless questions makes sense, because questions represent search targets, particularly from off-site. Using the Roomba to delete answers doesn't seem to have the same effect. It can affect in-site searches, as they do search through answers. But a highly downvoted answer doesn't appear high up in an in-site search very often anyway.

Culling old questions is helpful to the site. I don't see how culling bad answers is helpful.

2
  • 1
    Answers are search targets, too. I don't disagree that answers can fall out of favor due to their age, but what of patently incorrect answers? Should we really have to see them on the answer page? What of users with sufficient rep like yourself and myself - should we be culling them since we have that power? These are the things that this sort of feature is trying to suss out.
    – Makoto
    Commented Feb 6, 2016 at 20:36
  • 2
    @Makoto: "Should we really have to see them on the answer page?" Don't scroll down that far. "should we be culling them since we have that power?" If you have reviewed the answer and decided that it should go, then do so. Otherwise, no. Commented Feb 6, 2016 at 20:48
-1

There's a beef I have with this... It is that there's no way to audit these deletions. No way to redeem yourself or to inform the user what he could do to improve their answer. I don't think an automatic process can determine if an answer can be accurately and silently deleted. There should be some friction to get this done in a sane way.

7
  • Note that I never specified a specific time or score cut-off - I was more or less looking in the "months to years" category to avoid scenarios that you describe.
    – Makoto
    Commented Feb 5, 2016 at 18:53
  • 5
    This objection applies to everything the roomba does. What makes this proposal different?
    – jscs
    Commented Feb 5, 2016 at 19:36
  • @JoshCaswell err... no. See Shog's answer. Roomba uses many indicators, in this case it would be just score + age.
    – Braiam
    Commented Feb 5, 2016 at 19:38
  • I can always add more indicators. The first one was a rough indicator to gauge reaction and perception. I can clearly see that age alone is a bad choice.
    – Makoto
    Commented Feb 5, 2016 at 19:39
  • 5
    Shog's objection is specific to this proposal; your objection applies to everything the roomba does. There's no explicit audits of roomba question deletion; there's no way to "redeem" or "inform" for those deletions.
    – jscs
    Commented Feb 5, 2016 at 19:40
  • 1
    @JoshCaswell no. I don't have issues with what roomba does. It does what it does for a good reason: questions without answers or any indicator that someone cares are frustrating. If you see a question without upvotes nor answer for more than a year, people that arrive there because they have the same problem (or, the question is just misrepresented) will run into a brick wall. Deleting the question on those cases so it gets re-asked, with all that entails (looks fresh, it targets more eyes, etc.) is good. Deleting an answer because it isn't hot anymore isn't.
    – Braiam
    Commented Feb 5, 2016 at 19:44
  • 1
    You should edit the material from your comment into your answer, then.
    – jscs
    Commented Feb 5, 2016 at 19:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .