97

A while back after running into a pretty scary and incredibly insecure answer, I started to think about the inadequacy involved in notifying others. Then, after reading the post, Mod seems to have deleted all the answers to a supposed duplicate, it reignited the question: what can and should we be doing about dangerous answers?

Currently, the options for addressing dangerous and outdated posts are:

  • edit - usually by adding a notice or something of the sort, but, this can have blow back if/when post authors don't agree.
  • downvote - this is the most appropriate action but, often times it doesn't really carry enough weight, especially when posts are upvoted enough that newer users can't immediately see how controversial the post is (especially if they don't have highly upvoted explanatory comments).
  • comment - this is also appropriate but can lead to similar side effects as editing depending on the post author. Also, I've seen where these conversations get heated and nuked, leaving future visitors in the dark.

related post

The problem that I have with all of these actions is that they don't generally solve the full problem and they can cause messy blow back. So, I started trying to come up with a solution that could solve this problem in the most efficient way. Instead of trying to reinvent the wheel, I started researching Wikipedia's dispute model. In a nutshell, they use tagging to denote problems to the page author(s) and readers.

From wikipedia:

"Tags" are often used to indicate problems. [...] Tagging allows editors to specialize, teaches them and warns readers about subpar content. It is better if people solve the problems they encounter themselves, but not everyone may be able to. Editors are sometimes obliged to justify inclusion of tags, such as in the case of Template:POV.

* Emphasis mine. You can read more about this concept here. For full context, here is a screenshot of a handful of wikipedia's dispute templates:

wikipedia disputes example


I propose that a similar system be implemented for answers.

This system could be very similar to close voting, where a group of privileged users could vote on warnings that should be added to an answer. It should probably differ in a few ways as well:

  • This privilege might work best if it's only be available to gold (or silver?) tag badge users. Only knowledgeable contributors should be able to vote on these. My only concern with this limitation is low-traffic tags that might not have many tag badge users.
  • Warnings should be queued for removal on edits, similar to reopening questions. This is going to be tricky though, because an editor might not be addressing the warning(s), so possibly adding a checkbox per warning to these posts on the edit page like This edit addresses {{warning}} could control the queuing of warning removal. I can see how this will add a lot of complexity though, so if we can think of a better way, that'd be preferable.
  • It might be best to not show what warnings other users have voted for. This will lower the possibility of band wagon voting.

If this idea is applicable, then our next task will be coming up with warnings that most accurately cover dangerous answers without overlap.

6
  • 24
    "We need to find a way to gracefully deprecate answers that were once great but now get in the way, or are actively harmful... I'm still forming the idea, I'll be tossing it out on MSE today or tomorrow for discussion..." (SE Community Manager)
    – gnat
    Jul 9, 2015 at 17:20
  • 12
    Yeah, we're doing something like this. This is needed and I'll be posting a discussion about it on MSE at some point tomorrow, as more than a few sites are now old enough to need it. This was always a part of what we wanted the software to deal with, so we need to do it.
    – Tim Post
    Jul 9, 2015 at 17:39
  • 3
    I'm really glad to hear that @TimPost. I must admit that I'm really curious to hear about SE's vision on this :) Jul 9, 2015 at 17:52
  • Another closely related discussion: meta.stackoverflow.com/questions/292087/… I support doing something along these lines.
    – zwol
    Jul 11, 2015 at 13:53
  • Referenced in meta post Introducing Outdated Answers project (2021-02-18). Feb 19, 2021 at 13:42
  • "we're doing something like this." Looks like we're currently at 6 to 8 years without even so much as a status-planned tag. Mar 7 at 0:45

4 Answers 4

86

We need this, and it was part of the design criteria for the original site; that shiny new Stack Overflow that made its debut back in 2008. It's a problem for most of of the major tags, and it's now becoming a problem for many of the older sites in the network.

I'll be proposing v0.5 of this idea on MSE at some point tomorrow. What this involves is basically creating a new post type. We currently have:

  • A post (Wow, hey, I'm a post! (It's odd when you're named after an HTTP verb))
  • A post that was closed
  • A post that was migrated
  • A post that was deleted

This introduces a post that was deprecated, and probably a badge for the person that wrote it. It was great, it helped a ton of people, but stuff moves pretty quickly and now it needs to retire in favor of newer information.

The idea is, let folks vested in the tags it served decide that it's time for it to go to greener pastures, through voting. Put a special notice on it (as you suggested) and show it in a special way so the notice draws attention. Give it posterity, as programmers are still likely to encounter what it describes, and position it below the most recent answer that attracted the most votes. This allows it to continue to 'yield' to more current information.

It won't lose the accept mark (if it has one) unless the author of the question accepts something else. No rep is lost (well, not an issue since it's likely old enough for the rep to stick around anyway), the answer won't be portrayed as bad, but there will be clear signs to see more current information above it.

What I'm still working on is how to gate the privilege (tag badges are very likely), how we'll handle ageing of deprecation votes (if at all) and the badge details.

Please do not quote me on this as a plan for an exact implementation...

What bothers me is tags with such a low volume that there just aren't that many gold or silver badges for the tag. That complicates this from a performance perspective, so I have more noodling to do. It has to be open to 10 or 15k users in the absence of many tag badge holders.

But yes, we're doing this. Joel just reminded me that we haven't done it yet, so I'd like to get it done. Initially for answers, and we'll see how it goes.

24
  • Sounds interesting. How are insecure posts dealt with? Do they have a place in this model? Not all posts are dangerous due to deprecation (aging). Or is this just a wording quirk? I can wait to ask these question on your MSE post if you'd prefer :) Jul 9, 2015 at 18:02
  • 1
    @CarrieKendall You'll be deprecating the post itself, possibly because it's dangerous and highly voted, but doesn't advertise the use of functionality that is itself deprecated. The most common use would be 'everything this says is deprecated', but I'm looking at both cases in this. That's why the criteria is a bit tricky. I've got 9/10 of the problem solved, it's that last bit that gets mostly interesting, and I look forward to you co-noodling on MSE when I post it :)
    – Tim Post
    Jul 9, 2015 at 18:09
  • Ohhhhh, okay, I misunderstood. I have a few random ideas about low traffic tags but, I think I'll sit on those at least until your post. Jul 9, 2015 at 18:11
  • @CarrieKendall Anon feedback is also a great thing to sanity check with, if it's available and conclusive. As are down-votes in a specific window of (recent) time, as are flags in a specific window of recent time. Building it by just using natural signal from the site is pretty easy, making it hard to abuse makes it much harder.
    – Tim Post
    Jul 9, 2015 at 18:13
  • Yeah, based on the repetitive message Please don't use these to punish editors on the wikipedia page in my question, it seems that we have our work cut out. Making it hard to abuse but still usable is always tricky, though. Jul 9, 2015 at 18:25
  • An annotation with the ability to explain the deprecation and point readers at the more up-to-date information would be an important part of this functionality.
    – jscs
    Jul 9, 2015 at 18:33
  • 6
    Note to self: once Tim posts (there I said it) on MSE about this, ask him how the author of a deprecated post should respond if they are interested in updating their post with current information.
    – BoltClock
    Jul 9, 2015 at 18:38
  • 2
    @BoltClock Already thinking about a notification when a post reaches a threshold of votes for it, because (normally) it should be a somewhat-slow process. Then chat enters and makes me consider velocity too.
    – Tim Post
    Jul 9, 2015 at 18:56
  • This is very exciting. I'm looking forward to seeing your ideas.
    – Jeremy
    Jul 9, 2015 at 21:29
  • 4
    I don't like the implication here that the answer must have once been good. Whatever this ends up becoming, we're going to need a clear course of action for answers that were never a good idea, and that course of action needs to be more effective than throwing another downvote onto a +1000/-50 vote total. Jul 10, 2015 at 9:07
  • 1
    How to deal with potentially outdated answers has already been discussed a few times. If something is version-specific, so be it, maybe the answer should have a specific mention to indicate to which versions it applies. Such an answer can still be useful, even if it doesn't apply to the latest version, and shouldn't be penalised for it. That's generally very different from answers that have security flaws.
    – Bruno
    Jul 10, 2015 at 16:15
  • 2
    @TimPost regarding examples, this answer, before it was edited by a 3rd party to add a warning site, or this one. There seems to be a pattern on the ssl tag: some highly upvoted answers are also quite insecure: they're popular because they get rid of the certificate error messages, which are there for a reason.
    – Bruno
    Jul 10, 2015 at 16:32
  • 1
    Just wanted to say that I actually like the general idea of a tag based system as proposed by the OP better than the idea of described by Tim here. For one the tag based system is more generic and it also is able to handle subsequent edits more reasonably. Jul 11, 2015 at 8:52
  • 4
    @TimPost Did this ever go anywhere? I asked as this new request seems pretty similar. Aug 17, 2017 at 16:57
  • 1
    @KarlKnechtel It was a total non-starter internally; I could not get consensus on how it should work from a default view / sort standpoint. I tried repeatedly to revisit it and .. no luck.
    – Tim Post
    Mar 8 at 19:37
27

A few of the easy and noticeable warnings:

  • This answer is outdated. The content relies on outdated or deprecated techniques and should no longer be used.

  • This answer is insecure and should be used with great caution. It is dangerous for the following reason: {{CustomMessageFromVoters}}

Feel free to edit!

8
  • 2
    Shouldn't this be made a community wiki since you say "Feel free to edit!"?
    – Anonymous
    Jul 9, 2015 at 17:23
  • 1
    @Anonymous probably not. We don't want a long list of completely separate thoughts on this. Separate answers for or against would be better so they can be commented/acted on separately.
    – codeMagic
    Jul 9, 2015 at 17:33
  • 5
    We're going to do something like this Carrie - have you per chance seen this? What I'm working on right now is the criteria and the wording we show on those dearly deprecated answers. I'll be putting it out for discussion on MSE tomorrow at some point, since many sites are old enough to need this.
    – Tim Post
    Jul 9, 2015 at 17:35
  • @codeMagic Right. If they were providing an opinion in this answer, that would be bad, but this seems to just be for warning options.
    – Anonymous
    Jul 9, 2015 at 17:36
  • tl;dr - yeah, we need to get certain things that were once great out of the way, with posterity and thanks :)
    – Tim Post
    Jul 9, 2015 at 17:36
  • 1
    @TimPost I had not seen that. It looks like we're on the same page. Hah, sorry for posting this the day before you plan on addressing it Jul 9, 2015 at 17:48
  • 2
    @CarrieKendall <Phew> okay, I won't check my closets for evidence that someone has been living in them.
    – Tim Post
    Jul 9, 2015 at 18:21
  • 2
    I wouldn't treat insecure answers and so-called "outdated" answers in the same way. Whether something is outdated is in the eye of the beholder. Not everyone can use the latest version of every piece of software out there.
    – Bruno
    Jul 10, 2015 at 16:12
5

If there are "canned" warning messages for common types of potentially dangerous answers, they need to be customizable at least on a per-tag basis. For instance, the C family will want to have a warning along the lines of

The code in this answer provokes undefined behavior, which means, even though it may appear to work, a newer or different compiler might arbitrarily miscompile it

but many other programming languages have no such thing as 'undefined behavior', or if they do, its semantics are more like what C calls 'unspecified behavior' (which is far less troublesome).

2
  • I like this. I think it could be more generic since this will a common problem for many technologies, though. Jul 13, 2015 at 16:28
  • @CarrieKendall This is meant only as an example of a per-tag canned warning message. Right now I'm much more concerned with making sure the general functionality exists than with nailing down specific cases that need per-tag canned warnings.
    – zwol
    Jul 13, 2015 at 21:06
-18

Why not use the existing flagging pattern? I understand currently most (all?) of the VTD reasons end up in the answer being deleted, but it doesn't necessarily have to be that way.

Flag -> I am flagging to report this answer as... insecure|outdated -> more specific reason

Introducing a tagging system similar to Wikipedia is just one more... vertical, I suppose, in the editing workflow. It's dissimilar to markdown and doesn't follow any discernible HTML/XML format - the only two things users need to know to edit content.

As well, MediaWiki tags are usually to flag some specific piece of the content - not the content as a whole. Flags, on the other hand, do the latter.

Don't introduce new patterns where new patterns aren't needed.

11
  • 20
    Flags ask for mod intervention. Mods don't do content. So flagging is a waste of time.
    – bmargulies
    Jul 9, 2015 at 17:38
  • 1
    @bmargulies it doesn't have to be that way. Jul 9, 2015 at 17:39
  • 5
    Because we needed more ways to overload the term "flag"?
    – BoltClock
    Jul 9, 2015 at 17:45
  • 1
    @BoltClock how is this not a flag? How is this any different than the existing VTC flag system? One user shouldn't be able to make an ultimatum about the 'insecurity' of an answer. We've seen answers by people like the developer of iTerm that would completely mis/abuse a simple and under-developed system like a MediaWiki tagging system. Like most other moderation functions of the site, it should be a vote. And how do we start existing votes? With flags. Jul 9, 2015 at 17:48
  • 13
    In order for a moderator to appropriately respond to such flags, the moderator would need technical expertise in that area. But there are a gazillion topics on SO. So for that to be possible, we would need a gazillion moderators. I don't see how that would scale unless you "lower the bar" to include a certain subset of trusted users as well. (as is the case with the dupehammer)
    – Mysticial
    Jul 9, 2015 at 17:52
  • 2
    @Mysticial Again, you're assuming the new flags would be moderator flags. They could be entered into a review queue just like everything else. Or use the "trusted users" model, as you say, because it's been working quite well. Jul 9, 2015 at 17:53
  • Maybe I misread you, but your first comment (to me) implied that you wanted moderators to make the decision. (even though they currently don't)
    – Mysticial
    Jul 9, 2015 at 17:55
  • @Mysticial definitely misread me. I changed system->pattern. Maybe that will be a little clearer. Jul 9, 2015 at 17:56
  • A few comments: it's not a new pattern, it's heavily modeled on the close vote system. The biggest problem I have with this is that flagging isn't a privilege. I think for this to work, the people who can mark things as dangerous should have proven experience in the topic. Jul 9, 2015 at 17:59
  • @CarrieKendall then what is all the talk about {{Insecure}} tags like MediaWiki's? Jul 9, 2015 at 18:00
  • If you took from my question that I want it to work exactly like wikipedia, then I apologize for misleading you. I only used it as an example. I specified that I'd like it work like the cv system later in the question. Jul 9, 2015 at 18:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .