423

In our blog post last month on our Community & Public Platform strategy & roadmap for Q1 2021, we announced an initiative to address outdated answers on Stack Overflow. Today, I'll give you more information on our approach and how we will solicit your feedback.

Why outdated answers? Why now?

Finding correct and complete answers is the reason users visit Stack Overflow. The vast majority of our users come from organic search by googling their issues and clicking to Stack Overflow to find answers. On top of that, there is a large and passionate volunteer community — I'm looking at you, Meta — that cares deeply about ensuring that the question-and-answer artifacts on the site are of the highest quality.

Yet, we have heard through a variety of feedback mechanisms — from Meta to internal staff — that outdated answers are an issue. According to our Site Satisfaction survey, 13.8% of users say that answer quality is one of the top things that they find most frustrating/unappealing about the site.

If Stack Overflow users can't quickly find working solutions to their problems, this is kind of an existential crisis. Good answers to questions is fundamentally the value that we provide to the developer community.

So the why is pretty obvious. But why now vs. why not yesterday vs. why not eight years ago?

The first thing we did to kick off the project was read through Meta. It quickly became apparent that this is not a new issue. There are many discussions, going back many years, with many thoughtful suggestions about potential solutions. (If you're curious, you can find a subset of relevant Meta posts at the bottom of this discussion.)

I can't talk about why we didn't prioritize it then because I wasn't on the team — but I can speculate. It's a hard problem, and hard problems are easy targets for procrastination. And the urgency of solving it for a two-year-old site or a six-year-old site is different from the urgency of solving it for a 12-year-old site. And, frankly, for a time, Stack Overflow and the rest of the Stack Exchange sites were under-resourced from a product-development perspective.

But the past is the past. This year, we decided that it's time. Stack Overflow's senior leadership team laid out our 2021 objective to measure and improve overall community and content health. And the Public Platform team decided that one of the best ways to meet that objective is to focus on keeping content as accurate and up-to-date as possible.

We're starting with product discovery — and listening to you

Rather than jumping straight into implementing some of the solutions you've proposed, we are focusing initially on product discovery. We want to make sure we:

  • understand the problem really well, so we can build the right solutions.

  • attempt to quantify the problem using a data-driven approach.

To do this, we will be reaching out on Meta during discovery to let you know what we're working on and to ask for your feedback. No one knows the problem better than you, so we are all ears.

When we get to the solution phase, we will conduct UX research, share early designs, keep you abreast of experiments and let you know what's launching when.

How do we define outdated content?

Our first step will be identifying, with your help, what constitutes an outdated answer and what type of outdated answer is most prevalent on our site. So far, Meta has told us there are different flavors of outdated answers. There are answers that:

  • become obsolete as new versions of frameworks become available.

  • still work, but there is now a newer, better way to achieve the same end.

  • maybe were never the best, but the question askers accepted them and forever bestowed the green check mark.

  • now pose security risks or provide code that no longer works.

  • still hold value to a subset of developers who use legacy technology, but aren't valuable to developers on newer technologies.

We will be gathering this information by adding a few questions to our site satisfaction survey. Check out this post where we ask for your input on the proposed questions.

How will we surface outdated content?

Identifying stale or outdated content is a hard problem to solve, and we know there isn't a silver-bullet solution here. It will likely require a mix of automation and user intervention. But we hope to identify some heuristics that make it easier to narrow down the universe of possibly outdated answers.

We're in early discovery, so we don't have a solution defined yet on how we'll surface outdated content. However, once we have a defined list of use cases, we want to explore ways in which we can get help from the community to label content as outdated so that we can begin training a machine-learning model as a complement to manually flagging content.

There is a lot of nuance to each answer and the amount of expertise required to evaluate answers for any given tag or technology. Leveraging the community's collective knowledge here will help us get the best data set we possibly can.

As with any training exercise, failure is always a possible outcome. Either way, we'll share our learnings along the way.

We are tackling Stack Overflow first

We are focusing our initial research efforts on Stack Overflow, and not on other sites on the Stack Exchange network. We understand that the issue with outdated answers is not exclusive to Stack Overflow and other technical sites may have similar issues.

However, none of the other sites operate at the scale of Stack Overflow: more than 31 million answers, compared to 11 million for the rest of the network. And Stack Overflow is our oldest site, so it is the site where the pain is most acute.

Rather than trying to shoot for a one-size-fits-all solution that may actually fit none, we are laser-focused on uncovering the Stack Overflow use cases and coming up with initial solutions that work really well for Stack Overflow.

Stay tuned for other activity TBD

We are currently mapping out other discovery tasks we will undertake in the near future, and we will reach out to Meta when we have new updates and requests. Our future posts will have the , , and tags.

If you want to be considered for targeted surveys, usability tests and focus groups, please visit your Email Settings and opt-in to Research. This ensures that you are in the pool of people we contact when conducting UX research.

Research opt-in picture

Meta literature review

There are many posts on the topic of outdated answers, but here are some of the ones we discovered while doing our initial research.

Any initial feedback?

We're not locking in on any particular solutions yet, as we are more focused on understanding the what and why vs. jumping straight into the how. But we welcome any initial thoughts and reactions regarding the project overall, past experiences, or ideas you have about ways to attack this problem as answers below.

51
  • 220
    Nothing is ever obsolete, there's always someone out there working with tech that's old because they have to... I look forward to seeing what potential solutions you come up with that can get the right answers to the right people Commented Feb 18, 2021 at 16:42
  • 58
    Looks like some good research was done here, nice to see! Here's another discussion for the pile - arguably one of the first BIG discussions on the matter: meta.stackexchange.com/questions/103053/…
    – Shog9
    Commented Feb 18, 2021 at 16:57
  • 46
    Any initial feedback? Yes: Wow! This is gonna be a Herculean task, as you clearly understand, but it's absolutely fantastic that S.E. Inc. is embracing the problem. Can I create some sock-puppet accounts, so I can upvote this post some more? Commented Feb 18, 2021 at 17:06
  • 51
    Love the tone and style of this communication :-) Thank you.
    – QHarr
    Commented Feb 18, 2021 at 22:39
  • 37
    @Nick: I think you're conflating "obsolete tech" with "obsolete answers". Answers pointing out the lack of a solution to a problem can be rendered obsolete by the introduction of new tech that either purposefully or incidentally solves the problem. Of course, the original content would remain relevant to legacy environments that a reader might rely on. But they'd still need to be updated with the new information.
    – BoltClock Mod
    Commented Feb 19, 2021 at 6:08
  • 13
    My instinct is that the machine learning approach is unlikely to succeed, there's surely far too much variation in what defines an obsolete answer to get anything like a useful set of training data? Hopefully the machine learning bit will be a nice-to-have stretch goal, rather than something that serious time is invested in, before quicker wins are accomplished.
    – DPWork
    Commented Feb 19, 2021 at 8:10
  • 9
    I always liked the sorting by age-weighted votes a lot but it was never implemented. In the simplest variant one could for example just sort by the votes in the last X months. I really hope it may get a chance this time. Commented Feb 19, 2021 at 9:09
  • 24
    @BoltClock "Answers pointing out the lack of a solution to a problem can be rendered obsolete by the introduction of new tech" - That only makes the answer obsolete to people who have access to said tech, the answer is still relevant to those who do not Commented Feb 19, 2021 at 11:37
  • 8
    If quickly finding solutions to problems is the challenge, I wonder if the search functionality of SO should also be part of the discussion? Or is the assumption that people use google and arrive at the right questions and then only need to find the right answer within the right Q&A pair? Commented Feb 19, 2021 at 12:02
  • 12
    I am absolutely stoked about how you're tackling this task. That was such a great write-up, and it's really nice that you're coming to the community early in the process. Honestly it feels to me like you've thought of everything I could think of suggesting. Even your category break-downs are excellent. Thank you. Commented Feb 20, 2021 at 7:28
  • 21
    "...there is a large and passionate volunteer community — I'm looking at you, Meta — that cares deeply about ensuring that the question-and-answer artifacts on the site are of the highest quality." I just want to point out that the large and passionate volunteer community does not 100% intersect with the Meta community. I come to Meta when I see an interesting Hot Post or whatever on the side bar, but I rarely participate more than an occasional comment or vote. However, I'm very committed to the quality and purpose of main. Please make sure you reach out in other ways besides Meta posts.
    – MattDMo
    Commented Feb 21, 2021 at 20:30
  • 9
    Whatever happened to the "get rid of crappy questions project"?
    – mxmissile
    Commented Feb 23, 2021 at 15:38
  • 22
    Could we rename outdated to legacy. Many of the answers which reference old versions of software are very handy for maintenance of legacy systems. It is not uncommon that systems run on legacy software or have been written in it and have been designed 20+ years ago. These "outdate" answers are not outdated, they are extremely relevant for such a systems. Legacy might be a better name here.
    – kvantour
    Commented Feb 24, 2021 at 6:56
  • 9
    @kvantour Anita is already clearly differentiating legacy answers from other kinds of outdated answers, in the section How do we define outdated content? Yes, we want to keep those (useful) legacy answers, but we want them to be easier for people using legacy systems to find, but also we don't want those answers clogging up searches for the majority of people who are looking for newer answers. Of course, what counts as legacy (and how best to deal with it) varies, depending on the language / framework.
    – PM 2Ring
    Commented Feb 24, 2021 at 11:53
  • 9
    @gmoniava "...would be unfair to get downvotes to answers that were once correct." Why? Votes just means something is useful or not useful now. The past should play no role there. Voting is all about signalling to future visitors. Commented Feb 25, 2021 at 22:58

37 Answers 37

264

I can think one big request on this topic

Please, please, please don't mass delete anything!

Many times I've found in outdated questions, comments (in particular!!) a lot of information, ideas and guesses lead me to the answer.

So don't assume an outdated question for JBoss 3.1.0 is not important for someone looking for a solution for WildFly 21 (as an example). Many things can change, but technology evolves from the past and having a view on the "outdated past" can be helpful, in particular for nontrivial questions aka "hard ones".

6
  • 32
    I don't think anyone suggested deleting; but it's definitely important enough to underline it. Ideally, the goal would be sorting/filtering: the ability to present more recent versions first by default, or filtering to only show the answers for the most recent version by default, and allow users to see other versions as opt-in. Commented Feb 19, 2021 at 15:19
  • 8
    I agree, my comment is to underline the concept: don't delete or hide by default. Older answers can be useful even today. An idea could be to "tag by version" so an example an answer can be right for Java 4, one for Java 7, one for Java 56 as example, so anyone can browse by version ? (I if applicable of course )
    – Vokail
    Commented Feb 20, 2021 at 21:37
  • 17
    Thank you for posting this answer, otherwise I would have. The outdated answers tend to be the most valuable to me, as apposed to all the young whippersnappers who apparently get to work exclusively with only the latest and greatest. Legacy defines my working life.
    – Gimby
    Commented Feb 22, 2021 at 9:18
  • 1
    This decision seemed to have been made quite some time ago, to delete things in order to "clean". Have to have things tidy, right? lol... We absolutely should not be mass deleting unless the content warrants deletion for the defined reasons of being inappropriate (notably outdated is not in that reason set).
    – Travis J
    Commented Mar 2, 2021 at 19:41
  • 5
    +1 for mentioning the value of comments. In an ideal world, someone would have found time to edit everything valuable from comments into that answer or another. In the real world, that's not true, and deleting or moving comments to chat loses real value, especially when a couple useful comments had a lot of upvotes so they stood out. We're all capable of clicking "show all comments" and looking at the end of the chain to find recent comments on old popular answers, so lets do that while poking around important old answers that may merit some updates. Commented Mar 3, 2021 at 10:11
  • 5
    Future SE codebase: function isOutdatedAnswerForQuestion(question, answer) { /* A bunch of super complex logic */; if (question.tags.includes('JBoss') && answer.text.includes('WildFly')) { return false } else { return true } } Commented Mar 10, 2021 at 18:19
140

This is a positive thing; you're coming to Meta and doing your homework and making an honest effort to get this project kicked off on the right foot.

I will say though that your initial research material has a common theme here, and I feel that there's some low-hanging fruit that could be taken as a first iteration to at least reduce the impact of outdated or outmoded answers still appearing first in the ranking.

First iteration could be as simple as unpinning the accepted answer (again, as per the overwhelming research material there) and gathering statistics on how that impacts if users are finding those kinds of outdated answers.

36
  • 59
    Another way of approaching this would be to ignore the accepted answer status when sorting answers for display. That is all answers are sorted by their vote, if an accepted answer has fewer votes than another answer then it is placed below that answer. Commented Feb 18, 2021 at 18:23
  • 5
    I like this approach, but perhaps with a time component, say unpinning after 6 months? This would allow for all of the current uses for question authors while addressing outdated information. Commented Feb 18, 2021 at 18:26
  • 28
    @IanCampbell - the checkmark has always been about the OP, and only about the OP. It's just been horribly conflated this entire time to mean something it never could. I still fondly remember a question which has since been deleted in which someone writing Python wanted curly braces to represent code blocks (like in if statements, method declarations, loops, etc). Their response: accept their own answer which said that Python was a poor language. This would mean that the answer doesn't float to the top, but it's a good example of why accepted answers shouldn't in general.
    – Makoto
    Commented Feb 18, 2021 at 18:38
  • 44
    I wouldn't mind unpinning the accepted answer if we add some kind of a "Jump to accepted answer" button at the top. I've seen this practice on other Q&A websites.
    – 41686d6564
    Commented Feb 18, 2021 at 18:45
  • 15
    @41686d6564: If the answer is of value, then the community will vote accordingly. This has always been done on the site. Why do we need an additional feature for finding an ideal solution? (Relatedly, why are we presuming that the researcher must be spoon-fed "the" answer? It's seldom been the case that I've found direct and applicable value in an accepted answer, or just one answer at all.)
    – Makoto
    Commented Feb 18, 2021 at 18:48
  • 14
    @Makoto It's often the case that the accepted answer addresses the (very?) specific problem in the question. Then, you get other answers with more upvotes that maybe address the bigger picture or maybe offer an (arguably better) alternative approach, etc. Those are, of course, useful but highlighting the accepted solution is, in my opinion, often useful too. Plus, I see no harm in having this option just "sitting there" for those who may need it.
    – 41686d6564
    Commented Feb 18, 2021 at 18:56
  • 40
    @Makoto We are definitely considering unpinning the accepted answer as low-hanging fruit. More to come when we start down that road.
    – Anita Taylor StaffMod
    Commented Feb 18, 2021 at 19:56
  • 2
    @41686d6564 The harm would be if the option is largely unused or doesn't add much value most of the time when it is used. That would be bad from a UI perspective: adding more clutter to the UI is generally a bad thing that makes the interface more intimidating and makes it take longer to visually take in all the information on the page and find what you're looking for. It's something that would cost time to develop, test and maintain. Time that could potentially be better spent on other things.
    – NotThatGuy
    Commented Feb 18, 2021 at 20:19
  • 5
    It's nice to get the green checkmark, and while many site visitors may have the impression that the accepted answer's supposed to be the best one, I think most regular answerers realise that the OP often doesn't have sufficient knowledge to accurately evaluate the answers, and may be the least competent contributor to the page (ok, the people whose answers get a lot of downvotes may know even less than the OP). OTOH, (just playing devil's advocate for a moment), seeing a faulty accepted answer can motivate me to write a competing answer that addresses the shortcomings of the accepted one.
    – PM 2Ring
    Commented Feb 18, 2021 at 23:36
  • 8
    @Steven: I'm not advocating unchecking the accepted answer, I'm just advocating that it not be ranked any higher than the other answers.
    – Makoto
    Commented Feb 19, 2021 at 16:09
  • 3
    @computercarguy: For crying out loud, I'm advocating to just get rid of the sorting weight that it has. I am well aware of how to sort answers, but the average passer-by doesn't. Besides, I believe that the accepted answer is still pinned to the top no matter what sort you use.
    – Makoto
    Commented Feb 19, 2021 at 22:53
  • 3
    @computercarguy: The only time is when it's a self-answer that's been accepted. Every other time it floats to the top consistently.
    – Makoto
    Commented Feb 19, 2021 at 23:12
  • 3
    Instead of unpinning the accepted answer, simply add a warning stating Even if this answer worked for the OP, the community has preferred a different one by voting. Commented Feb 21, 2021 at 16:00
  • 3
    @mgarciaisaia: This isn't a debate or a suggestion. I'm saying that this is something that has been proposed some years back, and instead of waiting around to try and gather some data, doing at least this and measuring it would be more productive.
    – Makoto
    Commented Feb 21, 2021 at 20:52
  • 4
    I do not even understand the significance of accepted answer. The fact that OP accepted answer should have no bearing of the expected quality of said answer. It just means OP liked it. It could be all wrong. It could be that the avatar is nice. It could be 1000 and 1 reasons, and there is a good chance those reasons are not relevant to future readers at all. Upvote should be the only sorting mechanism,
    – SergeyA
    Commented Mar 1, 2021 at 19:57
104

I don't have any concrete suggestions for this, but I will say as the top answerer on the site in the last decade that this is something I hope to benefit greatly from. The frontend web platform has advanced significantly (I wouldn't say "by leaps and bounds" quite yet but it's definitely getting there) and many of my answers will have to be supplemented with new information.

Something I'd like everyone to keep in mind, not just for me but for this project as a whole, is that new information needs to be provided in a way that is respectful of history. A fair majority of edits to my answers adding new information actually do so respectfully, but from time to time, someone will post a comment, competing answer or edit asserting that an answer of mine is incorrect when in reality, it was correct at the time but is now outdated. Sometimes these answers will carry a "you should have known better" tone when in fact I did know better or best at the time. It's kind of irritating. This is a strictly social problem but I'd be interested to see what guidelines could be provided to keep people mindful as they update questions and answers with new information.

13
  • 29
    Agreed. I'd like to see a kind of gentle downgrading of the answer, with a "This isn't the best answer anymore" kind of flag. Commented Feb 20, 2021 at 7:29
  • 7
    I think it would also make sense once an answer is identified as outdated, maybe to lock it from votes. It is also bit unfair to get down votes on an answer which was once correct. Commented Feb 24, 2021 at 16:40
  • 4
    This also brings up another related topic. What if something gets a big red "no longer correct" banner, then gets edited by the original author to fix the problem ... should the banner be removed automatically, or should it go through another review queue, or is the answer now stuck with it until enough people vote to remove it? (I'm obviously guessing wildly about what is being planned, but this question I think needs to be kept strongly in mind in the design phase.)
    – tripleee
    Commented Feb 26, 2021 at 8:09
  • 2
    Same with JavaScript.... I get a bunch of downvotes because this is not have how to do it.... Well it was how to do it with IE8 or with the first version of bootstrap or react or angular. Just like how the one top answer for a duplicate says to use live() in jQuery. Commented Feb 26, 2021 at 17:46
  • @Flimm: That's the problem this project is trying to solve, isn't it?
    – BoltClock Mod
    Commented Feb 27, 2021 at 9:19
  • Sorry, there was a typo in my comment. I meant to say: I downvote answers that are not useful currently, even if they used to be useful before. Otherwise, they are taken up the space that belongs to better answers.
    – Flimm
    Commented Feb 27, 2021 at 12:23
  • 1
    @gmoniava: Some old answers (especially in tags like [cpu-architecture] or [performance]) contain explanations that were/are sort of right, but have some mistakes and misconceptions in their mental model of how things work. Sometimes suggest the right thing for the wrong reasons so have some practical value. But they sound convincing to people that don't already know better, and thus get a lot of upvotes over time. That's a different kind of old problematic answer than this project is really focused on, but in practice some will get flagged obsolete. I still want to be able to downvote them. Commented Mar 3, 2021 at 10:16
  • 3
    @gmoniava: also, if people don't maintain their old answers, they should expect downvotes when they're not currently useful. I have zero problem with downvoting obsolete answers, or with getting downvotes on my own answers where I should edit to add a note that this is an old sub-optimal way to do things. (Although without some mechanism to notify all the downvoters that the answer is fixed now, some won't get reversed.) Commented Mar 3, 2021 at 10:19
  • @PeterCordes In extreme cases a person may die :) So won't be able to maintain his old answer :) So because of such downvotes he may be 'remembered' as not someone who contributed :) Commented Mar 3, 2021 at 17:31
  • 1
    @tripleee I think the banner, while generated through voting/flagging, should become part of the answer body. This way, whoever edits, can write an up-to-date answer above it, followed by the banner, followed by the older version which may still be relevant to some.
    – CygnusX1
    Commented Mar 4, 2021 at 22:10
  • That sounds simple enough, and it's how duplicates used to work (I think? Before my time really) but injecting content comes with its own set of problems (edit wars by users who don't like the banner in their text; botched copy/pastes by users struggling with basic formatting, etc).
    – tripleee
    Commented Mar 5, 2021 at 5:34
  • What do you mean by "the top css answer"? I'm trying to check but I don't know if you mean number of questions answered, or number of accepted answers?
    – Ian Newson
    Commented Feb 19, 2022 at 0:18
  • @Ian Newson: Highest total answer score of all time. There's a leaderboard here. I haven't answered that many questions in the grand scheme of things, but many of them are popular answers to old questions that have almost certainly been supplanted by new standards. I suppose accepted answers do make up a decent chunk of them, though.
    – BoltClock
    Commented Feb 19, 2022 at 7:48
69

I think that a combination of four things (which are also already mentioned in other answers) will help here.

De-emphasize the acceptance mark It's really time for it. Acceptance just means that this answer helped the question creator most, it doesn't mean it helps anyone else. So why do we sort that answer at the top and do not trust voting more? I know that there is some mitigation, if the top most voted answer has kind of double the score there is already a resorting but it's much too weak. Acceptance can be a super-vote (say worth 5 normal votes) but that should be all about it. And make the mark smaller (visibly).

Easy to implement.

Sort by default by age-weighted votes Other websites do not only display votes, but also trends, i.e. they tell you something about the momentum of a score. One answer can have a large score from the past but negative momentum recently while another newer answer can have a low score but gaining lots of votes lately. Which answer would you like to present first to a reader? A universal solution is to age votes and compute a score that weighs votes less if they are older and then sort by that age-weighted score. Make this the default sort order.

Easy to implement.

My hope would be that these two things already solve the problem largely. They are universal and do not require much tweaking. They have been proposed abundantly in the past.

Emphasize the age of the content of an answer more. We have the creation date of an answer and edit dates to kind of bracket the age of an answer. However, edits can be minor or major and therefore the age of the content of an answer is difficult to estimate. If you had a system to identify minor and major edits. You may be able to estimate the age of the content and display it more prominently.

Example: Answer 1 (last major edit: June 2016), Answer 2 (last major edit December 2020) That could even be printed above an answer.

Medium difficulty to implement.

Introduce answer tags concerned with versioning or used technology We have tags for questions, so why not tags for answers (that basically extend the question tags). That way we could differentiate answers by what version/technology is used in this answer. This is just a rough idea. It would need to be discussed thoroughly before an implementation.

Example: Question with tags [aa] [bb] [cc] and Answer with additional tags [aa-version7] [dd]

Quite complex to implement. Answer tags should probably be used only sparingly and with this specific purpose in mind.

13
  • 1
    If a momentum-based ranking was introduced, wouldn't that cause the same snowballing effect?
    – user15072974
    Commented Feb 28, 2021 at 18:12
  • 2
    @Harith It would cause much less of it. With the current system, you would have to fight against the whole lifetime of votes. With a momented-based ranking you would have to fight only against a limited time period. The snowballing effect is largest with the current system. A bit of snowballing you actually want. Otherwise we could simply present random orders and every voter would have to read all the answers new every time. That's also not optimal. The optimum is somewhere in the middle. Commented Feb 28, 2021 at 18:20
  • Agreed - the accepted checkmark is pretty irrelevant once there are other answers with a few votes, and a month has passed. Commented Mar 1, 2021 at 9:17
  • 2
    @SteveBennett Many people seem to see it judging by all the answers here, only the company so far hasn't. Let's see. I did some research about the acceptance and the score of an answer and while it's irrelevant it also doesn't hurt too much. In "only" 20% of cases there is another answer with a higher score. In 80% of questions with at least score 2 the accepted answer is also the best (highest scored) answer. I try to make nice graphs out of it and then publish on meta. Commented Mar 1, 2021 at 11:52
  • 2
    +1 for age-weighted votes
    – stevec
    Commented Mar 4, 2021 at 15:20
  • 3
    Just a minor remark: answer acceptance is also problematic if the accepted answer is an easy “try this” post that may work most of the time, but the highest voted answer is a detailed explanation of the problem and provides several solutions that cover all edge cases, etc.; that’s not even necessarily because the detailed answer came a few years later — a few seconds later is enough. This happened to me once: of course the detailed answer took longer to write, but the OP already accepted the quick and dirty answer (which doesn’t even work in some cases), and deleted their account — kinda sad. Commented Mar 7, 2021 at 22:10
  • @SebastianSimon Currently we would probably edit the accepted answer and link to the more detailed answer in that case. I think it would be better to simply not pin the accepted answer if the score is much lower than that of another answer. Commented Mar 8, 2021 at 9:05
  • Aging of votes could also make people vote more. If I see a post with score 5 or more, I usually don't vote because "it has enough love already". If votes are constantly aging, I might consider voting regardless of current score.
    – anatolyg
    Commented Mar 8, 2021 at 16:42
  • @anatolyg I typically upvote everything that I find helpful regardless of score. But arguably things that I recently upvoted were useful recently to me and that is worth a bit more than something that was only useful to me in the past. That could be a justification. Commented Mar 9, 2021 at 6:57
  • "Sort by default by age-weighted votes". This. Yes.
    – maxpaj
    Commented Mar 10, 2021 at 11:28
  • 3
    +1 for versioning. Is it so hard to encourage posters to include the standard or software names along with the version numbers for questions and answers? I imagine going forward that should be straightforward to implement as some kind of requirement or cultural expectation. Reviewing old posts, however ... maybe some reward for those willing to test solutions in heavily-referenced posts?
    – r a
    Commented Mar 11, 2021 at 14:19
  • Are there examples of what (3) would improve that (1) and (2) wouldn't already improve for 99% of cases? Commented May 13, 2021 at 1:37
  • @MateenUllhaq None of this has been implemented, so I guess that there aren't really any examples available. But for example if you would know that an answer is 3 years old and you are searching for a technology that you know is only 1 year old, you could ignore this answer regardless of its score or acceptance mark. Commented May 13, 2021 at 5:55
46

Like most content curation problems that Stack Overflow faces, this is one the community has already put many man-hours of thought into fixing, so I'm certain that the best suggestions can already be harvested from the questions you've already linked to. Hence I'm not going to waste anyone's time by re-suggesting something that someone smarter than me probably already thought of years ago.

What I am going to say is thank you for finally addressing this, and thank you for doing it the right way - by looking to the community's suggestions first, not last. Even if our suggestions are rubbish and you end up using none of them, the fact you're coming to us as your initial port of call indicates that you do believe that we're likely to be able to help solve this problem, and that tells us we're valuable and wanted. Considering that so many actions from Stack Exchange Inc. in recent years have achieved the opposite, your simple act has already gone a long way to rebuilding some of those burnt bridges.

I'm excited to see what you come up with - just remember that when the criticism comes (and it will, because we're a... difficult audience), it's coming entirely from a viewpoint of what we believe is best for Stack Overflow, not because we're being deliberately obstructive.

36

This is a big task, but probably one that SE can't delay forever. I don't think this is a single issue, but more like several closely related issues. One issue is that right now we have no integrated way to attach versions to specific answers, something like this is almost certainly necessary to begin to solve this problem. The other issue is that the way Q&A works here inherently favors older posts, the voting system and the duplicate closing system both make it hard for new answers to replace older ones.

There was a version system in the short-lived Documentation feature on SO. That feature didn't work out, but the version system wasn't the issue, so it's probably worth looking at that. Though the simplest system might be a binary one, an answer is either "obsolete" or "current". This would provide much less information and is far less powerful, but it doesn't increase complexity as much as a real version tagging system would.

With a version tagging system surfacing new answers becomes much simpler, just default the sort order to put obsolete answers to the bottom, and it doesn't matter anymore that those answers have hundreds of points. There's quite some potential for abuse here, but that is usually something community moderation can handle.

It might also make sense to look at duplicate closing, in some cases not only the answer but also the question is obsolete. But that is probably the less common case.

And of course you have to deal with accepted answers, I've long advocated to simply remove their special status at the top of the list, which would remove this concern for your project entirely.

7
  • 8
    Python, one of the more popular languages on SO, has 2 major versions, and a handful of version-related tags, which are supposed to be used only in addition to the generic python tag. But frankly, it's a bit of a mess, (the version transition took 12 years), with a big percentage of questions missing the generic tag, and tons without a version tag, or version-related info in the question body. Admittedly, a lot of answer code does work ok in either version, or only requires minor changes to run. But there are a ton of answers where it's tricky to determine their version.
    – PM 2Ring
    Commented Feb 18, 2021 at 23:50
  • 18
    I would love to see the ability to version-tag code or sections of prose as allowed by Docs. That enables a sort of gradual improvement similar to what is seen every day as folks edit posts to improve formatting or update code - I've had a fair number of answers edited to add ES6 syntax or remove now-archaic notes, but unless a reader thinks to check the revision history this is hard to see.
    – Shog9
    Commented Feb 19, 2021 at 2:12
  • 2
    @PM2Ring the big new thing that is needed is a version tag or an obsolete flag for answers. For questions you can simulate this with tags, with the biggest issue being that people are not consistent enough about this and the tagging system not understanding versions. Version annotations also should probably be omitted unless the version dependence is very clear and obvious, unnecessary version tags would be quite harmful if the code works in older versions than tagged. Commented Feb 19, 2021 at 6:49
  • @MadScientist Regarding the versions, I wonder how we would tag for example an answer that contains solutions to different versions of a technology? Would a version tag also imply all later versions? Commented Feb 19, 2021 at 18:04
  • 2
    Versioning was my first thought as well. In many cases, the real problem with an answer that was posted 10 years ago isn't that it's "obsolete," but that the question was asked 10 years ago. It's probably still useful to someone, even if most people have long since moved on. Commented Feb 19, 2021 at 18:23
  • 5
    @Trilarion you probably need something like "works from Version X on" and "works until version X" to handle both new and obsoleted features. Having to tag every single version would be far too annoying, so I think you don't get around version ranges. Commented Feb 19, 2021 at 18:28
  • 2
    "With a version tagging system surfacing new answers becomes much simpler, just default the sort order to put obsolete answers to the bottom" What if an answer was tagged with version 7, and we're now at version 15. The version 7 answer might still be relevant and "the best", but in your proposal, it would be sorted to the bottom because it is only tagged with version 7. Commented Feb 23, 2021 at 14:48
34

Use Reddit-like sorting

I'm surprised I haven't seen this suggestion yet: default to sorting answers by "best", where "best" is a metric that takes into account many factors, including votes, age, accepted, etc. Perhaps this has been discussed previously and I just missed the discussion, but I feel that it's a decent way to make the top answer be the one that is most likely to be helpful.

Reddit uses a similar system, and it seems to work well for them. It doesn't require a review queue, it doesn't require more flagging options, it doesn't add an addition vote button, it doesn't change user interaction at all—except to hopefully require less scrolling to find the best answer. It simply takes information that is already available and makes use of it.


As some example scenarios (these are just here to demonstrate my point; don't read too much into the specific situations or numbers):

  • Imagine a question that gets an answer with a score of 100. A year later, another answer is posted that receives 50 upvotes within a month. It's likely that the new answer is the better one, since it got upvoted so quickly.
  • Imagine a question where the OP accepts an answer, but another answer comes along later that receives significantly more upvotes. It's likely that the new answer is better, since the community seems to strongly believe that that answer deserves more upvotes.
  • On the other hand, imagine two answers are posted. One is +90 and accepted, and the other is +100. A 10% difference is not much, so the accepted answer is likely to be better since the OP is more likely to know what works and what doesn't.
  • Imagine two answers that are +70/-30 and +25/-0, respectively. Even though the first answer's score is higher, it's likely the worse one, since 30% of people strongly believe that it is a bad answer. This can correct outdated answers (which will get downvoted for being obsolete), and has the added benefit of correcting for bad advice, regardless of how outdated the answers are.
10
  • 12
    "Reddit uses a similar system, and it seems to work well for them." but it's also hard to apply it to SO. Reddit's system...also has its problems. "Good" replies also get sunken because "good" is not up to the algorithm to determine. A well-researched and informative post can be "worse" than a one-off joke response. That's actual things I've seen on Reddit and I'm not much of a participant there. Moreover, on SO we deal with technical accuracy - it's hard for an algorithm to judge that. "Hotness" is not a useful metric.
    – VLAZ
    Commented Feb 19, 2021 at 12:09
  • 4
    Exactly! This is also a good post to read: evanmiller.org/how-not-to-sort-by-average-rating.html Commented Feb 19, 2021 at 12:16
  • 2
    @VLAZ I agree that hotness may not be a perfect metric, but I would argue that factoring at least a bit of "hotness" into the ranking is still better than simply the net upvotes minus downvotes. I think that penalizing controversial posts would actually help the issue with joke answers that you mention; a joke will receive a lot of upvotes for being funny, but it will also get downvoted a fair amount by people who don't appreciate the post. A well-researched an informative answer may not get as much of a reaction, but it will be more consistently upvoted. Commented Feb 19, 2021 at 19:37
  • We would need a metric to measure the success of any sorting order. Something like how much time people spend on pages. And the goal would not be to maximize it. Commented Feb 20, 2021 at 8:47
  • 3
    This may also be difficult from an implementation perspective because it would introduce an algorithm of sorts that can be infinitely tweaked with particular parameters. I'm not sure if the devs will want to introduce a mechanism that people can always complain about not working for their particular question.
    – Pro Q
    Commented Feb 21, 2021 at 21:35
  • 2
    @ProQ People always complain. Basically this sort order could be called relevance and the algorithm could be made public within a meta post and if the meta crowd feels it should be changed, they could open a feature request about it. That would be quite transparent. Google, amazon, facebook, ... they all show you basically sort orders that are complete mysteries. People are okay with it as long as it is useful. Commented Feb 22, 2021 at 9:13
  • 1
    I could imagine this alongside other solutions, that come up here. @Trilarion Making the algorithm public could also have negative side effects, as in that case it can be abused users, e.g. by rep-hunters, who want to make their answer ranked first, that's why the exact behaviour of the moderation-related algorithms on SO are kept secret.
    – FZs
    Commented Feb 24, 2021 at 8:14
  • 4
    I think that any kind of metric that factors in "hotness" will fail in a lot of scenarios. I posted an answer today on a question that already had some 3-year-old, upvoted answers. It also got several upvotes in the first couple hours. But my answer isn't more up-to-date than the existing answers; they're both fine! In fact it's really mostly a matter of opinion. It wouldn't be fair to the other answerers to promote my post as the "best" just because I procrastinated for 3 years.
    – trent
    Commented Feb 25, 2021 at 19:50
  • 3
    Every ranking system has pathological cases, and I doubt that trading the well-understood pathologies of "sort by active/votes/oldest" for the unexplored pathologies of "sort by hot" would be a beneficial exchange for SO. Unless your quality metric is "number of questions asked on Meta about why my answer is ranked lower than this other answer", because I'm pretty sure that would go through the roof.
    – trent
    Commented Feb 25, 2021 at 19:54
  • @trentcl That example is a semi-popular question (2k views) and is fairly "recent" (3 years). If you obtained only 3 votes versus 12 for the other answers, I don't think that's enough to say P(your_answer_is_best) > 0.2. If it were also 12 votes (which, incidentally, it is now), then perhaps one could safely argue that an answer receiving the same amount of votes within a shorter period of time is far more likely to be most useful. Regardless, it would make most sense to implement a good sorting algorithm for the really popular questions (300k views) first. Commented May 13, 2021 at 1:44
28

Some general thoughts

Accepted Answers

The green checkmark was a good idea initially and still largely functions as designed. We want the question asker to deign which answer helped them the most. By and large that has succeeded. The problem this created is that it then skewed the system to show the checkmark first, which assumed something not always true: that the questioner picked the best answer. It's such a problem it's basically a trope, as first noted by the Many Memes of Meta and more recently by this Monkeyuser comic.

I'm sure someone will suggest an easy fix here: just let moderators change checkmarks (or at least un-accept answers). There's two major problems with that

  1. Moderators are a poor choice to arbitrate content disputes on the merits of the post (for instance, I know very little about , , etc.) and it's rather nice to be able to decline mod flags asking for checkmark modification of any sort because we don't have that power. Giving this to moderators would also mean many folks would start flagging everything with a checkmark they thought was wrong
  2. It basically is voting for someone else, which I am remiss to ever see happen. Unless there is voting fraud, the checkmark needs to remain in the sole purview of the questioner.

What we should do is add logic to reduce the extra weight the checkmark provides an answer when the community disagrees. This could be based on voting, which (as Monkeyuser noted) is a better long-term metric for quality. This could be applied automatically without as much community impact (it would involve changing a default)

Outdated Answers

This question is a fairly good example for a question with outdated answers. The top answer is fine, but some answers like this one are based on older methodology (in this case this function the answer advocates was removed from the language 5 years ago)

The community has stepped in here to edit in some cautions about this, but they are merely edits, which can be removed or changed by anyone. What would work better here would be a post notice of some sort (sample syntax below).

Outdated post notice

Here's how I would work this

  1. Anyone with a bronze badge (20+ answers and 100+ upvotes in a given tag) or higher in the language it's tagged in (same rules about tag editing would apply) could propose a post notice indicating it's outdated
  2. Tag badge holders could then vote on the outdated notice in a review queue. Three votes approves it and they would be weighted as
    • Bronze - 1 vote
    • Silver - 2 votes
    • Gold - 3 votes
  3. Diamond moderators can set and remove as needed
  4. Once added, the post would fall below any non-outdated answers

There would need to be some sanity checks

  1. While you can freely propose one, you could not vote on any you proposed (same workings as the tag synonym system)
  2. There needs to be a moderator ability to turn such Outdated suggestion and voting privileges (not general voting) off without a full suspension. There are some gold badge holders who would readily abuse this to damage answers that they merely saw as "wrong"
  3. You can't propose them on any question that doesn't have another positively scored answer (prevents you from making a competing answer with a 0 score just to suggest an outdated notice)
7
  • 7
    Custom post notices might be more flexible and less work then version tags. In Wikipedia they work well. However sorting them at the bottom is maybe too much (you would basically override all the votes of the past). The warning should be enough for readers. Commented Feb 20, 2021 at 8:59
  • 1
    The notice would be even more powerful if it could be linked to the more appropriate answer. How the sorting order of answers should be influenced by the votes and links is something that could be investigated as well. Commented Feb 20, 2021 at 11:35
  • 1
    some gold badge holders who would readily abuse this to damage answers that they merely saw as "wrong" ...yes, please give me a "provably wrong answer" notice flagging feature! I have been wanting this for a long time. Commented Feb 22, 2021 at 5:54
  • I like this "Outdated answer" flag, also with the weight of votes depending on badge. Since there is no urgency to mark a post as outdated (contrary to closing an off-topic question), I'd suggest a higher votes treshold, though (subject to being lowered depending on analysis later).
    – Pac0
    Commented Feb 22, 2021 at 23:35
  • 1
    An "outdated answer" post notice might also be useful on sites like RPG.SE, where certain answers can be invalidated by errata to the rules that substantially changes the correct answer to a question.
    – V2Blast
    Commented Apr 24, 2021 at 21:57
  • 1
    You should then take also care of outdated answer which can ceases to be outdated someday in the future (ex: partially broken langage trivias abandoned then re-endorsed when repaired). I guess the outdated answer notice should not be linked to any other answers, as the new answer could be outdated someday and it will be a huge pain to deal with that then.
    – Zilog80
    Commented Feb 17, 2022 at 21:35
  • I guess also that my comment above implies the same approach that @GeorgeStocker suggested in his answer . Maybe we should look for a possible way to associate answers with a context, as the only thing we can take for sure is the past, not the future (product versions, env versions/dates, etc.). A dependencies approach like. We'll all be obsolete one day or another (and maybe i'm already that...).
    – Zilog80
    Commented Feb 17, 2022 at 21:44
27

This is a huge undertaking. As such, I would suggest that, once a course of action has been decided upon, it would be advisable to limit the implementation, in the first instance, to a select few tags. That way, any unforeseen, negative consequences would be limited and, thus, much easier to revert and/or redress. Once the basic process has been evaluated and proven successful, it could then be rolled out over more tags.

Also, it is likely that any remedial action will require the assistance (or, at least, cooperation) of Stack Overflow's vast pool of experts in the tags specific to any posts to be judged as outdated. Although the staff and moderator teams have, between them, huge expertise in many programming fields (particularly 'popular' fields, like C++, C#, and Python), there will be many areas not covered by the diamonds (COBOL, PDP-11, HtmlHelp, anyone?).

How and when to 'recruit' such expert helpers (Gold Tag-Badge holders), and exactly how they and the moderator team could work with staff to accomplish the mission is something for a later discussion, perhaps.

7
  • 4
    Gold badgers aren't the only experts. Perhaps there are tasks that silver badge holders (and maybe even bronze) can do. I don't think merely identifying outdated answers requires the same level of responsibility as single-handedly closing & opening dupes
    – PM 2Ring
    Commented Feb 19, 2021 at 5:07
  • merely identifying outdated answers ? Irrespective of who is involved, there is nothing mere about it.
    – QHarr
    Commented Feb 19, 2021 at 7:19
  • 1
    Are COBOL answers from 2015 going to be outdated in 2025? Python, C++ and C# are in active development, with fairly frequent updates, precisely because these languages are popular.
    – MSalters
    Commented Feb 19, 2021 at 10:41
  • @MSalters It's unlikely, indeed, that question about either COBOL or PDP-11 will become outdated! However, the point was that there are likely to be tags/languages still under active development in which staff and/or moderators will not have sufficient expertise to assess answers for potential outdatedness. Commented Feb 19, 2021 at 11:20
  • Rather, all questions about COBOL are per definition outdated. Doesn't mean they should be closed, but maybe whole tags could be marked as outdated.
    – Lundin
    Commented Feb 19, 2021 at 11:55
  • 11
    @Lundin - That attitude is precisely why we need appropriate experts in the various areas. COBOL is under active development (not just new code being written in COBOL, but various implementations of the language itself are actively (albeit usually modestly) updated). This just goes to show that even users with a 6-figure rep can be completely unknowledgeable about some subjects.
    – John Y
    Commented Feb 19, 2021 at 16:31
  • 2
    @QHarr Fair comment. I should have said "just identifying outdated answers [....]". The task of correctly identifying outdated content is hard, and I believe that in many cases it will require human input, not just machine learning. But just identifying that content doesn't have the same level of responsibility as deciding what to do with that content. I see it as similar to how we handle dupes. Any low rep user can flag a dupe, with more rep you can VTC the proposed dupe, and with a gold badge (or diamond) you can take unilateral action.
    – PM 2Ring
    Commented Feb 19, 2021 at 19:04
22

I think this a wonderful conversation to have and now is the time to have it.

One major issue I see is that currently questions and answers are two-dimensional to Google (the text is what matters); and two dimensional to users: tags are what we categorize by.

But, there's another category that we have to be aware of now: Version numbers. Every single answer on Stack Overflow was true and correct at one time, and it's possible that most if not all are still true and correct today, if you're still operating on the exact version you were when the question was asked.

That means, that for people out there (for quite a while), these answers we think of as outdated, aren't.

I do like how MSDN solves the problem: in the query string for the page (and as a drop down) you can see the documentation as it applies to a particular version of the libraries or APIs.

MSDN Documentation

MDN takes another approach (one that I don't necessarily agree with, but here we are): for a particular feature they label which browsers that feature works for, here's JavaScript's Array:

Enter image description here

Regardless of how we solve the problem, one of the major considerations is if there's a manual curation process involved, to give it reputation or recognition, because making sure we scope what the answer works for is as hard work as answering the question; and deserves the same recognition.

5
  • 4
    This sounds like step in right direction. I am not sure how would this work out in reality, but versioning is the only way out. There are answers that will almost never be outdated and there are answers that have very short version life-span, but they are still extremely valuable for anyone that needs to use outdated versions for whatever reasons.
    – Dalija Prasnikar Mod
    Commented Feb 19, 2021 at 18:15
  • 7
    FWIW, MSDN has been slowly moving away from hard-forked versions for something like a decade now; I suspect they're increasingly impossible to maintain. MDN's approach is more work per topic, but limits the growth on topics over time.
    – Shog9
    Commented Feb 19, 2021 at 18:26
  • I think this will be difficult to pull off, but way important. I know of many times where in the long tail of versioned answers, I've found "my" answer for a particular (usually newer) version that has a lower prominence simply due to time and/or votes.
    – Kit
    Commented Feb 19, 2021 at 21:58
  • 3
    Overall, an excellent post! However: Every single answer on Stack Overflow was true and correct at one time ... are you sure about that? (I can provide links to answers posted by my own good self that were never correct.) Commented Feb 20, 2021 at 19:40
  • 2
    @AdrianMole every accepted answer; or put another way upvoted answers. But I'm not going to get into a war over which upvoted answers are incorrect and which aren't. Commented Feb 20, 2021 at 22:49
20

There are a lot of different aspects when it comes to identifying outdated content.

I strongly support unpinning the accepted answer from the top, but some of the accepted answers actually benefit from it. A newer solution can be accepted by the question asker to give it visibility over the older outdated answers which have higher score. Neither the answer score nor the acceptance mark are a perfect indicator of how up-to-date the answer is.

Some technologies have received major updates or were outright deprecated. However, there are still people who use these deprecated solutions. As long as Stack Overflow answers are providing some guidance on how to use them, then they can still be useful. However, answers to problems unrelated to the outdated tools, but which use them to solve the problem, are outdated. It will be difficult to employ heuristics to define whether such answers are still useful or not.

Stack Overflow is also full of really bad advice. It would be great if these solutions were heavily downvoted or even deleted, but what is often the case is that the most dangerous solution is the easiest laziest one that you can implement. I would love to see how we come up with a solution to tackle this problem.

I believe that such a process should be a manual task. I know there have been proposals in the past that suggested a flag which could be only available to gold-badge holders, which would send an answer to a review queue. Other users can then vote what to do with such answer. I wouldn't trust myself to not make mistakes, and I certainly wouldn't trust AI to be infallible.

We have to keep in mind that if we remove or hide the outdated answers then we should make sure that up-to-date alternatives are available. This shouldn't be difficult given that most topics have been answered on the site 100 times over.

I certainly welcome such initiative and await further developments.

5
  • 3
    Speaking of dangerous answers, I don't need to tell you about the prevalence of SQL injection (and how some answerers even get aggressive when asked to remove that vulnerability). But that's an ongoing danger, orthogonal to the issue of outdated answers.
    – PM 2Ring
    Commented Feb 18, 2021 at 23:57
  • 1
    "Stack Overflow is also full of really bad advice." relevant. There is "outdated" and "bad". As you said, the latter might just be the easiest which is a big problem because "easy" can also get hundreds of upvotes. Which then makes it basically undeletable.
    – VLAZ
    Commented Feb 19, 2021 at 11:57
  • Concur. Sometimes an enlightened asker changes the accepted answer to a newer answer from a more popular answer they had previously accepted because the new one is now "most helpful." Commented Feb 22, 2021 at 14:12
  • 1
    If we unpin the accepted answer, I believe we should only do so after x amount of time has passed (and then still leave some sort of marking for historical context, perhaps just turn the checkmark gray, make hovering over it explain that it is an outdated checkmark, and no longer pin in to the top). Commented Feb 22, 2021 at 23:13
  • @wp-overwatch.com No, leave the checkmark as it is now. Just don't pin the answer to the top regardless of age. Same as we do with self-accepted answers
    – Dharman Mod
    Commented Feb 22, 2021 at 23:42
19

You have not mentioned (unless I missed that part) what action you plan to take once it's possible for you to identify outdated questions.

I would suggest that you may want just to visibly flag them and/or add a warning.

More drastic measures (like auto-deleting) should require a separate conversation.

12
  • 36
    No auto-deleting. Someone out there my still be using that legacy technology/framework.
    – Titouan
    Commented Feb 18, 2021 at 18:35
  • 2
    @Titouan: I don't know how I would feel about maintaining documentation for a service that was literally EOL'd in the last 10 years just because someone might still be using it. If there's no clearer signal to someone to start investing in newer infrastructure than the fact that their hired engineers can't find documentation or resources to maintain it, then that team's inevitable failure is not on our heads.
    – Makoto
    Commented Feb 18, 2021 at 18:39
  • 11
    Yes it would be on us because we would have deleted a possible helping answer. I agree not to put forward such answers but they must still be findable for that one person. You say "maintaining documentation", it's not really maintaining (in the sense updating) but really just keeping it on the site.
    – Titouan
    Commented Feb 18, 2021 at 18:42
  • 1
    @Titouan: It really wouldn't be on us if someone decided to continue to use Log4J 1.x in a production service in 2021 some six years after the project maintainers decided to move on to Log4J2. The goal is to maintain a repository of useful questions and answers that'll stand the test of time, and since software always changes, it is crucial to evaluate what to do with older content that is not applicable or maintainable (and yes there is a maintenance cost in curating the question and evaluating the answers to a question like that).
    – Makoto
    Commented Feb 18, 2021 at 18:44
  • That said @PM, I don't think that this is a separate conversation at all. Identifying out of date answers is half the battle and maybe a third of the point. I come to Stack Overflow all the time looking for answers through Google, and the worst feeling I've had while working on a custom Gradle plugin is answers from...2014 about outdated or no longer supported Gradle framework approaches on plugins. Not having them here so that I don't waste my time feels like the whole point of a project like this. Just having it tagged means...nothing, really.
    – Makoto
    Commented Feb 18, 2021 at 18:46
  • 5
    This is about identifying outdated answers not outdated question. The latter can be dealt with by the community by closing.
    – ChrisF Mod
    Commented Feb 18, 2021 at 18:48
  • 3
    @ChrisF I don't think any of the close reasons apply to an otherwise good question that's just outdated. I could down vote a question for not being useful but that doesn't make it unclear, opinion based, etc.
    – BSMP
    Commented Feb 18, 2021 at 18:51
  • 2
    @BSMP that may be true, but this question is about outdated answers not outdated questions. Outdated questions can be closed using the "Not reproducible or was caused by a typo" reason - specifically the "not reproducible" part.
    – ChrisF Mod
    Commented Feb 18, 2021 at 20:07
  • ...but this question is about outdated answers not outdated questions. @ChrisF Yes, I know.
    – BSMP
    Commented Feb 19, 2021 at 9:56
  • I am not sure you should downvote an "out of date" question, its not out of date to everybody and it was asked for a real reason in the first place, I have an XCode question that could be considered out of date, yet every year it seems to get an up vote, some people are still finding it useful. @Makoto there are all sorts of reasons that an engineer might be using a "dead" technology, from having to access an archive, to being in a regulated industry that doesnt move very fast, to its impossible to update. How many government departments are still using XP?
    – Gavin
    Commented Mar 2, 2021 at 16:05
  • 2
    @Gavin: The downvote tooltip says, "This answer is not useful", and if the answer isn't useful because I'm using a newer library than the answer provides a solution in, and the solution is either incompatible or deprecated on my end, then there's no reason why I shouldn't downvote an out of date question or answer!
    – Makoto
    Commented Mar 2, 2021 at 16:17
  • @Makoto fair enough :). There is one consequence of that though, if enough people down vote the accepted answer due to time, you will end up with acceptance with negative votes. Not an issue of course, just an observation.
    – Gavin
    Commented Mar 2, 2021 at 16:20
18

I think it is very important that when marking a post as outdated there should be immediate information as to what exactly makes the post outdated. This text should be editable the same way the posts itself can be edited.

At a minimum, I think the warning should list:

  • Why the post is outdated (is there a security risk? Is there now a better way? Does it flat out not work anymore)
  • Since when is the post outdated (either a time or the version of the software)
  • (Optional) A link to either question or answer that works for the newer version of the software

Some examples could be, for the non-existent language mouselang:

Warning: The function unsafe_eval has been deprecated since Mouselang 2 and has been completely removed in Mouselang 3. See [link] for a solution using eval

Or, for an answer that was always bad:

Warning: This code has a SQL injection vulnerability. NEVER use this with untrusted input since it gives any user full access to your database. See OWASP for more info

Or:

Warning: Mouselang 7.2 introduced the swordfish operator that provides a much easier way to solve this problem.

Extra details are really important. If we simply mark the answer as outdated without specifying since when, people running old versions of software can never be sure if it applies to them or not.

1
  • I guess the biggest problem is identifying such posts and how to correct errors with these warnings. We could have edited such warnings already into answer bodies but I hardly remember seeing such things. Commented Feb 22, 2021 at 6:47
17

What does "outdated" mean?

The section "How do we define outdated content?" hints at the likelihood that there are multiple solutions needed. It also misses the context of outdated questions. Outdated questions could be addressing issues which no longer exist (security patches or version updates have eliminated the issue), or be targeted at a deprecated release, framework, or even dead product.

The discovery of outdated content will be most effective when the discovery process is targeted for the reason, or classification, of outdatedness. Often, as well, the "outdated" content is not obsolete, even if outdated. A solution to a Bash issue which addresses, for example, Bash < 4.3, could be much different in Bash > 4.3. Backwards compatibility is often a design goal, if not a requirement, and answers targeting older versions of a framework, or language, can retain value, even while losing popularity. (The last bullet in the referenced section.)

Fast, Cheap or Good? Pick Two.

Fast: The discovery process developed fits into the same mold as any other development project. It's possible to design and develop a process which does reasonably well at "finding" outdated content based on some set of rules now. Changing the rules later could break the process.

Good: The discovery can be automated, even with the ability to alter the rules, yet it will still have false-positive results and false negative results.

Cheap: I'm hopeful that it's planned to involve the Community, and not just the 10K+ elite, in the discovery process, not just the classification process. Sorting the wheat from the chaff, which is something the community does already with votes and flags, is going to be an important part of the process. Gaining community involvement is not cheap, even if essential.

Possibilities

Some elements of the discovery can be implemented with something similar to flags or voting. Similar to both, yet distinct. Adding a new flag is contraindicated by the number of flags already being processed by the system, and the fingers behind the system. There's no point in adding to the work load of said individuals. Additionally, flagging is designed to have an effective result once the count reaches a threshold, or when the flags are processed and approved/denied by another user. The discovery process might add an entry to the "obsolete list" once a threshold is reached, but there shouldn't be any indicator on the post that it's been marked, or considered for marking, as obsolete until there's a decision on what to "do" with obsolete content. The voting system on comments, up-only with no effective actions other than an incremented display, is almost the idea. Again, however, the results should be silent.

The idea of silent results for discovery is two-fold. First is that, after some time, posts might have a significant amount of obsolete votes, and other users could decide not to "get involved" as it's already been marked. Second is avoiding "bandwagon voting." As a bonus, the voting could have a radio-button dialog, similar to flags, where the "reason" for considering it obsolete could be selected. The list being based on the eventually developed conditions to deal with established by the "team" of community.

Closing thoughts

I have personally been plagued by reading obsolete answers, or digging through mountains of questions from search results with similar questions from obsolete packages, trying to find an answer useful to me and my current issue. Most of the time I do, eventually, find a question and answer which is both relevant to my issue, and current enough to be useful. Some method of filtering out that would be helpful. There have also been times when I've had to deal with "obsolete" resources, and the obsolete content suddenly becomes very useful.

Ultimately, discovery of obsolete content is going to be best handled once there is some plan, even if subject to change, of what to do with the discovered content. The discovery of the content is pointless if what's discovered fits into a "category" that has no purpose, or planned response.

P.S.

Good luck!

2
  • 2
    Hmm. Particularly in the bash tag, I worry about the community getting things badly wrong in security-impacting ways. Focusing on the "10k+ elite", as you put it, is likely to be helpful -- folks who click the upvote button on answer that's short and simple and looks like it works (without understanding the unfortunate details of runtime behavior that make that answer dangerous to use when the range of potential inputs isn't carefully constrained) are a problem. (Not just an outdated-answer problem, but particularly one, since many newer features exist to make eval less necessary). Commented Feb 18, 2021 at 22:10
  • 1
    @CharlesDuffy The security concerns of eval in Bash is a different problem, a very legitimate issue. The 10K+ elite part wasn't to discount their value, it's to spread the work to a broader force. The "tally" of outdated votes could, and probably should, be skewed by the rep of the caster, or their gold-badge status on the questions tag(s). If such a system were developed for discovery in this process it could lead to a similar, permanent, system for security flags on many sites; ServerFault, for one, comes to mind. It still comes down to what's going to be done with the "discovered" posts.
    – Chindraba
    Commented Feb 18, 2021 at 22:22
13

Many answers and comments proposed algorithms, flags, tags, or combinations thereof for how answers should be sorted. Many have merit, like age/vote weighting and versioning. What’s missing is the idea of providing us, the SO users, with more flexibility in how we sort answers. We know what problem we’re investigating. We know what versions of X, Y, and Z tools we’re using. We know the problem’s context. The existing Active, Oldest, and Votes buttons provide only rudimentary capability. At the very least, they should allow for a reverse sort when clicked a second time. Another idea is to make the buttons additive, rather than independent, so that clicking active and oldest sorts on both criteria. I won't enumerate the many possibilities -- you get the idea. Such enhancements would be immensely helpful, especially on questions like this one with 55 answers.

A related issue that detracts from the intuitiveness of the interface is that the last sorting button click on the currently viewed question is applied to subsequent questions you visit. It's disorienting because there’s no relationship whatsoever between the current and subsequent questions. Revert to the default sort from question to question or, even better, give us the option so that we can make SO behave in a way that better meets our needs.

2
  • 1
    But I wonder... is it not the case that the large majority of questions have one "proper" answer? And only a minority have many?
    – einpoklum
    Commented Mar 4, 2021 at 22:35
  • The SO team can certainly quantify "large majority" and "minority" to appropriately determine priorities. Personally, though, it feels like I'm constantly fiddling with the answer sorting buttons as I follow the trail of a problem through SO.
    – cb4
    Commented Mar 4, 2021 at 23:02
13

A bunch of ideas to consider, in no particular order, numbered strictly for commenting convenience:

  1. Encourage answerers to be explicit about time-dependent assumptions and definitions.

    Example: Instead of "The C++ language mandates that XYZ", prefer "The C++17 language standard mandates that".

    ... of course this is a measure for future answers, but still important IMHO. Also, instilling this custom will also influence people editing older answers.

  2. Obsolescence should be addressed for both answers and questions, even if not necessarily in the exact same way.

  3. Try soliciting poster input. If there is indication an answer is obsolete, message the answer poster suggesting they review it. And if the poster doesn't do so - message the asker suggesting they review the answer for obsolescence.

  4. When there's an indication an answer may be obsolete - but not always - emphasize its age visually somehow.

  5. I shudder to suggest this, but - introduce a new review queue for obsolescence?

    ...

    (ducks) I'm sorry! I'm sorry! Don't hurt me!

  6. Give some thought to the problem of users who stopped being active entirely, and would not reply, even at a delay, to messages regarding their old questions and answers. Especially w.r.t. the choice of an accepted answer.

  7. I'm not sure this is a good idea, but it is conceivable that a question could have conditionally-accepted answers, i.e. accepted for a range of dates or versions or what-not. This is certainly not acceptable just as I've described it, but maybe someone can formulate it better.

  8. It may also be relevant to make some user achievements obsolete. For example, if I got a gold badge for C++ in 2008, and have not answered any questions since then, my demonstrated expertise is absolutely outdated. I'm not saying that this necessarily needs to be done along with Q&A out-dating, but it's another issue to think about.

  9. There should be a robust mechanism for overriding the marking of a post as out-of-date.

  10. Perhaps there should be two different indications: "May be outdated" and "Outdated". While both could be reversible, the gradation is probably useful.

5
  • 1
    I think your point 1. is a really good one: we encourage people to add context to their question, so encouraging the same for the answer seems like low-hanging fruit. I've sometimes seen Python-related questions where the accepted answer has been updated with different answers for different versions, demarcated with headings, which is one way of spanning multiple "contexts" in one answer, rather than needing a separate one for each.
    – Tim
    Commented Jul 27, 2021 at 16:48
  • Current A.I. are well suited to find which C++ version is a minimum requirement (aka: it sees a feature is being used that only appeared in c++17). I'd rather rely on an A.I. to tag C++ answers (which then will also fix all existing answers!) then hope that the author will not forget to add which version he is currently using (they will think it to be obvious that it is the latest version in many cases). Of course, the tag should be editable - so that people can fix any error the A.I. might make.
    – Carlo Wood
    Commented Apr 20 at 14:43
  • 1
    @CarloWood: "Current A.I. are well suited to find which C++ version is a minimum requirement" - I'm not so sure. Maybe for straightforward cases.
    – einpoklum
    Commented Apr 20 at 15:53
  • @einpoklum I work on a daily basis with Gemini and ChatGPT and know exactly how bad they are ;). But recognizing if features of a new version are being used is well in the range of what an LLM is capable of. Anyway, it doesn't have to an LLM perse. Lets for now assume this can be done and first concentrate on the ideal solution for sorting answers. Whether or not it can be automated seems an orthogonal problem that can be addressed later.
    – Carlo Wood
    Commented May 4 at 12:50
  • @CarloWood: "is well in the range of what an LLM is capable of." an LLM is entirely incapable of correctly parsing C++ code, which is undecidable even theoretically. But like I said, straightforward pieces of code could well be effectively parse-able.
    – einpoklum
    Commented May 4 at 13:41
10

I've seen all of the above describing outdated answers. The timestamp is usually my first clue, then I skim the answer and usually see something deprecated.

Some of the outdated answers can be of value to a developer stuck on a legacy system. Maybe indicate in the search result the age and version of the language it applies to.

2
  • I see the point but I would conjecture that their inability to move on to newer technologies isn't a burden we should have to bear explicitly.
    – Makoto
    Commented Feb 18, 2021 at 22:22
  • 6
    You make a valid point, but the situation is complicated by our standard procedure of closing duplicates. A lot of core questions are dinosaurs, with a mixed collection of ancient and newer answers. If a new question on the topic is posted, the OP has to work hard to show that version or best practice changes differentiate their question from the ancient dupe target. And the OP may not have the knowledge & skill to do that.
    – PM 2Ring
    Commented Feb 19, 2021 at 4:44
10

Wonderful!

I have a suggestion for how to categorize these types of accepted answers.

Outdated Accepted Answers (OAA)

There were five categories (as previously mentioned) I've changed the wording a little bit, and reduced the count to four:


  1. Isn't the best possible answer anymore
  2. Wasn't the best possible answer at the time
  3. Isn't a correct answer anymore
  4. Wasn't a correct answer at the time

or


  1. A better answer exists

    • Due to new tech or
    • Due to more thinking
  2. Accepted answer is wrong

    • Due to new tech or
    • Due to poor thinking

I think this or a similar kind of wording makes it simpler and clearer to determine how to categorize OAAs.

Here they are again, with some notes:

  1. Isn't the best possible answer anymore

    A better answer was found using new technology. It could still be the best possible answer for developers who use legacy technology

  2. Wasn't the best possible answer at the time

    A better answer was found later on (and this can be argued and voted upon)

  3. Isn't a correct answer any more

    Which means that it poses a threat to new developers, but could still work for developers who use legacy technology. (i.e.,

    • Causes build time errors
    • Causes run time errors
    • Causes very dangerous errors, etc.)

    Perhaps it was edited later on, becoming an OAA.
    Perhaps it contains a dead link.

  4. Wasn't a correct answer at the time

    • This one can and should be edited (because of a typo) or flagged (because it was invalid).
    • Perhaps it violates certain Stack Overflow guidelines.
    • Perhaps it is a hint instead of a direct answer.
    • Perhaps it is written in pseudocode, or something unexpected.

There may be more categories, but I think in actuality they will be very rare.

A few may be:

  • Perhaps the OAA is historical or has an excuse.
  • Perhaps the answer is correct, but only for new technology (and is an OAA for many developers who use the legacy technology)

(In the comments we can come up with some more 'rare' cases.)

4
  • Not fond of your new abbreviation. How can an Answer be "outdated" because it's too new for users with older technology? Also, I just engaged in a post using JodaTime, which was supposedly outdated when Java 8 came out in 2014.
    – Scratte
    Commented Feb 19, 2021 at 19:51
  • @Scratte I took the 5 categories from "How do we define outdated content?" in the original question, but made them clearer. The 4 cateogies would pair well with Machavity's answer about a a post notice. My intent was to make categorizing very clear. By doing this it's good to note that 'outdated' isn't the correct word to be used anymore. Which should make us rethink what the end goal is, since the title of the topic has the word 'outdated' in it. It's all beginning to sound rather arbitrary. (The rare cases were an afterthought.)
    – 0-1
    Commented Feb 19, 2021 at 20:20
  • 4
    I know the Question mentions outdated. It really scares me and the potential result of having a lot of content removed. Lots of systems are running on old technology. Not just 20 years old.. and it's still running, and people still face issues and want to find solutions.
    – Scratte
    Commented Feb 19, 2021 at 20:46
  • @Scratte Yes, I also saw you're comment below: I have those same fears that this will result in a lot of deletions. I agree with you, removal at this scale would be very very bad. The 'removal' solution would be worse than the problem itself, I hope we can get that through to the staff team. Answers shouldn't be deleted if they are old or suboptimal, because that's all opinion. Here's how I feel about an answer in a newer version of a programming language.
    – 0-1
    Commented Feb 19, 2021 at 21:04
10

I have a couple of thoughts on this - hopefully they're helpful!


Why do we have an "Accepted Answer" at all any more?

I may be in the minority here with this thinking, but I haven't seen it mentioned as of yet.

I could understand if Stack Overflow were still a one-off help resource like a support forum (which I'm guessing the original iterations came from) - in that case it makes a lot of sense to have a single accepted answer.

But to me and I think a lot of other people at this point, the value of SO isn't so much as resource for the person asking the question but moreso as a repository of knowledge created by people both asking and answering questions.

In that context, why do we bother with an accepted mark at all? At a very base level we really only need one measure of which question should be at the top; which answer is the most useful to people. And that is measured by upvotes. Maybe the OP's upvote could be worth +5 to give the answer they like the most a boost, but if they then abandon the question for the rest of time that little green check shouldn't be decided completely by them.

At the very least, the accepted mark could be removed for questions with community wiki answers (or maybe the question could be marked as a community question at which point the checkmark goes away).


Relevant Questions

Another common issue I see is that there is an old and highly upvoted question which answers a very commonly-asked question, generally in quite a specific way. Often users (new users especially) will ask another question similar to that question but often not identical. Those questions seem to be fairly often marked as a duplicate and closed - to the frustration of the user who asked it because they've already seen that question and it didn't solve their particular problem. They then might comment on one of the answers asking for a clarification in their particular case, to which they're told to ask a new question.... at which point I can imagine them just giving up entirely.

I don't have a fully formed solution here, but what about some sort system where you could ask a question about an existing question.

The issue with this is that it could get very confusing very quickly, and I could also see a really fine line between a question that has multiple related answers and a question that could simply be split into multiple different questions. But in many cases it could certainly feed into StackOverflow being a source of useful information.

Here's an example: How to check whether a string contains a substring in JavaScript?

The accepted states to use String.prototype.includes with String.prototype.indexOf as a fallback for older browsers.

This is a pretty simple question so it isn't too hard to have both of those in the same answer. But let's imagine that it's not. In that case, having all the extra information about "indexOf" isn't really answering the original question, but rather a more specific question "How to check whether a string contains a substring in JavaScript versions before Ecmascript 6".

Let's imagine a situation where there aren't two relevant and important ways of doing what the question asks, but 10. Having 10 different question and answers might be one way of handling it, but what is the likelihood that all of those will exist without getting marked as duplicates, and that people can actually find the right one for them? And let's be honest, if it were just one question with 10 answers it's pretty unlikely that people are going to get down to the 10th answer... unless all of the previous 9 didn't work for them for some reason.

If instead when I searched for it, there was one well-written question, with an accepted answer useful for the most recent version, and then a listing of other questions each with upvotes as well, I could see that being a really powerful tool.

This could come in the form of a brand new question, but with links to the old one prominently displayed - and could be shown directly in the old question.

Also, I do realize that there is a "Linked" and "Related" section in the sidebar, but neither of those do quite what I've described here. Linked seems to be mostly closed duplicates while Related only seem very tangentially related (maybe by keyword or question?).

Here's an example:

'Relevant Question' example

In addition to the section showing the relevant questions, it would be great if there were a way to move answers to these other questions - maybe even "promote answer" to new question, where it would retain all its points. That's definitely a strange concept for a Q&A site, but if we're thinking of SO as a source of info I think that it would be a way to make multiple relevant answers more accessible.


Points for old questions & edits

I realize that points aren't much of a motivating factor beyond a certain point, but I'm going to bet that there are still a bunch of users who are motivated by them.

Asking and answering new answers give you lots of points, but you don't really get points for doing work on old questions... +2 is pretty sad when a single answer can give you +1000s. Tipping the balance in favour of working on old questions & answers could be motivating to some users - especially if there's chance of more relevant & recent answers being promoted.

6
  • 2
    You aren't alone in thinking that SO is a repository and not one on one anymore.
    – 10 Rep
    Commented Mar 12, 2021 at 0:00
  • The conundrum at the start of the “Relevant Questions” section (question being closed as duplicate and then asking for clarification leading to suggestion that a new question should be asked) is ideally resolved by editing the question closed as duplicate. But this requires educating the users about this etiquette. Your proposal to split a question in “How to do X?” and “How to do X in version Y?” is interesting, but often the exact requirements are unknown for new questions, and determining them for all new questions may be tedious. Commented Mar 17, 2021 at 10:26
  • @SebastianSimon I don't disagree - but the way it is currently depends on both the people asking the question and the reviewers behaving in the best possible way; I obviously don't have figures but I've seen more than a few questions from new members where they simply give up and don't ask again - although possibly for other reasons. My thought behind splitting/related questions is to allow an approach where users can show that they did look for other options, as well as build a more useful legacy in terms of being able to follow from one related question to another. Commented Mar 17, 2021 at 23:04
  • Improving links in the sidebar is a good idea. Maybe something like - "questions related to earlier versions of this software"
    – MT1
    Commented Mar 23, 2021 at 7:03
  • Why did you misspell JavaScript? The question hasn't been in that state for the past 5 years. It is "Stack Overflow" (the last section, "Proper Use of the Stack Exchange Name"), not "StackOverflow". Commented Jul 22, 2021 at 17:31
  • 1
    Having accepted answers is super useful. It tells readers whether an answer, and which one, was most useful in solving the problem/question posed. The problem is not that accepted answers are a thing, it's that over time their inability to be curated by others (except for outright deletion) leads to incorrect information being presented as still accurate/correct. That problem could be entirely mitigated by simply unpinning the accepted answer from the top of the answers list.
    – TylerH
    Commented Jul 26, 2021 at 16:10
9

For [python] questions, the way I would like to see the information (at least on really common questions) is grouped by descending minimum version number. For example:

How to remove a string from the beginning of a string

3.9+

print("my string".removeprefix("my "))

3+

def removeprefix(s, prefix):
    if prefix and s.startswith(prefix):
        return s[len(prefix):]
    return s

print(removeprefix("my string", "my "))

2

def removeprefix(s, prefix):
    if prefix and s.startswith(prefix):
        return s[len(prefix):]
    return s

print removeprefix("my string", "my ")

Grouping the answer by minimum version would encourage someone to keep the answers updated and check from time to time that it's currently the best way of doing things.

It would be good if answers had tests and code review (and profiled alternatives where that makes sense), because on these really popular/common questions, Stack Overflow acts more as a software library of snippets (or a map of human intent to snippets of code through Google), not just as the unofficial documentation. So it would make sense for answers to have all the basic things done in package management, as in a minimum (and possibly maximum) version, maybe we could also list dependencies, so that there would always be a pip install <package you need for this answer> to copy and paste from on every answer.

I would also like a way of flagging the accepted answer if it becomes outdated, which would override the absolutist sorting of green checkmarks if accepted.

4
  • Mod flagging an outdated answer is not good because the mod may not be an SME. (most mods are really knowledgeable, though, but they don't know everything). That's why you can't flag an answer that is wrong as NAA.
    – 10 Rep
    Commented Mar 4, 2021 at 1:08
  • 3
    I like this idea. The only issue is that questions have version tags (e.g. python-3.9) but answers don't so the grouping would have to be manual or clever ML. Plus I am not sure if this imposed "grouping moderation" will go down well with users.
    – xjcl
    Commented Mar 4, 2021 at 19:43
  • I've observed this pattern quite a lot on Python questions, where the original answer becomes a kind of noticeboard for either the OP or other editors to update when new versions enable better solutions. I suppose a challenge is then who gets the credit - does the OP benefit in points if other editors put in effort to maintain the answer in their absence? Maybe enabling multiple "accepted answers" for different [ranges of] versions might be one solution. Of course, not all outdated answers are necessarily due to obsolescence because of new versions, though it may be a large part of it.
    – Tim
    Commented Jul 27, 2021 at 16:50
  • This is my solution too - it should apply to every language with versions (including C++ thus). It is sad that this answer only got 9 up votes; imho that proves that the voting system isn't working: "answers" that are posted shortly after a question is posted get a lot of attention and THEREFORE a lot of up votes. Very good answers that are posted much later get very little attention and therefore only a few up votes: the number of votes are meaningless when deciding what is the best answer.
    – Carlo Wood
    Commented Apr 20 at 14:31
8

I raised one of the questions you reference (How to deal with hugely upvoted, bad and outdated answers?).

I like the suggestions made elsewhere of not pinning the accepted answer, but in the case above, that wouldn't have made any significant difference.

The "bad" answer appeared correct (and had once been correct). As it was when I opened my Meta question, it wasn't broken enough that it didn't work, but it did present a security risk, and loading the correct version of a library is unlikely to be an obvious thing for new developers to check when debugging.

So I believe there does need to be some user- or mod- driven way to flag answers -ideally with a custom message- so a banner appears somewhere near the top of the answer and makes it explicitly clear there are concerns.

In the case above, something like

The information in this answer may be incorrect or dangerous:

jquerylatest is no longer maintained as of version 1.11. It does not include the latest security patches.

You may need to assess your own usage to determine if you are at risk.

The middle line being case-specific.

If an editor believes their change addresses the issue, I could see an argument for allowing them to clear the flag (or put it to a vote), but... IMO Anything that fundamentally rewrites an answer to that extent should probably be a new answer, with the old one kept around for historical reference [with the health warning].

One added bonus: If the warning is impossible to miss, people will skip past those answers automatically, meaning the answer sorting/accepted answer questions can largely be ignored.

As to applying the warning... I'd suggest 10k+ users? (Possibly with a vote and queue if that's not too much overhead).

If the answer owner disputes the flag, escalate to a mod to adjudicate.

3
  • We could already edit warnings into the answer body. The only difference here would be that the answer creator cannot remove it that easily and it's positioned above. However, I hope that many bad answers already have warnings in the text. That would be the direct way of improving them. Commented Feb 22, 2021 at 6:43
  • 1
    @Trilarion Every single warning would be inconsistently formatted/placed. Why not have something clear, unambiguous and impossible to miss, rather than relying on every editor to get it right every time? Ideally with an icon and standard layout. As to allowing the OP to remove it... I'd be happy to have a "This addresses the out of date problem" checkbox on the edit that removes the flag.
    – Basic
    Commented Feb 22, 2021 at 9:29
  • Basically we could take all the highly upvoted comments for each answer and put them above the answer. Your proposal is more specialized on warning/issues with a post, but there is also additional overhead. We would probably need new review queues for quality control of additions and removals of such warnings. If we had just some conventions and additional html elements (like a warning element, that is automatically placed at the start), we could already do it within the answer bodies. It's a good idea, we just should discuss it more, I feel. Commented Feb 22, 2021 at 9:51
8

Replace/Split the question

Many people have raised good points with the troubles of addressing content in old answers, but I'm thinking more about when there are multiple valid answers that depend on context (such as version, environment, etc) -- and importantly, that the context can change over time, which changes the "best" answer.

My suggestion is: just allow a new question to be asked. There can even be tools to split/duplicate the question and move some answers. The important thing is linking between these questions in a special way (similar to how duplicates work today) indicating there are other variants of the question that depend on context. The questions also get some type of qualifier to distinguish them:

  • "How do I do x?" becomes "How do I do x in Foo v5?"
  • and we get a new question: "How do I do x in Foo v6?"

It's still a judgement call of how exactly to qualify the questions ("in version x", "that is compatible with very-old-OS", "using the non-async way"), and at what point the answers are sufficiently different to warrant a split.

This might even depend on the answers themselves: for example there's a huge swath of web-related questions where 10 years ago the only practical answer was some variation of "using jQuery, do x" but today it's not needed, and these old questions could get "..with jquery" added as qualifiers.

Having new fresh questions also gives more (newer) users the chance to answer, which is an excellent way to learn and a way I personally used to derive a huge amount of value from the site.

The old question and answers still exist, but ideally over time search engines and site search finds the new, up to date answer first -- or at least provides cross-links.

6
  • 6
    Nice solution if ALL answers on a question are outdated. Doesn't work if the accepted answer is outdated but another answer already shows the modern way.
    – BDL
    Commented Feb 26, 2021 at 9:26
  • 1
    Not every old question is outdated. Without a very strict versioning system I don't see any benefit in duplicate orgies and people answering the same (new) questions with the same answers again and again. The difficulty would be how to detect if a question is genuinely new. However, it would mean less curation activity, so maybe there is something to it. To me it feels too radical. Commented Feb 26, 2021 at 9:29
  • Why would a question be outdated? If somebody asks "how do I frob an array?" in 2010 the question is still valid in 2021. If there are new frobbing techniques that come out, then those can still answer the old answer. We don't need to do "How do I frob in 2010?" followed by "How do I frob in 2011?" etc. Especially if the old answers are still valid. Java 8 introduced the Stream API so you could do an iteration over a collection with it however, a regular loop still works and is not in any way "worse" or "outdated".
    – VLAZ
    Commented Mar 2, 2021 at 0:51
  • 1
    Also worth noting that people can ask about old things today. If somebody is stuck maintaining an archaic code base running on Windows 95, their question now is not "outdated". It's very much up to date because that's a concern they have now. Presumably it has not been addressed before on SO, either.
    – VLAZ
    Commented Mar 2, 2021 at 0:54
  • I've modified my answer a bit to focus more on "splitting" and context-specific questions, especially to deal with what @VLAZ notes about "asking about old things". I don't think it's the question that becomes outdated, but it's the context. This is really an alternative way to manage "multiple valid answers where the best depends on the context, which changes over time".
    – gregmac
    Commented Mar 3, 2021 at 4:46
  • 1
    @BDL: In the scenario you describe, the question can be split up instead of modified.
    – einpoklum
    Commented Mar 4, 2021 at 13:56
5

I have a few ideas regarding this issue

  1. An indication for the answer with most recent upvotes.
  2. A new sort order like "Upvoted recently".
  3. Reducing the size of the accepted check mark to something like the timeline icon size.

Note: Please don't remove the green check mark. Not all accepted answers are outdated & many are still useful. I have seen instances where the accepted answer is a heavily downvoted answer but it helps me with my problems.

6
  • 2
    Nobody is suggesting to remove the green checkmark. There are suggestions to unpin it, which means that the accepted answer with the checkmark will no longer automatically float to the top of the list of answers, it will be sorted just like the rest of the answers. There are also suggestions to modify the colour of the checkmark on outdated accepted answers, and / or to add some other kind of visual indicator on (prominent) outdated answers.
    – PM 2Ring
    Commented Feb 24, 2021 at 5:13
  • 2
    @PM2Ring Ref: meta.stackoverflow.com/a/405309/5407188 The top most upvoted answer in the related question suggests removing the green checkmark. Thought that I would mention it in my answer.
    – Kolappan N
    Commented Feb 24, 2021 at 7:34
  • 2
    Oh, ok. Adrian isn't actually saying that he thinks that the green checkmark should be removed in some cases, but he is suggesting that it would be good to get some feedback from the community on that option. So it's perfectly fine for you to give your feedback. From what I've read here so far, the consensus appears to be that giving non-OPs the ability to completely remove the checkmark would be too radical a change of a fundamental Stack Exchange component. But we generally do want some new way to indicate that the accepted answer is outdated (or may have never been particularly good).
    – PM 2Ring
    Commented Feb 24, 2021 at 11:33
  • @PM2Ring All of the options he suggested says about removing the mark or reassigning it. None of them says about leaving the mark as it is, to indicate that it helped the asker.
    – Kolappan N
    Commented Feb 24, 2021 at 13:34
  • 3
    "instances where the accepted answer is a heavily downvoted answer but it helps me with my problems." How does a downvoted answer that also happens to be accepted help you with your problems? The only purpose that I can imagine is acting as a negative example. Kind of: however you do it, don't do this, even though one guy thought it's a good idea. Commented Feb 24, 2021 at 19:13
  • 2
    @Trilarion It was a rare case. As far as I can remember, it has something to do with configuration and people downvoted it because it was not working. There was even some comments that the answer didn't work. I had the exact problem as asked in the question and it worked for me. If not for the accepted answer mark, I would never have tried a solution in a downvoted answer that people have commented as not helpful. I know it was a rare case, but still having an indication that an answer is marked as solution by the asker is something that is useful. I only request to not remove it completely.
    – Kolappan N
    Commented Feb 25, 2021 at 3:51
5

How about adding version tags to answers, a min and max tag and some specialized tags like security risk, undefined behaviour

  • min works for this and after
    • a new feature is required
    • changed behaviour makes this only work from this version
    • no value if it works in the "original" version
    • auto filled with the version the OP added if available
  • max works for this and earlier
    • deprecated after this version
    • change behaviour after this version making the answer wrong for later version.
  • security risk
    • buffer overflows
    • all the other nice things static and dynamic analysers could tell you about your many faults.
  • undefined behaviour
    • it is not defined what the result should be from the language or product

Edit of these must happen in a comment so everyone can see why. This means everyone can add these so some form of voting by upvote the comment until some limit is passed.

Example:

  • Gold
  • Silver + 1 upvotes
  • Bronze + 3 upvotes
  • none + 6 upvotes Including the OP and upvoters. So if an none suggest the tag a gold or silver upvote will make it happen

This should show which answers are good for which version or just bad.

A lot of my answers are pseudo code or concept code so feel free to add those as tags too.

10
  • Wouldn't your min and max tags mean needing to add many hundreds of version tags to the system? I assume you mean min-someLanguage-1.4 , max-anotherLanguage-4 for example
    – charlietfl
    Commented Feb 28, 2021 at 14:53
  • @charlietfl Depends on what granularity you would want. Many already have major versions like python2 or python3.
    – Surt
    Commented Feb 28, 2021 at 16:29
  • OK , just pointing out that your min/max concept needs more detail due to different interpretations of what you are proposing.
    – charlietfl
    Commented Feb 28, 2021 at 16:31
  • 2
    @Surt not all languages have a clear cut "version". With C++ you might have something specific to certain compiler or certain version of a compiler. With JavaScript you can have browser-specific behaviour like something that shows up in IE11. With JS in general standard support is not completely universal - modules were introduced in ES2015 but browsers started adding support in 2017 while Node.js added experimental support in 2019. Then we also have weird scenarios where something works in version n, doesn't in n + 1 but works again in n + 2 or similar because it's briefly broken.
    – VLAZ
    Commented Mar 2, 2021 at 0:42
  • I'm not opposed to something like a min and max - I'm pointing out the above to show potential pitfalls. I'd prefer something more streamlined to avoid the whole mess but I'd like some sort of marking for "this works here". I'd also like to point out that this shouldn't cause a big maintenance later on. Because it's another potential problem - having to review and update any answer with "max = n` to max = n+1 for any new n that comes out is going to be untenable in the long run.
    – VLAZ
    Commented Mar 2, 2021 at 0:45
  • @VLAZ no need to update or rather add a max before it is relevant.
    – Surt
    Commented Mar 2, 2021 at 10:21
  • 2
    1. Versions of what? Do you want SO to maintain a table of all versioned entities used in answers. 2. People often don't know what the minimum version is, and can rarely tell what the maximum version will be.
    – einpoklum
    Commented Mar 4, 2021 at 13:58
  • @einpoklum there are already a lot of tags, C++11, C++14, C++17, C++20, same with python and some others, users who feel an answer should be tagged and no appropriate tag exists could make/suggest one.
    – Surt
    Commented Mar 4, 2021 at 14:07
  • @Surt: Those are just 4 entities. Now multiply that by the number versions for each library, and the number of popular libraries, and the number of platforms, and the versions of each platform etc. etc.
    – einpoklum
    Commented Mar 4, 2021 at 15:50
  • @einpoklum The libraries have their own tags if needed, some of which implicit requires some tag of the base language so you dont need a tag lib7python3 but instead lib7 (which implicit requires python3) or if both are needed two tags should do it lib7 and python3, so no explosion of tags are needed.
    – Surt
    Commented Mar 4, 2021 at 16:10
4

As some already pointed out that there are issues around handling duplicate questions.

It should be considered if a new question can be closed as a duplicate of an old outdated question.

The problem is probably with classifying what is outdated. Some old answers do get updated as technology evolves. Some old questions do get useful answers for newer technology as well. Often updates in the technology stack allow simpler solutions as they get addressed in newer SDKs.

4

I already outlined my suggestions to fix this in the question I raised here: Mark questions or answers as out of date?

  1. Mark answers or questions which are for an older version (i.e. no longer the correct answer)
  2. Allow multiple accepted answers and specify versions via tags per answer
  3. Specify some sort of "applies to versions" tag
  4. Be able to search for / filter answers based on any of the above suggestions
3

Personally, it would be helpful to me to be able to collapse all of a question's answers and see something meta about them all.

For example, I'd like to see sparklines (see MS Excel) that show both upvote and downvote trends for each answer. Something like this would let me more easily understand which question is more relevant now.

May also be helpful to visualize edit points with that data, but probably not for the initial MVP work.

1

One way of approaching this would be to ignore the accepted answer status when sorting answers for display. That is all answers are sorted by their vote, if an accepted answer has fewer votes than another answer then it is placed below that answer.

Site users are now empowered to make the decision on what is "accepted" or not.

Second there should be a mechanism to flag an answer as outdated without removing it completely. Outdated answers should be placed lower in the sort order (perhaps in a clearly demarcated outdated answers section).

The users should be able to flag accepted answers as outdated too. Thus pushing it down into the outdated answers "section".

10
  • 1
    Point 3 (your "second") combined with point 1 is about where I think the answer lies. Some set of flags/tags for outdated answers, or questions, with sorting/visibility attached to their use.
    – Chindraba
    Commented Feb 18, 2021 at 18:29
  • Isn't downvoting already the mechanism by which we "flag an answer as outdated without removing it completely"? That affects the sort order as you're suggesting.
    – cigien
    Commented Feb 18, 2021 at 18:29
  • 6
    you can't rely only on sorting answers compared to their vote count as old answers would stay with higher vote count even though they are outdated because newest answer may not have the same activity now compared ot before and may be harder to find. However I agree with the flag mechanism.
    – Titouan
    Commented Feb 18, 2021 at 18:30
  • 7
    The problem with flagging outdated answers: who will handle those flags? Moderators can't be SMEs for every language, so it has the same problem that inaccurate answers do. Commented Feb 18, 2021 at 18:51
  • 2
    @JohnMontgomery Those could potentially go into the H&I queue to have someone edit in the relevant version numbers to the answer (ex. "This answer is valid for versions 2.xx - 3.xx" or "No longer works as of version 4.0.") if that's all that's wrong with it.
    – BSMP
    Commented Feb 18, 2021 at 19:15
  • 1
    @JohnMontgomery it could be a new review queue that silver/gold badge holders for the tags in the question have access to. Doesn't have to be diamond mods doing the work. Commented Feb 18, 2021 at 19:23
  • 1
    I don't think outdated answers should be pushed down per-se, but could just be a new notice that's added to the top of the answer once it has been marked as outdated. A different notice could be displayed based on the five flavours mentioned in the question - "This answer has been marked as containing security vulnerabilities", "This answer applies to an older version of FooBar", etc. Commented Feb 18, 2021 at 19:26
  • @JohnMontgomery well the users. You can just create a new queue for people to review the outdated request.
    – Titouan
    Commented Feb 18, 2021 at 19:27
  • 1
    I don't know if we need a whole "outdated" section, and that would likely complicate the mechanisms for sorting answers. But some kind of visual flag could work. It probably just needs to be as visible as the accept tick, maybe in the same location, or near it.
    – PM 2Ring
    Commented Feb 19, 2021 at 4:50
  • 2
    @Bsmp I like the idea of using the H&I queue. Instead of adding another queue, just use a dead one.
    – 10 Rep
    Commented Mar 4, 2021 at 4:14
0

I am glad to see the possibility that the issue may be resolved.

I hope to provide some food for thought to the team dealing with this, rather than a solution or answer.

Herein is a random collection of thoughts, meant as feedback on the project goals. There is no intention of providing any developed plan, or plan of action. First, it's too early in the process to even do such, and secondly, I'm not versed enough in the back-end, or site mechanics to be able to propose

The "Grand Gesture"

This question has already disclosed the idea of handling outdated content. Therefore, there exists nothing to be gained by doing all the exploration, design, development and implementaion in secret and pulling a bif reveal as some kind of "Grand Gesture". Just work on it, keep users in the loop, post an update once in a while. It'd even be nice to know, once or twice, of an idea that was tried and failed - prove the team is willing to try, and willing to learn.

Keep it under control

Make small steps. Do things that probably have minimal impact, then build on them when they don't fail. Be ready, and know how, to roll back each step if it doesn't work as planned. Small steps, with small impact ought to also yield easy to monitor, and classify, metrics. The metric can help decide if the step was the right one, and to show progress to the users.

Leverage the power of the masses

There are a bunch of users here who are working on keeping the site clean. There has been a lot of posts on meta about the issue, as the question has sampled so generously for us. Find a way to allow interested users to opt-in at all phases. During discovery phases there is even the possibility that many users would be willing to work with a less than perfect UI, just to get this rolling. There are probably enough stats in the collected database on users to know which users are active in the curation for any tag, or group of tags. Use that data. (More on tags below.)

Reducing the workload

Reading, and thinking about, the list of flavors given, I think some of them can be passed over, or reduced.

The question listed different "flavors" of outdated answers:

  1. become obsolete as new versions of frameworks become available.

  2. still work, but there is now a newer, better way to achieve the same end.

  3. maybe were never the best, but the question askers accepted them and forever bestowed the green check mark.

  4. now pose security risks or provide code that no longer works.

  5. still hold value to a subset of developers who use legacy technology, but aren't valuable to developers on newer technologies.

maybe were never the best, but the question askers accepted them and forever bestowed the green check mark.

The third group is the easiest group to deal with.

Do nothing.

Such answers are not, based on the given criteria, "outdated". They may not be the best solution available, with or without competing answers. Comments and voting resolve the issue sufficiently, as they were designed to do. These answers may fall into a different group as well, and can be dealt with in the fashion of that group. (This presumes that the team doesn't decide that the "green check mark" is no longer a sacred cow and chooses to change something in how that works in a unilateral fashion, which I will touch on as the final part of my answer.)


still work, but there is now a newer, better way to achieve the same end.

The second group is almost the same as the first group, except that there is no check mark to deal with. If the old answer still works and a newer method has become available it is not an outdated answer, it is merely no longer the "best" answer. The solution is to write a new answer, showing the "new" method.

As an example of both the second and third groups in one case, see how things went on this question where the original answer(s) presented a simple, perhaps less than brilliant, solution. Later, 3 years later, the OP accepted a newer answer with more information, better explanations, sample codes, newer methods (including upcoming method in a newer version), and a significant amount of work. As I am unable, or don't know how, to see which answer(s) were accepted before the current one, I don't know how good, or bad, the first accepted one was, nor if there was a switch of accepted answer prior to the latest version. One comment suggests that the current accepted answer is not the first one accepted. As a case study, without the OP changing their acceptance or with the unpinning of accepted answers, it would still be the second answer in vote order.


now pose security risks or provide code that no longer works.

I'm doing a hand wave on the second half of the fourth group: "provide code that no longer works." I suspect that code which no longer works does not actually exist, without falling into the first or last group. If it worked in the past, and updated frameworks, languages, or products have removed it, it should still work in, at the minimum, the version in service at the time it was written. The very existence of Retrocomputing more or less proves that nothing in computers is ever "gone." The first half, security risk, remains to deal with.


become obsolete as new versions of frameworks become available.

still hold value to a subset of developers who use legacy technology, but aren't valuable to developers on newer technologies.

I'm considering the first group and the last group to be functionally equivalent. Legacy technologies and obsolete versions of frameworks are two phrases for the some concept, which includes older versions of languages, operating systems, editors, and anything else the site scope covers. Any answers from either group are operationally the equivalent in the context of cleaning up the site's collection of outdated answers.


So, in context of this proposal, there are only two groups do deal with:

  1. Answers which now, or even when written, pose security risks.

  2. Answers which have become obsolete as new versions of frameworks, languages, or other technologies become available, which still hold value to a subset of developers who use legacy technologies.


Tag grouping

Don't try to handle the entire project as a monolithic issue. Handle it as a collection of modules. Find a tag, one tag, that is likely to have issues with outdated answers. A language, a framework, or a product. If it has a fast release cycle then it's probably a good candidate for research. Take advantage of users with a significant amount of rep related to that one tag who are identified as active in curation in that tag. The rep level need not be gold-badge, or even bronze-badge, though it could be. Allow the users who have opted in to use something like the flag system to mark answers, in that tag, which are outdated. Collecting why it's outdated is pointless in discovery - either it is, or it is not. The same process can be applied to the search for unsafe answers, in a tag determined to be likely to have such.

The same process can be used on multiple tags at the same time, as long as the tags are unrelated. Treat each tag as its own project, unconnected to any other tag's process. Track metrics related to participation of the users, how often they actually flag a question relative to how often others have done so. Low participation? Find another tag. Good participation and low flag count? Probably a good tag to use in later testing. Good participation and high flag count? Mark that tag as one to get attention when the process is working, mostly.

Post notices

User edits, gold-badge or not, of old answers can lead to many problems. Edit wars being only one of them. Meta has plenty of examples, and I'm sure that others here, especially the team working on this, has seem more than I have, so I'll not bother linking to more than one. A post notice is less likely to be involved in a tug-of-war and has the benefit of being much more visible than what a user can do with edits. A post notice added to an outdated answer need not have any pointers, links, or other information in it - merely a notice, or warning, that the answer is outdated. One point to keep in mind about outdated answers is that they are still valid as long as the version of the associated technology to which they apply is still in service somewhere. The user base is probably loaded with people tasked with maintaing legacy (or Paleolithic) systems. Look what happened recently with the sudden increase in demand for COBOL developers.

Unsafe practices post notices

Answers offering unsafe practices, or code, as a solution to the question also ought to have a post notice. Importantly, in this case, the post notice needs to be distinct from other system-wide notices. These are, after all, serious warnings, not just advisory notices. They need to be harder to ignore than all the other post notices we have gotten used to skipping over when the page loads. Color change would be enough. Additionally, these notices need to have something in them to "validate" the warning - a link to an SO answer which explains it well, or the product's official notice of a vulnerability, etc. Close reasons, not matter how well worded are still based on an opinion. That the majority of the active users consider "Shopping-list" questions as POB is good enough for the site to close a question. It is still just the opinion of said users. A security warning, if one is going to be posted, needs to have objective, verifiable classification as a risk.

As an extra consideration, the wording of, and link(s) within, the post notice for unsafe answers should be written by, or at least under the strong guidence of, on-site SMEs, gold-badges perhaps.

Our "Sacred Cow"

Accepted answer treatment is a tradition here. I've not followed the debates about that here, and I'm sure there is plenty of it. I do know there are many supporters of the current status, and some who treat it like a religious war. For the most part I don't see the need to change that. "For the most part." I can envision that unpinning the accepted answer, when and if it is determined to be outdated, allowing the sorting to act like a self-answered question, might be helpful to future readers of the answer list. The psychology of voting on SO seems to favor the accepted answer, even when it is a short, low-quality answer with other, often multiple, well-written, well-documented and much better answers in the list. In the majority of cases, I doubt that unpinning the accepted answer would change the sort order anyway. I do think, however, that if the accepted answer is unpinned, either for being outdated as above, or as part of some other major overhaul, that the question be marked, prominately, as solved and include an highly visible link directly to the accepted answer.

0

You could easily increase visibility of potentially outdated answers by showing full year in the timestamp.

Lately, I often find myself wondering whether the answer I'm reading is still up to date. A good indicator of it is when it was posted. Sadly, the date format Stack Exchange currently uses is bad, which was already mentioned at least once.

Here's what we have now: enter image description here

Here's what would make me know what I'm looking at with just a glance:

Enter image description here

I'm not even asking to change the default to ISO 8601, nor to make this a user-configurable option, just make the actual year visible without hovering over it. It's not much, but it's honestly helpful.

3
  • 2
    This doesn't do anything to increase visibility at all. IIRC the date is displayed based on the user's region preferences (could be wrong though), but regardless, hovering over the date will always show the date in YYYY-MM-DD HH:MM:SSZ (UTC) format.
    – TylerH
    Commented Sep 24, 2021 at 15:32
  • 3
    @TylerH: Re "the date is displayed based on the user's region preferences ": No, that was never implemented Commented Sep 24, 2021 at 17:11
  • 1
    Yes, it does increase visibility, although indirectly. It's more that it increases the visibility of new answers. When I see 2020 clearly under the answer, I can trust that it's relatively new. When I see it's from 2013 and the library it's concerning had its first full release in 2014, I know to be careful about it. Having full date on recent answers (in addition to "2 days ago" for example) instead of omitting the year and implicitly saying that it's from the current year would be nice too.
    – Harvastum
    Commented Sep 26, 2021 at 15:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .