-34

Preface: I know this is going to be quite a controversial opinion and problem, however it is worth continuing the discussion on how to deal with this.

I've used Stack Overflow for a while now; even more before I ever made an account, probably like a lot of you. One thing I think that could be done better is calling attention to deprecated / unsafe / inefficient code in answers that you end up finding from a Google search. I think this could be done better with the help of automation and allowing our community to work together to tackle the issue as one.

I propose that Stack Overflow help developers across the word improve their time and lessen their mental burden by allowing the community to flag answers containing code as deprecated and unsafe. These flagged answers are not meant to add more work to the moderation backlog. They are meant to be picked up by workflows. Before the flag can be submitted, the flagger must choose the language of the code and the version it was written in. These flagged answers once they obtain enough flags can automatically run through automation workflows that use a interpreter with the language and version that was selected after finding the most common language and version chosen in the flags. The workflow can then test the code on one or two interpreter versions ahead to check for errors and or deprecation / unsafe warnings from that language's linting or auditing tool. If errors or warnings are found then we know the code is old. Stack Overflow can then automatically add a yellow outline around the answer indicating that the answer contains deprecated or unsafe code. This answer can then be further de-prioritized and left for the author to improve.

A prime example of such a can be seen here: How do you create an incremental ID in a Python Class

The highest rated answer is using Python 2 after being called out in a comment. I would bet there are many other questions out there with bigger answers where developers have to spend time reading through a longer answer and or actually running the code only later to find out that it's old code.

If you have skimmed through this or read up to here, I appreciate you. Any comments or criticisms are thoroughly welcomed! I wrote this up during a frustrated work break to vent. <3

enter image description here

10
  • 12
    To whom would this be "reported"? Do you realize that answers are not necessarily expected to contain fully runnable code, and that "interpreters" don't exist for all languages? If you want to mark an answer as needing improvement by the author, simply downvote it and, optionally, leave a comment. Dec 2, 2021 at 5:30
  • 4
  • @CodyGray the flags are not meant to add to the moderators queue, they are meant to be taken care of by automated workflows. I realize that a lot of answers are not fully runnable. However you can use logic to find commonly used built in function names in the Answer's code to test those functions by themselves. SurajRao, Thank you!
    – Ian Smith
    Dec 2, 2021 at 5:34
  • 7
    What if there are no function names, it's just a snippet? At any rate, I am not familiar with any such tools for the vast majority of technologies covered on this site. Can you give examples? What "automated workflow" would you use to determine that an answer was recommending the C++98 approach, instead of the C++17 approach? What about in assembly language? If the tools don't already exist, who would build and maintain them? And what about the case where someone is specifically using older versions for compatibility reasons? Dec 2, 2021 at 5:41
  • I'm not a C++ guy, so I can't really comment on that. My high level view of this is that it's targeted toward very popular pages, with Answers that have 50k+ views where the code in question is not in a grey area. AWS' CodeGuru is an example of a technology that is handling this at scale for companies. StackOverflow could consider building something similar. As time goes on I think the developer community could improve the method in how these workflows would determine old code.
    – Ian Smith
    Dec 2, 2021 at 6:11
  • Those are quite rare, would an automated process even be needed for such cases? I imagine someone with enough motivation could dash through a few dozen of those pretty quickly. Even easier would be to do a code search for known old+bad code.
    – Kevin B
    Dec 2, 2021 at 6:32
  • 15
    "I'm not a C++ guy, so..." Well, yeah, that's kinda the whole point. You appear to be assuming that the entire world of programming is the small sliver of universe that you know intimately, but that's not the case. Stack Overflow is much broader than a single programming language, and even in the cherry-picked case of Python 2 -> 3, it's not so straightforward as you make it out to be. Dec 2, 2021 at 9:35
  • <Insert a reference to a Your Common Sense post> (the closest I could find was Should the answer be the simplest ever possible, even at the expense of quality/security?) Dec 2, 2021 at 16:09
  • 2
    "Are you tired of running into deprecated code in the highest voted answers on Stack Overflow" - no, not really. Wisdom dictates that you look at all answers. Research takes time, it's just the way it is.
    – Gimby
    Dec 3, 2021 at 9:14
  • Here is an example of an answer (approx. 2500 upvotes) that is (apparently) not being updated. Apr 11, 2022 at 19:42

2 Answers 2

31

The idea that Stack Overflow would maintain automated tools that test code and check it is, quite frankly, absurd.

The top 10 tags are:

  1. JavaScript
  2. Python
  3. Java
  4. C#
  5. PHP
  6. Android
  7. HTML
  8. jQuery
  9. C++
  10. CSS

Each of these has a variety of versions available. And a variety of tools to test the code for what is good or bad practices. The code can also run on a variety of platforms - different operating systems, or perhaps different browsers for the web languages.

Just because I am more familiar with the subject matter: JavaScript has 11 "versions" out. These are the various ECMAScript specifications: ES1, ES2, ES3, ES5, ES6 (a.k.a. ES2015), ES2016, ES2017, ES2018, ES2019, ES2020, ES2021. That means that code should be able to be checked against 11 different things. Plus one more every year as the new specs come out.

Furthermore, the versions do not actually help. There is a lot of code that does not conform to a version because it conforms to a proposal. Features for the ECMAScript spec start out as proposals and then after being ratified are added to the main specification. There are environments and tools that do allow you to use a feature from a proposal which is not officially ratified yet (but presumably on track to be, eventually).

Aside from the problem of "what is valid code, even" there is an additional problem to figuring out "what is invalid code, even". There are three major linting tools for JavaScript code: JSLint, JSHint, ESLint - which one should the automated workflows use? Moreover, once they pick one what configuration do they use with it? Very simple code example if (a == null) return; - this is valid (not syntactically wrong) code but is it correct? JSLint says "no" because it wants you to use === instead. ESLint has the eqeqeq rule to determine whether you need == or ===. Usually it is recommended that you use strict equality with === however, that can introduce a bug with the code here. This is a real thing I have done - I installed a linter, it told me to change == to === and it broke some functionality. Because loose equality was intended when the code was written.

This is just a very, very simple example of just one thing. For one language. And that language has 11 different versions out there. It is ludicrous to expect this can all work uniformly for all JavaScript code past and present.

There are other languages to consider as well. HTML and CSS in particular are not exactly "runnable" in the same sense as JavaScript, Python, or Java are. They can be analysed via static checking tools but not really checked if they "work". How do you verify <input type="text" /> does what it is supposed to do? It is valid HTML but what if the question asked for how to make a button - the correct code would be <input type="submit" /> or <button></button>. Or it might even be <a class="btn"></a>. Which one is it? Are any "unsafe"? How do you automatically check for it? CSS is even more elusive without some HTML elements to style - a CSS rule could be valid but "harmful". Like turning all text green on a green background instead of only some of the text green on non-green background.

All of the above assumes you even have code that is complete in some way. Here is some code from an answer of mine (for the record, it is Java):

return LongStream.iterate(0, 
    nextYear -> ((nextYear + amount) * (100 + interest)) / 100
)
.skip(years)
.findFirst()
.getAsLong();

This is the entire code block. No tool I am aware of will be able to run this. Neither do I know of an automated process that can even make it runnable since amount, interest, and years are missing. In this case it is at least somewhat easy to assume they are integers (other types do not much sense for the code), however, a lot of stuff is still missing. It would also be, to my knowledge, impossible to verify if the code is correct even if it was somehow converted to be runnable. As a side note: Java has 21 major versions out - that partial code should somehow be able to be automatically ran in all of them.

It is time to talk about the automated running process. As shown, there is a lot to consider. But somehow the automated process has to be able to not only load the code and run it thus proving it is syntactically OK. You also expect the automated process to test the code:

However you can use logic to find commonly used built in function names in the Answer's code to test those functions by themselves.

No, bluntly you cannot. Sure, it is possible in theory but to my knowledge there is no way to generically create tests out of source code. Here is a very simple example in JavaScript:

function a(x, y) {
    return b(y, x);
}

function b(x, y) {
    return x * y;
}

a calls b and b uses the arguments that are passed into it. To generate a test, you need to:

  1. Find that a and b are functions.
  2. They expect two arguments.
  3. To call them with two arguments.
  4. The arguments are of the correct type.

The above example is exceptionally simple and there are tools that can statically analyse the code and can determine that the arguments should be numbers, which is correct. However, if * was + then it static analysis cannot reveal what x and y should be in b and consequently in a because + could apply to numbers or strings. Or a mix. Is a("world", "hello") the correct invocation or a(2, "4")? Or is one of these a bug? Maybe both. You expect a lot out of automation that it cannot deliver. The problem just gets larger and larger with each language, versions, environment, etc. That has to be considered.

1
  • 1
    And don't get started on the batsmurf insanity that is a C++ parser. Dec 2, 2021 at 18:01
5

Sure, I'm sometimes frustrated by sorting through outdated answers, but as a developer, the responsibility to figure out what works currently and safely is on me. There are ways the community can help guide people: commenting to say that something is outdated and suggest a way to update the answer, posting new answers with updated code, or commenting or downvoting bad practices.

I know there's a tendency for people to blindly copy & paste from answers on SO—I've seen people go as far as to start a question with "I'm not actually a programmer, I just copy & paste" or to post a question because they copied code but didn't change the variables and are now surprised that the code doesn't work in their project.

But what you're suggesting puts the onus on people and systems other than that developer—on the community to constantly check that answers still work as they are, on some other set of developers who would build automation tools (to check what, exactly?), on linters for every framework of every language, on semantic versioning...anyone but the developer who's relying too hard on SO answers and not enough on building up good practices in their field. That's without even getting into the logistics of how quickly code can change within just one language, let alone across many popular and obscure ones.

The only thing I could imagine being reasonable and useful like this would be similar to how some news sites will automatically put a little box saying "This article is more than xx years old" at the top of an article. That doesn't mean the article is irrelevant or invalid, and it isn't meant to do any more than remind you to check the dates and relevance of things you read and what you rage-post, but that's a bonus, not something to be expected of news outlets. The responsibility still lies with the reader.

3
  • 4
    Re "I just copy & paste": Hence the full Stack Overflow developer - "Full Stack Overflow developers work almost entirely by copying and pasting code from Stack Overflow instead of understanding what they are doing. Instead of researching a topic, they go there first to ask a question hoping people will just give them the result." Dec 2, 2021 at 15:53
  • 2
    The more general term is Cargo Cult Programmer. Screw Stack Overflow, these folk will copy from ANYTHING. Dec 2, 2021 at 18:04
  • Uncle Bob zealotry is also a form of cargo cult programming. Dec 4, 2021 at 13:20

Not the answer you're looking for? Browse other questions tagged .