48

A pet peeve of mine on Stack Overflow and in real life is when someone seeking help says, "it's not working" or "I get an error." These are useless descriptions that always require a follow-up question like "How does what's happening differ from what you expect?" or "What's the exact error message you're receiving?"

I know there are other heuristics in place to detect bad questions and prompt the user to improve; might it be possible to detect when someone uses a nebulous term like "an error" or "not working" without elaborating?

Clarification:

I agree with what Ed Cottrell says below that false positives are a big possibility. However, we wouldn't necessarily need to block the questions so much as provide a prompt:

We see you've mentioned receiving an error; in order to receive the best answers to your question, make sure you post the full details of the error you've received.

4
  • 32
    "Hi, I'm Bubbles, your Stack Overflow assistant. Would you like help describing the error?"
    – user4151918
    Commented Feb 6, 2016 at 5:31
  • 4
    I think even if we had question wizards that guide posters through the various parts questions should contain, they would simply click to the end. It's kind of Murphy's Law: if anything can be ignored it will be ignored. Commented Feb 6, 2016 at 22:28
  • 8
    @GertArnold That may be true, but the fact it was ignored makes it that much easier to tell people making honest attempts vs. those who don't. People who blatantly ignore advice thrown in their face about posting a good question probably aren't making an honest attempt, which means less work from everyone else deciding what to close and what to leave open.
    – jpmc26
    Commented Feb 7, 2016 at 5:57
  • meta.stackoverflow.com/a/316142
    – Robert Harvey Mod
    Commented Feb 10, 2016 at 16:10

5 Answers 5

38

tl; dr this probably can't be done in a useful way (high signal-to-noise ratio), would be very difficult to do, and is probably not a good idea. While I like the idea, I don't think it's feasible.

This is awfully hard to implement because English is a difficult language to parse.

What about these examples?

  1. I get an error and it doesn't work. should probably be blocked.
  2. I get an error in my log file "invalid syntax something." Maybe; depends on the something and other details.
  3. I get an error 3 times, but the fourth time it works. Close; depends on what other information is provided.
  4. I get an error 1064 syntax error. Very close; it would be nice to know the real error message, but a MySQL 1064 is a familiar, specific kind of problem.
  5. I get an error.

    ERROR 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AS

    Now we're talking. The user said, I get an error., but then gave the full error details.

  6. I get an error., followed by huge amounts of code and text, with the specific error message buried somewhere down the post, probably not formatted correctly. As Mr Lister pointed out in the comments, this is very common, but the proposed feature would (1) have a hard time detecting this and (2) likely do little to help with readability of this type of post.

Figuring out how to parse the many possibilities is a pretty complex task. Figuring out when the user used a phrase like "an error" without elaborating is a natural-language processing problem, and a really hard one.

This proposed feature seems very likely to result in false positives. On the other hand, we have downvotes, close votes, and comments at our disposal to handle and clarify the posts in question already. On top of that, it would be one more thing we're asking users to read before posting. As evidenced by the moderator flag and review queues, the most problematic posts are by people who clearly didn't read and understand most of the instructions presented to them, anyway.

In short, this is complicated to implement and, in my opinion, not realistically likely to solve (or significantly improve) the problem that it attempts to address.

21
  • 1
    Added some clarification. I agree that false positives are possible (like the "subjective" flagging we used to have). I was thinking it'd be helpful to encourage users to post sufficient details, not block them if we think we detect insufficient details.
    – Jacob
    Commented Feb 6, 2016 at 5:20
  • 9
    @Jacob In theory, I think a prompt would be nice. In practice, I'm skeptical. People already don't read the prompts they see (we get hundreds of questions posted as answers every day, and that's just one such problem). And, as I said, figuring out when to prompt the user would be very difficult. I like the idea; I just don't think it has legs unless you're hiding an AI that can pass the Turing test somewhere. :)
    – elixenide Mod
    Commented Feb 6, 2016 at 5:35
  • 3
    Quite. Your example 5 is one of the easier ones, even. In reality, people don't use quote blocks to show error messages, and they will put stuff such as code blocks in between the "I get an error" and the actual error. I don't really think detecting this is doable.
    – Mr Lister
    Commented Feb 6, 2016 at 16:02
  • 3
    How about my process returns (some large negative number) - where the OP does not even realize it indicates an error?
    – Jongware
    Commented Feb 6, 2016 at 20:10
  • This answer is a good comment on the issue, but nothing more. OK, there are, gotta say, inevitably, going to be false positives and false negatives. So what? Almost all medicines you can buy have (possible or actual) side-effects, yet doctors prescribe them, because it's the sensible thing to do. Commented Feb 7, 2016 at 23:02
  • 1
    @KarolyHorvath The point of my answer is that this is not a "sensible thing to do." It's a tremendously complex feature with little upside and a large downside. In medical terms, it's the equivalent of treating a skin blemish with an expensive, 10-drug cocktail that has a high likelihood of causing serious injury.
    – elixenide Mod
    Commented Feb 7, 2016 at 23:18
  • @EdCottrell: Perhaps that was your point but you never talked about it. What's the large downside? Some false positives? I don't see it that way. Elaborate. Tremendously complex? No, it's a really simple one, that is, if you accept false positives (and false negatives, but those already default to the current behaviour, resulting in no additional harm). Commented Feb 7, 2016 at 23:23
  • Sorry for the wording of the first sentence. My point is, you have a hidden assumption here... you want to be precise, because false positive are baaaaad and that makes the whole task extremely difficult. Commented Feb 7, 2016 at 23:31
  • @KarolyHorvath it is tremendously complex. Feel free to try implementing an algorithm that can efficiently distinguish between good posts and bad posts on this metric with greater than 50% accuracy. It's not easy at all, and it would mostly just add clutter to the "ask a question" page. In any case, I did edit my post to make my position clear.
    – elixenide Mod
    Commented Feb 7, 2016 at 23:32
  • @EdCottrell: I feel I should accept this challenge.... There are so many bad posts on SO that showing the message whenever some hardcoded expressions appears in the text would likely pass the test. Commented Feb 7, 2016 at 23:35
  • @KarolyHorvath there's no "task" here. This is a feature request, and it's not a good proposed feature. That's what my answer says. It sounds like you want an explanation of why false positives are bad, which is way beyond the scope of this question. Short answer: the SO developers have better things to do than implement features that mostly generate noise.
    – elixenide Mod
    Commented Feb 7, 2016 at 23:36
  • @EdCottrell: Short answer: False positives aren't bad. A high percentage of false positives are bad. Your answer and comments doesn't convince me that's the case. Commented Feb 7, 2016 at 23:39
  • @KarolyHorvath False positives are, by definition, errors, and thus almost everyone would say they are bad. High numbers of them are, as you say, clearly bad. As to whether I'm right that this would cause lots of false positives, be hard to implement, and be even harder to implement efficiently, well, you're welcome to disagree.
    – elixenide Mod
    Commented Feb 7, 2016 at 23:44
  • @EdCottrell: You see, that's what's I'm saying. You're categorizing them as bad. It depends on the context. Imagine doctors banning AIDS test because they can have false positives. Commented Feb 7, 2016 at 23:49
  • @KarolyHorvath I think you're missing the point. Some false positives are tolerable. As you said, high levels of them are not. An HIV test that gave a "positive" result in a majority of healthy patients would be pretty useless. You might want to read about Bayesian statistics, if you're not already familiar. False positives for relatively rare conditions are generally very bad.
    – elixenide Mod
    Commented Feb 7, 2016 at 23:53
11

Do we want to force low-rep users to click some check-boxes before submitting a question? Like

  • "yes, I included the full text of any errors I mentioned" (maybe hidden if the OP's heuristic didn't trigger)
  • "yes, I used a debugger to find which line / instruction had the segfault, and/or to try to solve it myself" (depending on tags, maybe)
  • "yes, I searched the web and/or stackoverflow for this issue"

In the x86 / assembly tags, it seems like 95% of the questions from users with under 50 rep are:

  • A: people asking for debugging help who could probably have solved their own problem if they knew how to use a debugger. Even if they only knew enough to find what instruction their program segfaulted on, they'd be half way to solving their problem. (And would save readers of their questions a ton of frustration trying to follow their horribly-written code)

  • B: duplicates that a search finds the answer to easily (although sometimes you have to know the right technical terms)

A short FAQ / checklist to weed out people that didn't really need to ask a question would be great. Some kind of AI heuristics to steer people towards the right part of a hand-maintained FAQ would be something I'd be a lot more interested in working on, compared to hand-holding a stream of people that don't grok stuff that's already been written tons of times in many tutorials and answers.

2
  • 1
    Oh god, I love this solution. Hell, there are times where I've forgotten to post something important despite having dealt with incomplete questions for years (not just on SO) and for a variety of reasons ("thinking the details were sufficient" to "oops, forgot"). The only change I'd make is changing it from low rep checkboxes would be to not require the checks be checked at high rep (just a helpbox off to the side). Commented Feb 8, 2016 at 19:59
  • 1
    What result would clicking these checkboxes have? does it change the question in any way? if not, then it isn't really solving the problem and is more likely to just cause confusion. "But i clicked the checkbox saying that i did!"
    – Kevin B
    Commented Feb 8, 2016 at 20:27
3

How about a straightforward message for low-reputation users above the textarea that says

Please provide as much details as you can, e.g. expected results, console logs, and error messages.

Additionally, maybe you can make the message appear only (and stay) after the user selects the textarea, so that it'll get the user's attention.

The user can dismiss the message and it won't show up ever again.

I think trying to detect text patterns might be too difficult.

2
  • 1
    For the people who are posting these problems (i.e. the ones who have already put in no effort), providing as much details as you can will just result in them ignoring that, or copy-pasting their entire code (or screenshots of their IDE, which I guess can be useful)
    – Tas
    Commented Feb 7, 2016 at 22:43
  • A hardcoded text will be likely ignored. OTOH, if it's a response to their actual question, it might elicit some addition info. Commented Feb 7, 2016 at 23:14
3

Showing a (non-blocking) prompt is definitely a good idea, and I think we sould show it for all the (positive and false positive) examples provided by Ed.

This simply boils down to economy. These questions are frequent, there's only one poster, but many readers, and many comments and time spent waiting for a reply.

With a little UI modificaton and canary testing it would be also very easy to collect statistics and measure whether such a feature makes a difference. Arguing about what would work in practice is nice and all, but can be very misleading, and it's such a 20th century thing...

2

This prompt already kind of exists with this close reason:

Questions seeking debugging help ("why isn't this code working?") must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself. Questions without a clear problem statement are not useful to other readers. See: How to create a Minimal, Complete, and Verifiable example.

Yes, that's after the fact but that seems to be more effective. This came up on one of the many duplicate meta requests for comments on down votes: People are more willing to fix things when it's clear that they have to do so. The text of your suggested prompt states:

in order to receive the best answers to your question

But that's not why we're asking them to do it. We're asking them to do it because a) it's difficult to impossible to answer without that information and b) the site rules require it. Many users don't actually care about getting the best answer, they just want something that works.

I'm not necessarily against having the prompt but if it's added I think it should just use the text from the existing close reason.

1
  • That is indeed good text for the situation.
    – Jacob
    Commented Feb 6, 2016 at 23:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .