15

TL;DR: The low quality of questions has been linked to decreased activity of users who have > 25k reputation. Improving question quality would benefit both ends. What would be the best ways to achieve it?

Improving the quality of Stack Overflow questions has been discussed before. One suggested solution is to add tag-tips.

The nice thing about this is that it will allow the community to create standards (or a "tag-tip" wiki) for what each tag should say. -Kermit

Another suggested solution is to add small barrier for new users before they can ask questions.

I think the goal of any barrier would be to encourage better questions. To that end, what if your first question required filling out a form, rather than a free-form text box? -Bryan Oakley

One of the most interesting solutions, is to add machine learning heuristics to detect bad questions right away. This is already implemented for bad answers. And there has also been a contest to find an algorithm that predicts whether a question will be closed. Indeed, there is also a detailed analysis of closed questions.

Feature analysis reveals stack overflow urls and code snippet length as top differentiating features to predict a ‘closed’ question. -Correa, Sureka

In addition to that, Stack Overflow is currently launching a series of tests to

educate new community members about Q&A.

Problem

  • Asking good and clear question with reproducible example is not easy for first time users
  • Even though there are generally good guidelines, new Stack Overflow users don't find them or don't know how to apply them in a specific area (tag).
  • To address this problem, e.g. under javascript tag info, there are specific guidelines to ask JavaScript-related questions.

Possible solutions

  • My idea: Community supported tag-specific sample question which shows up for new users.
  • Sample question shown would be related to first tag and user could modify it or delete it.
  • Tag-tips: More guided help for first time users.
  • First question requires filling out a form rather than a free form text box
  • Machine learning heuristics to detect bad questions before they are posted.
  • Efficient way of using voting/flags and comments.

Downvoting is actually a quite effective method to force people to rethink their question and make it better. But it still requires manual work and active education of new users by asking the same questions over and over again. Is there any other, more efficient, ways to improve quality?

20
  • 4
    Wait, what? I'm not sure what it is you're asking here.
    – Makoto
    Commented Apr 12, 2016 at 17:41
  • 1
    I am doing feature-request. Like this post Commented Apr 12, 2016 at 17:44
  • I can see that, but I'm not really sure what kind of feature you're requesting. Are you asking for R to get support to embed runnable code in a question?
    – Makoto
    Commented Apr 12, 2016 at 17:45
  • 1
    I am asking stackoverflow to implement tag specific model answer. And it has been already discussed here Commented Apr 12, 2016 at 17:46
  • @FrédéricHamidi Yes, sort of. In R shiny context that would been minimal working r shiny single-file app. Where users then could add their functionality. That would make answering questions so much faster. In the upper comment there should read "model question" not answer. Commented Apr 12, 2016 at 17:53
  • 4
    Ah, so a sample of a good question for the user to base their question off of. Seems like a good idea, but... who's going to select the "model question"? As it is even among the few gold badge holders of a particular tag there's disagreement upon which questions are good and which are bad. Basing it off of votes isn't perfect either, since often poor quality questions with bounties get a lot of votes.
    – Kevin B
    Commented Apr 12, 2016 at 18:50
  • Well maybe it shouln't be complete template but bare minimum to guarantee easier reproducibility of problem in specific environment. Commented Apr 12, 2016 at 18:54
  • @Mikael, so I was on the right track after all. For starters, are you positive that a vast majority of the questions tagged [shiny] would benefit from a single-file-app template? Commented Apr 12, 2016 at 18:55
  • 1
    Some tags already have that information within the tag wiki. stackoverflow.com/tags/javascript/info
    – Kevin B
    Commented Apr 12, 2016 at 18:56
  • 1
    @FrédéricHamidi I made fast calculation that from 23 questions asked during last two days 3 used single-file app format which makes reproducibility easier. Many of the questions were not reproducible at all. Commented Apr 12, 2016 at 19:04
  • @KevinB Thanks for info, that would be of course one option. Maybe community wiki could be shown in that orange box next to question box for primary tag? Commented Apr 12, 2016 at 19:05
  • 2
    Related: meta.stackoverflow.com/questions/319980/… Commented Apr 12, 2016 at 19:28
  • "Tag-tips" has been discussed before and seems to be generally up-voted feature request. Commented Apr 13, 2016 at 5:01
  • Users are provided with lots of information on how to ask a good question. The vast majority simply choose to ignore it all because they don't care about our quality standards, they're just going to ask the question they want to ask without even reading anything we put in front of them. Adding one more thing for people to ignore isn't going to change a thing. All of the people actually making an effort to ask a good question, and read all of the existing information on what an acceptable question is are already asking good questions.
    – Servy
    Commented Apr 13, 2016 at 20:44
  • 2
    Relevant: Eternal September
    – Bergi
    Commented Apr 15, 2016 at 10:57

4 Answers 4

8

I don't think more guidance is a bad idea. I think there are several possible ways to go about improving question quality. I am posting them as separate suggestions so they can be individually vetted.

Targeted Close Reasons and Quicker Closure

Essentially more close reasons for questions from low reputation question askers. I know this may seem "elitist" but there is certainly data to suggest that low reputation users have questions closed at a much higher rate than that of higher reputation users. Furthermore, their posts are probably deleted at a higher rate but that data is not public so I could not provide it for analysis.

 Rating Range       Asked    Closed    Percent Closed
    01-100      3,379,045   185,783        5.50%
   100-1000     4,441,445   174,819        3.94%
  1000-10000    3,745,184   136,437        3.64%
 10000-100000     503,212    17,904        3.56%
100000-1000000     17,468       630        3.61%
     All       12,086,354   515,573        4.27%

Given this, it could be easier to distinguish questions which need to be closed if there were perhaps another 3-5 close reasons which only apply to users who are essentially learning the system. This would provide more granular feedback as well to them.

Any feedback on what type of close reasons would work here, or if an expanded amount of reasons would help, is welcome.

In addition to more reasons, this set could also face easier routes towards closure, perhaps less total votes required; such as maybe one less, as in 4. Also this could be paired with Brad Larson's idea (inspired by an MSE post) for a downvote trigger for closure.

These metrics could make it easier to clean up questions quicker, and perhaps with the voting aspect, in a more automated fashion as opposed to manually having to pick and choose close reasons.

1
  • I like this idea, one problem under small activity tags is that question closing can take quite much time because there are not so many people voting or commenting. And new users need to get fast feedback what is wrong. Commented Apr 14, 2016 at 20:39
6

I don't think more guidance is a bad idea. I think there are several possible ways to go about improving question quality. I am posting them as separate suggestions so they can be individually vetted.

On Topic Checklist

Show the "on topic" checklist to users who have less than 111 reputation (basically new users and also takes into account users with the association bonus). The excerpt shown is taken from https://stackoverflow.com/help/on-topic . This would only be visible to users who have less than 111 reputation.

enter image description here

While it is certainly reading which can be skipped it as at least front and center as opposed to behind several clicks. This request is also present on Meta Stack Exchange here: https://meta.stackexchange.com/a/277613/178816

5
  • 4
    As an extension to that: The actual form for the question could be hidden and only get visible after clicking on a confirm-button. This would prevent the user of just skipping it and proceed directly. Clicking I'm ready to ask could beneficial.
    – Matt
    Commented Apr 14, 2016 at 5:36
  • 4
    @Matt New users already have to read this page and they must tick the checkbox thanks, I will keep these tips in mind when asking and click "proceed". I don't know how long this exists, but I don't see any effect. It's just two clicks more and then they ask their crappy question anyway. Commented Apr 14, 2016 at 21:31
  • If all new posters bothered to read what is presented to them, we would not have a problem. How is adding more text for them to ignore going to help?
    – Raedwald
    Commented Apr 16, 2016 at 17:56
  • 1
    Maybe add a quiz that they first have to get right?
    – CherryDT
    Commented Oct 26, 2022 at 17:26
  • @CherryDT - This would perhaps create too large of a barrier to entry for onboarding. However, it may make sense to do that in response to a question asker's first question being closed.
    – Travis J
    Commented Oct 26, 2022 at 19:45
5

Instead of the checkbox "I will keep these things in mind" which can be mindlessly clicked by anyone without reading the text above it, maybe an actual quiz would help. On Upwork for example, when you become a freelancer you have to complete an "Upwork readiness test" before you can actually apply to any job. Something similar could be implemented here.

Yes, this increases the barrier of entry, but I actually think that's a good thing. (If someone is in a rush, the question will probably not fit the site anyway, especially if it's their first. If someone's English is so bad that they can't understand the quiz, then they also won't be able to ask a proper question or understand the clarification questions or answers correctly. If someone simply can't be bothered by spending a bit of time and energy to familiarize themselves with the rules, they also won't deem it worth their time to take the required effort for posting a good question (and won't know how to, anyway). And so on.)

And yes, people can google what the answers to the quiz are. There isn't much we could do about that, but at least this should hopefully invoke the sense of knowingly having cheated (and therefore being more likely to run into troubles afterwards due to missing knowledge), for what it's worth.

1
  • 4
    As long as the KPIs include quantity (new signed up users per month, new questions per month, etc.), there isn't much hope. Commented Oct 27, 2022 at 1:03
3

I don't think more guidance is a bad idea. I think there are several possible ways to go about improving question quality. I am posting them as separate suggestions so they can be individually vetted.

Tag Based Question Asking Tips

I like the idea of tag based suggestions for the question asking screen. Have a section on question asking in the wiki interface which is maintained by the community. The tag with the largest amount of questions tagged in a question has its question asking section pegged to the right as suggested in your proposal here. I think that is a decent approach.

1
  • I guess most of the closed/downvoted questions have some rather generic deficiency (like missing minimal example, ...) but there might be some tag specific examples of closed/downvoted questions. The difficulty would be here to identify them. Commented Apr 14, 2016 at 20:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .