177

The problem:

Currently, to ask a question, a user has to tick a button at the bottom of a page of text:

How to ask page (as of May 2018)

Nowhere am I told that Stack Overflow isn't a forum. At no point is it checked that I have read the page of text. There's no delay before I can tick it. I do not need to have opened any of the links on the page. My expectations are that this site will accept my question because it's about programming. How different can it be to other groups?

This causes new users to think that any question will be accepted here, and then they feel unwelcome because we rapidly downvote and close their post.

In comparison, if I want to join a closed Facebook group (with much lower quality standards) I am presented with the rules. I have to read these rules because - and here's the key step - I am quizzed on these rules. A text box pops up and says "Is XYZ allowed in our group?". This answer is manually checked by the Admins, and after that I can join.

The proposal

Stack Overflow obviously gets too many new users for each one to be manually approved. Instead, we should have an automated test. It looks like this:

Revised how to ask page with a mini quiz

This page has 5 questions from a list of 20+. They are randomly selected and ordered. Critically, they are all answered in the link to the help centre.

A user should read the link, and then tick the correct questions. A "Continue" button will appear. If the user got all 5 right, they are allowed to proceed to the Ask Question page.

If they get it wrong, they are told that they got them wrong, and then asked to re-read the help centre page, which is linked. The questions will also be re-generated, to help prevent guessing.

After 3 incorrect answers, the user is given a 5 minute cooldown, to force them to take the time to read the help centre page.

This continues until they get all 5 questions correct, and they are redirected to the Ask Question page.

Advantages

  • Increase the probability that users will have read the linked help centre page.
  • Decrease the number of off-topic / poorly presented questions which experienced users have to sort through - and hopefully the increase overall quality of Stack Overflow.
  • Consequently, decrease the number of downvotes / close votes given to new users. This will make the website feel more welcoming.
  • Clarify new users' expectations - even if they don't read the help centre page especially thoroughly, they know that we're a bit different - few sites have a 5 question quiz to join.
  • This treats everyone the same. Women and minorities are not being singled out / treated differently by this page. Claims of discrimination should rapidly drop.

Disadvantages:

  • This reduces the number of new users asking questions (and the total number of questions), which reduces views, user engagement, and ad-revenue.
  • People will still leave thinking that Stack Overflow is "super elitist" (1 2 3 4 5) and "doesn't want to help anyone" because they have to do a quiz to get in.
  • 3% of users will be able to get in by just guessing the answers to these questions. More questions would reduce this - 10 questions drops to 0.1% can guess.
26
  • 30
    Jeff Atwood posted a relevant comment: "a possible related feature request is to requre a user to both create an account and spend {x} minutes reading {y} different questions on the site before they can ask a question. This can easily be measured in JavaScript, for example in Discourse we've always measured exactly how long every post is read by every user in the system, because we believe reading is fundamental." Commented May 1, 2018 at 23:42
  • 2
    Also probably worth mentioning a previous idea for testing new users Commented May 1, 2018 at 23:44
  • 2
    I actually like this idea. The knock against previous similar ideas was that they didn't test anything, they just showed extra messages, and if a user is prone to ignoring such messages, they'll be useless. This idea is similar to the basic online introductory courses we have to take when we start work at the hospital. It's nothing difficult, just a few basic questions to verify that you actually cared to even skim the reading material. Commented May 1, 2018 at 23:50
  • 8
    This.. actually doesn't seem like a terrible idea out of the gate. I don't really think any of the disadvantages is one. We can't shut out LQ questions completely. Similar to the CAPTCHA problem we can only make it hard enough to keep out most of the bad and still let the good in. Many of the people who call SO elitist use the site as a resource, a resource that couldn't exist if we do things their way.
    – Seth
    Commented May 1, 2018 at 23:51
  • 2
    Give it... 15 minutes, and someone will have scripted a bot that bypassed the test, and will boast about it on Reddit/Twitter/whatever, reducing the utility of the test a bit (not entirely, and possibly enough to make it worth it by stopping the worse questions.... Who knows). Although, while I get the intent, I feel a barrier to entry won't make us feel more welcoming....
    – Patrice
    Commented May 2, 2018 at 0:03
  • 3
    Yeah. I suggested such a 'read the rules and take the tour' minimum time a couple of suspensions ago - it was rejected as too much of a barrier to new accounts:( Commented May 2, 2018 at 0:11
  • 23
    @Patrice Someone who has the patience and the expertise to write a bot, and doesn't have the patience to read a paragraph or two and answer some questions?
    – Zev Spitz
    Commented May 2, 2018 at 0:52
  • 1
    @ZevSpitz one person has the skill to build the bot and 'one up' the big bad stack overflow.some of the people who complain about stack do have the skill to do that.
    – Patrice
    Commented May 2, 2018 at 1:03
  • 65
    "I feel a barrier to entry won't make us feel more welcoming" Let me be blunt and address the elephant in the room: do we really care about being unwelcoming to people who won't spare five minutes to read the rules?
    – Passer By
    Commented May 2, 2018 at 5:01
  • 19
    @Patrice I feel a barrier to entry may make us feel more welcoming. It may be ironic, but it may also be true: being deceptively friendly and easy just means a bigger shock when people learn out that it's actually kind of hard.
    – abarnert
    Commented May 2, 2018 at 5:26
  • 3
    @ZevSpitz Have you never heard of 4chan?
    – abarnert
    Commented May 2, 2018 at 5:50
  • 3
    A few days ago I suggested to give some rep for passing one or more such tests. Maybe enough to allow them to comment.
    – TaW
    Commented May 2, 2018 at 11:42
  • 3
    I once made a radical suggestion that all first posts have to go through review before they were allowed to pop up on the main site. But I even thought myself that this wasn't a sensible thing to ask since there would be too much moderation effort. Then later someone presented statistics over how much work is currently being put into review queues and other forms of moderation, and it turned out that the amount of work already exceeded the amount of new posts. If there was only one review per question, we would reduce the moderator workload.
    – Lundin
    Commented May 2, 2018 at 11:42
  • 6
  • 4
    I think a large part of the confusion centers around what is meant by a "specific programming problem". The problem with bad developers is that they lack the skill to deconstruct a project into a collection of specific problems. They think that "I need to make an app that consumes X, calculates Y, and returns Z" is a specific programming problem when, clearly, here it manifestly too broad. They read these criteria and honestly think "yep, that's me and my question!" when the consensus here would entirely be the opposite. I don't think you can address that in a simple way.
    – J...
    Commented May 3, 2018 at 16:29

6 Answers 6

86

Pusheen Yes!

Usability fact #1, the Prime Directive: users don't read.

Now, we could be forgiven thinking that the users of Stack Overflow - who we can assume are programmers, and therefore more clued-up and attentive than the average bear, are used to reading - but the evidence, from consistently terrible questions to the fact that most users have no idea how reputation works... demonstrate that this belief is abjectly false.

The majority of Stack Overflow users are no smarter and give no more s**ts than the average bear, and it's time we stopped pretending and hoping they are/do; maybe in an alternate timeline where the Summer of Love never happened, but not this one.

As such, it's time to stop wasting time and effort on making help pages and tours more readable and discoverable, and accept that the only way to realistically deal with the hordes at the gate is to treat them as five-year-olds by default: by forcing them to prove a minimal level of competence and comprehension before we allow them to post questions (and get discriminated against by having those questions closed or downvoted).

This proposal does exactly that, and should be fast-tracked into production as a matter of urgency. Although I'm sure Joel will torpedo it as a matter of course.

3
  • 10
    I can agree, though, that many of the proposed solutions for the 'unwelcome' problem will be either not effective, or counter-productive. The sole fact that is important, and obvious, to so many OP's who get the Q's down/close voted is that they have not got an answer to their question. Everything else is irrelevant. They did not get an answer, so it's somebody's fault, they don't have a mirror handy and the only people they can see in front of them are SO user-moderators. So, simple, it's obviously their fault. 'hostile' and 'toxic' are no longer cool, so 'unwelcoming' it is for now. Commented May 2, 2018 at 12:58
  • 4
    "Now, we could be forgiven thinking that the users of Stack Overflow - who we can assume are programmers, and therefore more clued-up and attentive than the average bear, and used to reading", hahaha, you are so forgiving: '7 years ago, the problem was that programmers seem to have stopped reading books; today [~2 years ago] it's that programmers seem to have stopped reading anything - up to and including their own code.'
    – Braiam
    Commented May 3, 2018 at 17:42
  • 2
    @Braiam ..and today, programmers seem to have stopped having any of their own code at all. Unfortunately, because of the reading problems that you linked, and the extremely high probabilty that code downloaded from 'homeworkexamples.com' wil not just 'work' without some polishing, the download ends up on SO with 'Please help me':( Commented May 3, 2018 at 19:48
14

I think the motivation behind this proposal gets to the heart of a very important issue. I suspect many new users are misled into thinking that asking questions is easy. And that's dangerous. When you fail at something hard, you think, "I just learned something"; when you fail at something easy, you get either embarrassed and discouraged, or defensive and angry.

However, I don't think a quiz is really the best way to handle this.

I also think they ought to try out the guided new-user question idea they've already announced before adding major new features/interactivity/whatever to the new user process.

What can be done instead is just to change the content of the tour in two ways:

  • Focus more on examples than explanations. (For example, maybe show them what a MCVE is, and why it is one, and why that matters, instead of just telling them that this is a Q&A site and questions should be Q&A questions, with links to what that means.)
  • Don't try to sugar-coat things in the tour. (For example, maybe show them what it looks like to get your question closed, and how to deal with it properly.1)

1. Of course reopens really don't work nearly as well as they should… but that's a whole other problem, and any solution may have unintended consequences

12
  • 2
    Isn't this just renaming the help centre to the tour?
    – Passer By
    Commented May 2, 2018 at 5:06
  • @PasserBy The intention is just the opposite. The tour is mostly the same information as the help center, presented the same way, just less of it. Instead, it should be something different: instead of explanations without examples, simple examples without much explanatory text.
    – abarnert
    Commented May 2, 2018 at 5:08
  • Makes sense. How should the examples be acquired? It probably suffers the same problems with question templates
    – Passer By
    Commented May 2, 2018 at 5:11
  • @PasserBy That… I have no idea, actually, but I'm hoping someone else can do the hard part. :) See this other currently-open discussion.
    – abarnert
    Commented May 2, 2018 at 5:12
  • 1
    "How to deal with it properly", BAHAHAHA. The users who can't deal with closed questions are the ones who don't care about asking them correctly, so good luck with that one!
    – Ian Kemp
    Commented May 2, 2018 at 11:59
  • 5
    @IanKemp I don’t think that’s true. Some are unwilling to do any work, and some would do a bit of work if they knew it was necessary: the intellectually lazy vs the naive. Current policy assumes everyone is naive, and if we’re just really nice they’ll all be good. I think the difference is real, and we should aim policy at helping the naive—and if it drives away the lazy even faster, that’s a bonus, not a problem. And implying that their first question may be closed is a way to do that—some think “What can I do avoid that”, others go away thinking “SO is mean” without wasting or time.
    – abarnert
    Commented May 2, 2018 at 16:13
  • 2
    Don't forget that not all questions are debugging questions. The most interesting questions are almost never debugging questions, so make sure the phrasing for [mcve] doesn't discourage regular questions about how / why stuff works or is the way it is. (Obviously there are many bad non-debugging questions, too, and we don't want to encourage open-ended teach-me-concepts or write code for me questions either.) Commented May 3, 2018 at 17:13
  • 1
    @PeterCordes Good point. The difference between good and bad “why” questions is generally much more obvious than between good and bad debugging questions—but is that true for novices? If not, that presumably means that demonstrating what makes a good “why” question is harder, not that it’s any less necessary… I don’t have an answer to that, except to say that (a) “don’t sugar-coat” is equally important there, and (b) “examples not explanations” may only make sense for low-hanging fruit that can at best only solve half the problem, but I’d still be happy with that as a start.
    – abarnert
    Commented May 3, 2018 at 17:21
  • Yeah, half way through writing that comment, I ran into the same problem of trying to describe what makes a good non-debugging question, and mostly gave up on trying to be specific because it wasn't going well. Commented May 3, 2018 at 17:23
  • @PeterCordes That being said, I don’t want to dismiss the danger that too much focus on examples could mislead people into taking the examples too restrictively and prevent them from asking good “why” questions. But I think we’re currently so far from giving novices a strong sense of what to ask here that there’s a lot less risk of overconstraining them than continuing to underconstrain them. We just need to keep your point in mind and be careful.
    – abarnert
    Commented May 3, 2018 at 17:23
  • @abarnert: I'd like to see more structure in the question box to better guide people, like a section for "what should happen", and a section for "what did happen". Maybe give them a checkbox for "I'm debugging something I'm trying to write", and if they check it, add some headings to their question. Many bug-reporting systems have headers / bullet points like this where submitters are expected to fill in all the ones that apply, so they don't forget important stuff like version number or config file. Commented May 3, 2018 at 17:26
  • @PeterCordes Agreed. But since it sounds like SO is already at the start of an experiment for guided question asking for new users, I figure it’s better to let that experiment play out first. Because then we can snarkily judge it with 20/20 hindsight. :) Or, more seriously, because we can hopefully have some data or what did and didn’t work rather than proposing something based on first principles and educated guesses. (Even though I think your specific idea is very likely to be helpful.)
    – abarnert
    Commented May 3, 2018 at 17:32
10

Using the power of remorse and downvotophobia:

A picture is worth a thousand words

Another approach would be to show a simillar message (for example, as a banner at the very top of the question) once the question gets at least 2 downvotes. The message could be something like:

Your question got downvoted, huh? That's a bummer. Maybe there is something wrong with the question. Have you read this before asking?

This will put the blame on the askers for not reading the "how to ask" page before. This will also clear away the unwelcoming and hostile accusations SO users get by having the askers take full responsability for the, now justified, downvotes their questions got.

8
  • 4
    Maybe if that were just as visible/findable after they get downvoted (and if there were better notifications about downvotes) it would be helpful. But if it’s only on the Ask page, imagine how frustrating it would be to know there’s a link you were supposed to read if X happens, but now that X has happened you can’t even find it… (Of course you or I would just go to ask another question, but I’m not sure the typical target user would think that way.)
    – abarnert
    Commented May 3, 2018 at 17:29
  • 2
    @abarnert Good idea I was thinking of editting that into the answer. But there is a benifit of having it in the Ask page: they may get affraid of their question getting downvoted and read it first (out of both curiosity and fear). Also the link is findable, it is on the Ask page. Commented May 3, 2018 at 17:35
  • 3
    This assumes people understand the concept of what a downvote is prior to accessing the site and at this point I don't assume anyone's knowledge Commented May 3, 2018 at 17:35
  • 3
    @abarnert: the system could display a banner for logged-in users looking at their own downvoted questions after they're asked. I do like the idea of pre-emptively warning people it's a possibility, and maybe encouraging them to go look at stuff to avoid it. That might work. Commented May 3, 2018 at 17:35
  • 2
    @ibrahimmahrir Now that I think about it, you're right—putting it in the Ask page would definitely help prime people for the possibility, and for the idea that there may be something they can do about it before ranting and rage-quitting. I'm not sure being on the Ask page is sufficient (again, how many of the target users would think to look for it by starting to ask another question?), but being there plus, say, a banner as suggested by Peter Cordes? That sounds great.
    – abarnert
    Commented May 3, 2018 at 17:46
  • 4
    That is ... a ... searching for words ... daunting? fearsome? scary? banner ...
    – rene
    Commented May 3, 2018 at 18:05
  • I feel like if a user trying to post a question just skipped over the text/links/etc without thinking about it, in their mind this would be the first time SO has shown them this information. I can see people getting annoyed because the site waited until after they asked a question to tell them how to ask a question. Commented May 3, 2018 at 18:14
  • 3
    @rene the whole site should have a warning symbol, similar to 'ionising radiation' or 'laser light', that clearly identifies a risk; 'Skilled Developers Who Hate Wasting Time - Danger of Forthright Honesty'. Commented May 3, 2018 at 19:55
7

Update: looks like this alternate idea for giving users an expectation of our standards for questions has already been proposed: Let's send new users off to see the Wizard.
And so has Pre-fill question-box with text for new users (A/B study proposal)

I still think it's relevant to mention this idea as an answer to this question, because these things also take up time for new users (although all of it is spent actually working on their question), and also give new users a very big clue of what we require for questions, especially debugging questions.


I'd like to see more structure to the question box itself, at least for new users. For debugging questions, maybe pre-fill the textbox with section headers.

Maybe even take users through a question-asking wizard with multiple steps:

  1. Is your question about debugging some code you're trying to write? (if users say no, later steps could encourage them to make an attempt and come back here instead of asking a gimme the codez question instead of an interesting "why" / "how does this work" question)

  2. (if yes) Paste your code here. (And format it for them, maybe with guidance showing how to do that?) Possibly try to detect if it's long and encourage an MCVE of just the part they're working on, not the whole program.

  3. What is it supposed to do

  4. What actually happens exactly, with error messages copied/pasted

  5. When you single-stepped through it in a debugger, what did you find out about values in variables or which loop / if conditions were taken? Was it what you expected? Which line did you program crash on? (And this is the point where we send users to a tutorial on using a debugger for their platform/language, because in some languages (especially assembly) that would solve at least 50% of the debugging questions.

    Hopefully many users solve their problem on their own at this point.

As they go through the wizard, build up the question textbox lower down on the page. When they're done with the wizard steps, leave them with the question box filled out with their question, and encourage them to scroll down and make sure everything was formatted ok.

And for non-debugging questions (usually the more interesting kind of SO question), maybe just take them more quickly to a textbox because there are fewer required components / too many different kinds of questions.

The button for skipping the wizard as a low-rep user could say "Skip the wizard, I know how to ask questions that won't be downvoted"


Or instead of a wizard, maybe just more structure in the question box to better guide people, like a section for "what should happen", and a section for "what did happen".

Maybe give them a checkbox for "I'm debugging something I'm trying to write", and if they check it, add some headings to the question textboox.

Many bug-reporting systems have headers / bullet points like this where submitters are expected to fill in all the ones that apply, so they don't forget important stuff like version number or config file.

I'd be ok with questions from new users including boilerplate ### What actually happened section headers if it meant that more debugging questions actually showed some debugging effort by the asker. And maybe more importantly, if it meant that questions included debugging results that make it easier to see the problem! http://idownvotedbecau.se/nodebugging/ isn't just about the imbalance in fairness of effort by askers vs. answerers, it's about expecting us to debug non-tiny blocks of code in our heads without seeing variable values or whether the loop runs at all or which if() bodies run.

Actually, semi-standardized question layout would probably just plain be an advantage for quickly seeing the different pieces of info you need to be able to answer. If you already see the answer from the code, you don't need to wade through the boring parts.

2
  • This is a good suggestion (I think it was trialed not long ago) but it should be presented as a feature request of its own for more visibility.
    – Tim
    Commented May 3, 2018 at 20:31
  • @Tim: If anyone else wants to write it up as a feature RQ, let me know. I may not get around to it myself. Commented May 3, 2018 at 20:43
-1

This feels very close to a duplicate of a previous suggestion of mine:

The Good, The Bad & The Ugly

My suggestion uses a similar design and layout to the review queues audits, where you make a decision on a post and are presented with a pass or fail.

-2

This will not help.

I can already imagine blogs and copy-paste tutorials popping up like "stackoverflow test answers", "pass stackoverflow test", "testless stackoverflow". I can even imagine browser extensions being written for that very purpose. If passing the test requires more time than copy-pasting an answer across pages, it won't work.

In fact, I think that nothing from Stack Overflow's side could help. The cyber-barbarians will keep on coming. No gates will hold them. We can only pray that given time, average level of knowledge of newcomers will rise.

3
  • 3
    You think people are going to take the time to search out an extension for this one specific step. I bet that 99% of people who can’t be bothered to read and answer also will not be bothered to try to trick it.
    – Tim
    Commented Sep 15, 2020 at 11:56
  • "No gates will hold them." Why? You could disable registering on SO tomorrow and voila, barbarians gone. This would be the less drastic variant. Something like a captcha. And you could use it to do actual work for the site like detecting if other questions contain research... Commented Sep 15, 2020 at 13:22
  • Someone who has the research skills required to "cheat" like this has already demonstrated, ipso facto, one of the desired skills for new users. Commented Mar 11 at 5:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .