100

This post is prompted in part by What can we put in a question template to help people ask better questions? where Jon Ericson and others have indicated an intention to develop a question wizard separately from a question-asking template. But earlier than that, I was prompted by Tim Post's answer from early 2016 that described a need to build and test some Wizard demos on JSFiddle in advance of A/B testing such a wizard. This post is an attempt at laying the groundwork for such efforts and demonstrating what such a wizard might look like.

New users ask bad questions all the time. It's a recurring problem that causes people to discuss quality improvement on Meta every 6 - 8 somethings, causes moderation-minded users to spend a lot of time closing and revising questions, and causes answering-minded users to get frustrated by FGITW answers or users who answer duplicate/poor quality questions.

Luckily, we have had a few attempts at improving this situation with various question wizards over the years. The problem is that a site like Stack Overflow is hard to build a wizard for... and I think the current wizard fails pretty badly in some areas. There are millions of questions on this site, and a wizard can't hope to cover every single case. It can, however, cover a large portion of those cases by combining them into a few broad buckets. These combo buckets already exist in the form of reasons to close a question as off-topic: general hardware/software, server/networking, tool recommendations/tutorial requests, typos, No-MCVE code dumps, etc.

If you flip this around, and look at questions that are explicitly described as on-topic, you see a few similar buckets:

  • specific programming problems
  • software algorithms
  • software tools commonly used by programmers

Each question must also be a practical, answerable question unique to software development.

I think developing a wizard around these 'on-topic' buckets would be a decent start, so I've done just that. Below you'll find a rough Code Snippet demo of what such a wizard could look like. You can re-run the Snippet to see what each different button does. There's even a red free-hand circle hidden somewhere.
Please note that the options, exact verbiage, and code-behind are purely for demonstrative purposes. They do not necessarily convey my actual opinions on what is on-topic, what kind of info is needed for a type of question, etc.

document.getElementById('Q-debug').onclick = function() {
    var debugPage = document.getElementById('debug-page');
    var splashElements = document.getElementsByClassName('splashPage');
    debugPage.style.display = 'block';
    for (var i = 0, max = splashElements.length; i < max; i++) {
        splashElements[i].style.display = "none";
    }
};
document.getElementById('debug-step2').onclick = function() {
    var debugPage2 = document.getElementById('debug-problem-desc');
    this.style.display = 'none';
    debugPage2.style.display = 'block';
};
document.getElementById('debug-step3').onclick = function() {
    var debugPage3 = document.getElementById('debug-mcve');
    var step3Container = document.getElementById('step3-container');
    step3Container.style.display = 'none';
    debugPage3.style.display = 'block';
};
document.getElementById('debug-step4').onclick = function() {
    var debugPage4 = document.getElementById('debug-title');
    var step4Container = document.getElementById('step4-container');
    step4Container.style.display = 'none';
    debugPage4.style.display = 'block';
};
document.getElementById('Q-tool').onclick = function() {
    var toolPage = document.getElementById('tool-page');
    var toolContent = document.getElementById('tool-content');
    var toolMain = document.getElementById('tool-mainbar');
    var toolForm = document.getElementById('tool-question-form');
    var splashElements = document.getElementsByClassName('splashPage');
    toolPage.style.display = 'block';
    toolContent.style.display = 'block';
    toolForm.style.display = 'block';
    toolMain.style.display = 'block';
    for (var i = 0, max = splashElements.length; i < max; i++) {
        splashElements[i].style.display = "none";
    }
};
document.getElementById('tool-step2').onclick = function() {
    var toolPage2 = document.getElementById('tool-problem-desc');
    var toolEditor = document.getElementById('tool-post-editor');
    var toolContainer = document.getElementById('tool-wmd-container');
    var toolButton = document.getElementById("tool-step3-container");
    this.style.display = 'none';
    toolPage2.style.display = 'block';
    toolEditor.style.display = 'block';
    toolContainer.style.display = 'block';
    toolButton.style.display = 'block';
};
document.getElementById('tool-step3').onclick = function() {
    var toolPage3 = document.getElementById('tool-mcve');
    var toolEditor2 = document.getElementById('tool-post-editor2');
    var toolContainer2 = document.getElementById('tool-wmd-container2');
    var toolButton2 = document.getElementById("tool-step4-container");
    this.style.display = 'none';
    toolPage3.style.display = 'block';
    toolEditor2.style.display = 'block';
    toolContainer2.style.display = 'block';
    toolButton2.style.display = 'block';
};
document.getElementById('tool-step4').onclick = function() {
    var toolTitle = document.getElementById('tool-title');
    this.style.display = 'none';
    toolTitle.style.display = 'block';
};
document.getElementById('Q-eng').onclick = function() {
    var knowledgePage = document.getElementById('knowledge-page');
    var knowledgeContent = document.getElementById('knowledge-content');
    var knowledgeMain = document.getElementById('knowledge-mainbar');
    var knowledgeQuestion = document.getElementById('knowledge-question-form');
    var splashElements = document.getElementsByClassName('splashPage');
    knowledgePage.style.display = 'block';
    knowledgeContent.style.display = 'block';
    knowledgeMain.style.display = 'block';
    knowledgeQuestion.style.display = 'block';
    for (var i = 0, max = splashElements.length; i < max; i++) {
        splashElements[i].style.display = "none";
    }
};
document.getElementById('knowledge-step2').onclick = function() {
    var knowledgePage2 = document.getElementById('knowledge-problem-desc');
    var knowledgeEditor = document.getElementById('knowledge-post-editor');
    var knowledgeContainer = document.getElementById('knowledge-wmd-container');
    var knowledgeButton = document.getElementById("knowledge-step3-container");
    this.style.display = 'none';
    knowledgePage2.style.display = 'block';
    knowledgeEditor.style.display = 'block';
    knowledgeContainer.style.display = 'block';
    knowledgeButton.style.display = 'block';
};
document.getElementById('knowledge-step3').onclick = function() {
    var knowledgePage3 = document.getElementById('knowledge-mcve');
    var knowledgeEditor2 = document.getElementById('knowledge-post-editor2');
    var knowledgeContainer2 = document.getElementById('knowledge-wmd-container2');
    var knowledgeButton2 = document.getElementById("knowledge-step4-container");
    this.style.display = 'none';
    knowledgePage3.style.display = 'block';
    knowledgeEditor2.style.display = 'block';
    knowledgeContainer2.style.display = 'block';
    knowledgeButton2.style.display = 'block';
};
document.getElementById('knowledge-step4').onclick = function() {
    var knowledgeTitle = document.getElementById('knowledge-title');
    this.style.display = 'none';
    knowledgeTitle.style.display = 'block';
};
document.getElementById('Q-other').onclick = function() {
    var ontopicPage = document.getElementById('ontopic-page');
    var splashElements = document.getElementsByClassName('splashPage');
    ontopicPage.style.display = 'block';
    for (var i = 0, max = splashElements.length; i < max; i++) {
        splashElements[i].style.display = "none";
    }
};
html {
    height: 100%;
    margin: 5px;
}
body {
    min-height: 100%;
    display: flex;
    flex-flow: column nowrap;
    font-family: Arial, "Helvetica Neue", Helvetica, sans-serif;
    font-size: 13px;
    line-height: 1.3em;
    color: #242729;
    background: #fff;
    margin: 0;
    padding: 0;
    vertical-align: baseline;
}
h2 {
    margin: 14px auto;
}
button {
    color: #fff;
    background-color: #0095ff;
    box-shadow: inset 0 1px 0 #66bfff;
    padding: .61538462em 1em;
    vertical-align: middle;
    min-height: 2.46153846em;
    box-sizing: border-box;
    font-weight: 400;
    font-family: inherit;
    line-height: 1;
    text-align: center;
    text-decoration: none;
    border-radius: 2px;
    border: 1px solid transparent;
    border-color: #07c;
    margin: 5px auto;
    max-width: 400px;
    cursor: pointer;
    transition: all .1s ease-in;
}
button:hover {
    cursor: pointer;
    color: rgba(255,255,255,0.9);
    background-color: #07c;
    border-color: #005999;
    box-shadow: inset 0 1px 0 #3af;
    text-decoration: none;
    transition: all .1s ease-in-out;
    outline: none;
}
div[id$="-page"] {
    display: none;
}
.container {
    margin-top: 60px;
    position: relative;
    width: 100%;
    flex: 1 0 auto;
    margin: 0 auto;
    text-align: left;
}
.container div[id$="-content"] {
    padding: 24px 15px;
}
div[id$="-content"] {
    box-sizing: content-box;
    margin: 0 auto;
    background-color: #fff;
}
.ask-page div[id$="-content"] {
    min-height: 750px;
}
.ask-mainbar {
    width: 665px;
}
.form-item {
    padding: 10px 0 5px;
}
.ask-title {
    width: 668px;
}
.container div[id^="debug-"], .container div[id^="tool-"], .container div[id^="knowledge-"], .container div[id^="ontopic"] {
    display: none;
}
div[id^="debug-"] h2, div[id^="tool-"] h2 {
    margin: 0px;
    margin-top: 10px;
}
div[id^="debug-"] h5, div[id^="tool-"] h5 {
    margin: 5px 0;
}
.form-item label {
    display: inline-block;
    font-weight: bold;
    padding-bottom: 3px;
}
.wmd-input {
    box-sizing: border-box;
    padding: 10px;
    height: 200px;
    line-height: 1.3;
    width: 660px;
}
input {
    color: #9fa6ad;
    margin: 5px 0;
}
input[type="text"] {
    box-shadow: 0 1px 2px rgba(12,13,14,0.1) inset;
    padding: 8px 10px;
    font-size: 14px;
    color: #3b4045;
    background: #fff;
    border: 1px solid #c8ccd0;
}
input[id$="title"] {
    width: 591px;
}
p {
    clear: both;
    margin-bottom: 1em;
    margin-top: 0;
}
a {
    color: #07c;
    text-decoration: none;
    cursor: pointer;
}
.form-submit {
    display: block;
    padding: 10px 0 15px 0;
}
.container input[type="submit"] {
    color: #fff;
    background-color: #0095ff;
    box-shadow: inset 0 1px 0 #66bfff;
    display: inline-block;
    position: relative;
    padding: .61538462em 1em;
    vertical-align: middle;
    min-height: 2.46153846em;
    box-sizing: border-box;
    font-weight: 400;
    font-family: inherit;
    line-height: 1;
    text-align: center;
    text-decoration: none;
    border-radius: 2px;
    border: 1px solid transparent;
    border-color: #07c;
    transition: all .1s ease-in;
    outline: none;
    cursor: pointer;
}
input[type="submit"]:hover {
    cursor: pointer;
    color: rgba(255,255,255,0.9);
    background-color: #07c;
    border-color: #005999;
    box-shadow: inset 0 1px 0 #3af;
    text-decoration: none;
    transition: all .1s ease-in-out;
    outline: none;
}
<h2 class="splashPage">What kind of question do you have?</h2>
<button id="Q-debug" class="splashPage" type="submit">I have an error or code that isn't working correctly</button>
<button id="Q-tool" class="splashPage" type="submit">I have a question about a programming tool or IDE</button>
<button id="Q-eng" class="splashPage" type="submit">I don't understand some aspect of a programming language</button>
<button id="Q-other" class="splashPage" type="submit">I want to ask about something else</button>
<div id="debug-page" class="ask-page">
    <div class="container">
        <div id="content">
            <div id="mainbar" class="ask-mainbar">
                <form id="post-form" class="post-form">
                    <div id="question-form">
                        <h2>What language are you using?</h2>
                        <h4>Pick a language tag in the field below, and up to 4 additional tags that relate to what you're trying to do.</h4>
                        <div style="position-relative;">
                            <div style="position-relative;">
                                <label>Tags</label>
                                <div class="tag-editor" style="width: 658px; height: 35px;">
                                    <input placeholder="Pick a tag such as c++, android, or haskell, max 5 tags" style="width: 638px" type="text">
                                </div>
                            </div>
                            <div id="tag-suggestions"></div>
                        </div>
                        <div class="form-submit cbt">
                            <button id="debug-step2" type="button">Next Field</button>
                        </div>
                        <div id="debug-problem-desc">
                            <h2>Describe your problem</h2>
                            <h5>(Don't worry, we'll get to your code next)</h5>
                            <div id="post-editor" class="post-editor">
                                <div style="position-relative;">
                                    <div class="wmd-container">
                                        <textarea id="wmd-input" class="wmd-input processed" name="post-text" cols="92" rows="15" placeholder="Be specific and detailed! No one will help you if they have to guess at the details."></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="step3-container" class="form-submit cbt">
                                <button id="debug-step3" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="debug-mcve">
                            <h2>Include the code that's not working</h2>
                            <h5>(Your actual code, not a screenshot. If you get an error message, include that here)</h5>
                            <div id="post-editor2" class="post-editor">
                                <div style="position-relative;">
                                    <div class="wmd-container">
                                        <textarea id="wmd-input2" class="wmd-input processed" name="post-text2" cols="92" rows="15" placeholder="Include a complete demo of your code problem here. It MUST be reproducible."></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="step4-container" class="form-submit cbt">
                                <button id="debug-step4" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="debug-title">
                        <h2>Pick a title for your question</h2>
                        <h5>(Summarize your problem in question format)</h5>
                            <div style="position-relative">
                                <div class="form-item ask-title">
                                    <input id="title" class="ask-title-field" name="title" maxlength="300" placeholder="Be specific. Use proper grammar. Ex: Why isn't my function returning true?" data-min-length="15" data-max-length="150" autocomplete="off" type="text">
                                </div>
                            </div>
                            <div class="form-submit cbt">
                                <input id="submit-button" value="Post Your Question" type="submit">
                            </div>
                        </div>
                    </div>
                </form>
            </div>
        </div>
    </div>
</div>
<div id="tool-page" class="ask-page">
    <div class="container">
        <div id="tool-content">
            <div id="tool-mainbar" class="ask-mainbar">
                <form id="tool-post-form" class="post-form">
                    <div id="tool-question-form">
                        <h2>What tool are you using?</h2>
                        <h4>Pick a programming tool or IDE in the field below, and up to 4 additional tags that relate to what you're trying to do.</h4>
                        <div style="position-relative;">
                            <div style="position-relative;">
                                <label>Tags</label>
                                <div class="tag-editor" style="width: 658px; height: 35px;">
                                    <input placeholder="Pick a tag such as visual-studio, chrome, or eclipse, max 5 tags" style="width: 638px" type="text">
                                </div>
                            </div>
                            <div class="tag-suggestions"></div>
                        </div>
                        <div class="form-submit cbt">
                            <button id="tool-step2" type="button">Next Field</button>
                        </div>
                        <div id="tool-problem-desc">
                            <h2>Describe your question</h2>
                            <h5>(If you have a problem, specify your desired outcome)</h5>
                            <div id="tool-post-editor" class="post-editor">
                                <div style="position-relative;">
                                    <div id="tool-wmd-container" class="wmd-container">
                                        <textarea id="tool-wmd-input" class="wmd-input processed" name="post-text" cols="92" rows="15" placeholder="Be specific and detailed! No one will help you if they have to guess at the details."></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="tool-step3-container" class="form-submit cbt">
                                <button id="tool-step3" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="tool-mcve">
                            <h2>Include any screenshots here</h2>
                            <h5>(Screenshots should illustrate sections, windows, or menus of the IDE or tool you're using)</h5>
                            <div id="tool-post-editor2" class="post-editor">
                                <div style="position-relative;">
                                    <div id=tool-wmd-container2 class="wmd-container">
                                        <textarea id="tool-wmd-input2" class="wmd-input processed" name="post-text2" cols="92" rows="15" placeholder="Please use Stack Overflow's built-in image uploader. Don't link an image from a 3rd party website."></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="tool-step4-container" class="form-submit cbt">
                                <button id="tool-step4" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="tool-title">
                        <h2>Pick a title for your question</h2>
                        <h5>(Summarize your post in question format)</h5>
                            <div style="position-relative">
                                <div class="form-item ask-title">
                                    <input id="tool-title" class="ask-title-field" name="title" maxlength="300" placeholder="Be specific. Use proper grammar. Ex: How do I dock a window in Visual Studio?" data-min-length="15" data-max-length="150" autocomplete="off" type="text">
                                </div>
                            </div>
                            <div class="form-submit cbt">
                                <input id="tool-submit-button" value="Post Your Question" type="submit">
                            </div>
                        </div>
                    </div>
                </form>
            </div>
        </div>
    </div>
</div>
<div id="knowledge-page" class="ask-page">
    <div class="container">
        <div id="knowledge-content">
            <div id="knowledge-mainbar" class="ask-mainbar">
                <form id="knowledge-post-form" class="post-form">
                    <div id="knowledge-question-form">
                        <h2>What language is your question about?</h2>
                        <h4>Pick a programming language in the field below, and up to 4 additional tags that relate to your question.</h4>
                        <div style="position-relative;">
                            <div style="position-relative;">
                                <label>Tags</label>
                                <div class="tag-editor" style="width: 658px; height: 35px;">
                                    <input placeholder="Pick a tag such as Rust, C, or Python, max 5 tags" style="width: 638px" type="text">
                                </div>
                            </div>
                            <div class="tag-suggestions"></div>
                        </div>
                        <div class="form-submit cbt">
                            <button id="knowledge-step2" type="button">Next Field</button>
                        </div>
                        <div id="knowledge-problem-desc">
                            <h2>Describe your question</h2>
                            <h5>(Be sure to include your expectations or reasoning)</h5>
                            <div id="knowledge-post-editor" class="post-editor">
                                <div style="position-relative;">
                                    <div id="knowledge-wmd-container" class="wmd-container">
                                        <textarea id="knowledge-wmd-input" class="wmd-input processed" name="post-text" cols="92" rows="15" placeholder="Be specific and detailed! No one will help you if they have to guess at the details."></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="knowledge-step3-container" class="form-submit cbt">
                                <button id="knowledge-step3" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="knowledge-mcve">
                            <h2>Include any references or source material</h2>
                            <h5>(List the source of your knowledge or question here, along with anywhere you've already looked for an answer)</h5>
                            <div id="knowledge-post-editor2" class="post-editor">
                                <div style="position-relative;">
                                    <div id=knowledge-wmd-container2 class="wmd-container">
                                        <textarea id="knowledge-wmd-input2" class="wmd-input processed" name="post-text2" cols="92" rows="15" placeholder="Keep it brief; don't list 20 variations of the same source"></textarea>
                                    </div>
                                </div>
                            </div>
                            <div id="knowledge-step4-container" class="form-submit cbt">
                                <button id="knowledge-step4" type="button">Next Field</button>
                            </div>
                        </div>
                        <div id="knowledge-title">
                        <h2>Pick a title for your question</h2>
                        <h5>(Summarize your post in question format)</h5>
                            <div style="position-relative">
                                <div class="form-item ask-title">
                                    <input id="knowledge-title" class="ask-title-field" name="title" maxlength="300" placeholder="Be specific. Use proper grammar. Ex: Why doesn't C++ have garbage collection?" data-min-length="15" data-max-length="150" autocomplete="off" type="text">
                                </div>
                            </div>
                            <div class="form-submit cbt">
                                <input id="knowledge-submit-button" value="Post Your Question" type="submit">
                            </div>
                        </div>
                    </div>
                </form>
            </div>
        </div>
    </div>
</div>
<div id="ontopic-page" class="ask-page">
    <p id="ontopic-blurb"><span>Redirect users to <a href="https://stackoverflow.com/help/on-topic">https://stackoverflow.com/help/on-topic</a> so they can realize their question is probably off-topic, with some helpful callouts (examples below), and a link somewhere that lets them ask anyway:</span></p>
    <p>
        <img alt="On-Topic Help Page" src="https://i.sstatic.net/EUAIt.png">
    </p>
</div>

Before you think "a question wizard would be terrible to use for me, an experienced user", I want to be clear that this demo is a new user wizard. While building the demo, I thought up some criteria for what kinds of new users would be sent to the wizard instead of the blank Ask Question page. For example, when any of the following conditions are met:

  • It is the user's 1st, 2nd, or 3rd undeleted question
  • The user has not asked a question in X months
  • The user has a reputation of 40 or less
  • The user's most recent question was closed as off-topic or was deleted
  • The user has a net negative score on all questions asked

(Or any combination of the above)

Makyen also pointed out that, given a decent wizard implementation, experienced users should be allowed to click a button on the Ask Question page to use the wizard willingly. Some users may prefer this over a blank textbox if the experience is nice enough.

Some reasoning for each of the above conditions:

It is the user's 1st, 2nd, or 3rd undeleted question

Obviously, any software wizard is targeted at new users rather than experienced users. In this case, any time a user asks their first question, even if they have answered many questions or been a member for a long time, it's a good idea to guide them through the process, because sometimes we impose some crazy expectations on askers, and the first few times you do something with such high expectations, you're likely to skip a step/miss some part you didn't think of. I'm not totally sold on the undeleted bit; the thought there is to make sure a user has at least three quality questions on record (users typically don't delete good/positively-scored questions, only bad/closed/negatively-scored ones) before the system considers them ready to be released into the wild.

The user has not asked a question in X months

I am less sold on this one. The reasoning is that familiarity with quality expectations can fade over time, and expectations/on-topic questions change over time as well. A user may ask several good questions, leave the site or not pay attention for a while, and then come back to find the next question they want to ask is now off-topic, or they may forget that you need to include some specific piece of information when debugging your code.

The user has a reputation of 40 or less

This criterion is intended directly for users who are new and/or have not asked many 'good' questions. This threshold would be passed when someone has a question with 4 upvotes, or four questions with one upvote each, or anywhere in between, where any scenario would be a decent indicator that the user has a decent grasp on question expectations. Someone who reaches 40 reputation without asking questions would still be caught by one of the other criteria.

The user's most recent question was closed as off-topic or was deleted

If a user's last question was closed or deleted, that's a strong indicator that they did something wrong; they asked an off-topic question, or their question was low quality, etc. We obviously want their next question to be better, which is where a wizard can be handy, making it obvious what needs to be included in a question.

The user has a net negative score on all questions asked

When a user asks consistently poor questions, there's a pretty good chance they'll be question-banned. Before that kicks in, however, and especially after a question ban is lifted, we want to make sure the user receives guidance so that they don't get banned or re-banned.


This demo unfortunately doesn't illustrate anything like duplicate question prompting (I didn't have time/interest), but I do think Chris Baker's 2014 wizard illustrations cover that need nicely, and could be integrated nicely into a wizard like this.

One point of contention regarding how I structured the demo is that I prompted users for a question title last instead of first. I did this because describing the problem first tends to lead to a better, more descriptive question title.

Prompting a user for the title first has the not-insignificant benefit of letting the system search for duplicates faster/longer before question posting, but it also leads to question titles like "javascript onclick function" and "my code doesn't work", which are absolutely useless at describing what the actual problem/question is. A long-term benefit of more descriptive titles is better search results and, ultimately, more accurate duplicate suggestions. This is one of those situations where I think it will be difficult to have our cake and eat it, too.

Why don't we just make a new site for new askers/users?

That's been suggested, and shot down, multiple times. It comes with its own problems:

Why don't we just use our existing quality enforcement tool - close voting?

For starters, we already do. Without close votes or the review queues, the site would be in a far worse condition... but it's not enough. Most users don't use all their close votes every day, and at any given time the review queue is sitting around 10,000 questions in need of attention. Additionally, close voting is reactionary. It's putting a bandage over a wound each time the wound gets reopened.

What we need is to address the root cause; to change the behavior causing the wound to get reopened in the first place. Unfortunately Stack Overflow is not yet in a position to take over the world and institute training from birth on how to ask a good SO question, so the next best step is addressing the problem as soon as they walk through our door: asking a question.

What's wrong with our existing new user process for asking questions?

While our How to Ask page is not as bad as software EULAs, it's still a hefty wall-o-text that a user is expected to read, engage with (there are several links, each to a new wall-o-text replete with their own sets of links to even more text...), and then signify that they've read by clicking a checkbox, then another button, before finally making it to a separate page to ask their question. You can easily spend 30 minutes clicking through links trying to learn everything there is to know about asking. While such extensive content might seem like meta porn for us users with a vested interest in making the site better, it leaves a bad taste in the mouth of new users who just want to get help with their problem.

Gently corralling users into a wizard that presents them with only enough information for the step they're on, piece-by-piece, helps keep them on track and moving (everyone loves feeling like they're making progress), and it provides the important pieces of information to the users in a way that provides step-by-step education combined with hands-on experience/engagement, which is critical for knowledge retention. Any educator will tell you that giving a student a textbook and testing them on it produces worse results every time compared to teaching them module by module, with an opportunity to apply what they've learned before moving on to the next part.

How will this compare time-wise with the current process?

That's a good question for some A/B test results to answer. Comparing both the time it takes for users to ask a question with these methods, and the end result (answered, closed, up/downvoted, etc.) will provide the key data on whether this kind of thing is worth keeping around or ultimately scrap-worthy.

Your demo doesn't illustrate enough depth/versatility/functionality in some area

This is just a proof-of-concept; it doesn't even present what the end-result would look like, let alone any features that might be useful or necessary to include in a piece-meal wizard like this. Multiple inputs for code or explanations, a place to show exactly what research you've done, a way to reorganize/order input sections after you've filled them out, a method to search the site for similar problems/questions while you type... these are all great suggestions and features that are beyond the scope of my proof-of-concept... please feel free to take my concept (or build your own) and run with it, providing an augmented concept/demo as an answer to showcase some feature you think is important.

25
  • 38
    The user has a reputation of 20 or less Well, you might have to set the bar higher on that one. I see repeat offenders all the way upto 1K and beyond. You should consider setting more elaborate criteria, such as X of their questions have been downvoted.
    – cs95
    Commented Oct 29, 2017 at 6:37
  • 3
    Furthermore, keep in mind that while questions might be on topic, they can still be low quality. "Explain null pointer exception to me" is on topic, but shows 0 effort in research on OP's part. If your wizard cannot figure out how much research OP has put into trying to answer their own questions, I can't see too much use for this.
    – cs95
    Commented Oct 29, 2017 at 6:39
  • 9
    I really like what you've done with the tags > description > code/specifics > title order. It would be interesting to see if it helps new users tag questions better and provide better titles to their questions. Commented Oct 29, 2017 at 6:44
  • 3
    I hate to be glass-half-empty guy, but it would be interesting if the wizard calculated the time difference between the display of its first page and the dismissal of its last and maintained counts of when this value is above or below, say, 60 seconds - a 'click-through' counter. Commented Oct 29, 2017 at 10:56
  • 24
    They should also consider adding an option asking if they plan on returning at a later date or will be present in the question. 1) I am posting my question but leaving, only to return at a later date. 2) I will be present at the time of my posting the question. - type of thing. There are too many people who post and leave, or are not responding to either comments or answers posted. That is also a big problem. I've had that in my thoughts for a while now. Commented Oct 29, 2017 at 13:19
  • 2
    ♫ follow the yellow brick follow the yellow brick follow the yellow brick road... ♫
    – Jon Clements Mod
    Commented Oct 29, 2017 at 20:00
  • 3
    Ack, the demo wizard is horrible. Code, screenshots, and references should be interweaved as part of the prose so that can be explained and given context (e.g. "This JavaScript throws a FooError: some lines of code ... but when I try to fix it like this: some more lines of code I get a BarError instead."). Your proposed question form deters this kind of meaningful structure and instead asks users to dump all their code, references, and screenshots in an unexplained heap at the bottom of the question.
    – Mark Amery
    Commented Oct 29, 2017 at 20:43
  • 6
    The user has a reputation of 20 or less I think this should also include people who have come from a different site. Many other sites seem more forgiving, and asking in c++ is like the end-boss of the internet
    – Tas
    Commented Oct 30, 2017 at 0:12
  • 1
    ooh yeah, nice post TylerH it's really time to address the root cause of our headache, I tried something similar a few years ago but my meta and js-skills are not as good as yours or the time was not mature. It's related has an answer from Ed and some comments from Skeet, Will, Pekka, Robert etc. Commented Oct 30, 2017 at 14:32
  • 1
    @TylerH I'm not suggesting that all code be presented as backticked inline code snippets, although I can see how my comment could be read that way given that I can't show newlines in a comment - I'm suggesting that code blocks be interspersed amongst paragraphs of explanatory prose. Any wizard that asks for code in a separate textbox to prose will necessarily prevent users from laying things out in a sensible way; it's not an "assumption" so much as a fundamental problem with the core design idea.
    – Mark Amery
    Commented Oct 30, 2017 at 16:10
  • 1
    @MarkAmery The next step could easily be a "drag-and-rearrange" view where they can order the code snippet or the prose as they wish, or there can be a tickbox for multiple code snippet entries, etc. Again, this is just a proof-of-concept illustration, not even a fully interactive demo. I'd love to see you take this and improve the robustness in an answer.
    – TylerH
    Commented Oct 30, 2017 at 16:16
  • 1
    @silencedmessage I agree with you completely. As I stated, it was an example ;-) If they do decide to incorporate something like that, that they'd have to word it properly. Commented Oct 30, 2017 at 16:27
  • 2
    I have issue with the inactivity timer suggestion: I've asked 5 questions in 3 & 1/2 years. I've also posted 240 answers. Should I get the wizard? I'd suggest any time constraint be either rep capped (apply only below x rep) and/or include other site interactions (like answers) as resets. Commented Oct 30, 2017 at 20:30
  • 2
    @TemporalWolf Should you get the wizard? Sure, if it's been 12 or 18 months (or whatever time amount is picked) since you last asked. That is addressed in the question above; expectations for askers change over time. It's worth considering more complex criteria, but we shouldn't make the assumption that high-rep or long-term members don't ask bad questions, because they do. Some do repeatedly.
    – TylerH
    Commented Oct 31, 2017 at 14:43
  • 2
    @Llopis This is just a proof-of-concept, not a full demo. Any such implementation would of course make sure it enforces its own paradigms.
    – TylerH
    Commented Oct 31, 2017 at 14:44

4 Answers 4

29

The ideal wizard would require minimal explanation or thinking regarding what to include from the user and also guide them to do things they should preferably do before asking their question.

I think the problem with a simple disclaimer stating that they shouldn't e.g. post a screenshot is easy to ignore - if we're forcing them to read through a few options and pick the one they plan on doing, we can follow that up with a message catered specifically to them that explains the reasoning behind the rules as well as not allowing them to trivially ignore it (by e.g. at least forcing them to pick the correct option before being allowed to continue). It can still be ignored, but then at least we know they're actively choosing to ignore it.

Here's an example of one part of a potential wizard:
(No fancy JS here, hopefully the format makes sense. The questions should be presented one at a time.)

  • My code is giving the wrong output.

    1. What code are you posting?
      1. [x] My original code. It does a few things, some of which maybe works (e.g. reading from file, converting that data and writing it to another file).
      2. [x] A new short program I wrote specifically to narrow the problem down. There aren't any parts of the code that works, because it does exactly one thing, and that's the thing I have a problem with.
    2. Which part of your code are you posting?
      1. [x] Only the relevant function / file / part of the program.
      2. [x] The complete program.
    3. How will you get your code here?
      1. [x] I will write some example code here on the website to bring the basic idea across.
      2. [x] I will post a screenshot.
      3. [x] I will post a link to my code.
      4. [x] I will copy-paste code that I compiled and ran exactly as is.
    4. How have you tried to fix your code?
      1. [x] I have not done anything.
      2. [x] I tried to think what the problem could be.
      3. [x] I tried using a debugger or adding print statements, but I'm not sure how to do this.
      4. [x] I used a debugger to try to figure out where exactly the program is going wrong.

    [Regarding the above points: They are the essence of a Minimal, Complete, and Verifiable example and failure to select the correct option at any point should give an explanation of what's wrong with what they selected as well as why we want the correct option and a reference to the MCVE page.]

    1. What language, tools or libraries are you using? [Tags]
      [text box]
    2. In a few sentences, what exactly do you want your code to do?
      [text area]
    3. Summarise what you wrote above in one short question: [Title]
      [text box]
    4. Post your code here:
      [text area]
    5. What input are you giving to your program?
      [text area]
      [x] The program takes no input
    6. What is the exact output that your program gives?
      [text area]
    7. What is the exact output that you want your program to give?
      [text area]

If we're talking about A/B testing, I might suggest:

  • Splitting the top (MCVE) and bottom (actual question template) parts and testing those separately.
  • Not starting with something too simple, as providing some vague guidance may actually cause more frustration than the good it does.
  • Just implementing one option (e.g. wrong output) at first, letting other options (e.g. error or question about language features) default to the default non-wizard question box.
13
  • 25
    Don't forget "have you read the API documentation". Commented Oct 30, 2017 at 7:47
  • 3
    ...and have you checked for typos? Commented Oct 30, 2017 at 13:10
  • Not a bad fleshing out attempt; regardless of the implementation, I think the key here is an asking experience where a new user is prompted with only a couple lines of text at a time, while simultaneously given an opportunity to ask part of their question. Any implementation that follows that rule of thumb would be a much better experience for users than being sent to read a lengthy instruction manual somewhere else, and then remember it all while asking on a separate page.
    – TylerH
    Commented Oct 30, 2017 at 14:09
  • 8
    [x] I will post a screenshot. I will murder everyone if this is included. Srs.
    – user1228
    Commented Oct 30, 2017 at 15:49
  • 4
    [x] I'll post my programming question on Meta if that guy @Will doesn't write a personalized answer for me here.
    – user50049
    Commented Oct 30, 2017 at 17:09
  • 3
    @Will If you select that option, SO downloads a virus onto your computer that converts all your text files (.docx?, .pdf, .txt, source code, etc.) to images.
    – Nic
    Commented Oct 30, 2017 at 18:59
  • 9
    @QPaysTaxes Don't destroy data, that is impolite; simply replace their web browser with one that shows a static image of a web browser when you run it. Commented Oct 30, 2017 at 19:28
  • You should remove the first #1. I'm not sure that it's really necessary. Start with 1. What code are you posting? The subquestions should be 1.1, 1.2, etc. Otherwise I reference #1 and no one knows which 1. I'm referring to.
    – JeffC
    Commented Oct 30, 2017 at 20:46
  • The main #1 and #2 are redundant. I'm not sure from the formatting if 1.1. is supposed to feed to #2 or ? It's not clear. For #3.1, I wouldn't encourage people to write code in the editor here. It's bad enough when they bring things from some other editor and it has poor whitespacing, typos, and whatnot. You give bad options, e.g. 3.2, 3.3 but you don't describe what messaging you would present if they chose that option. Are you just going to point them at ASK and hope they figure it out or do you plan to provide specific, actionable feedback for this particular problem?
    – JeffC
    Commented Oct 30, 2017 at 20:51
  • @JeffC The intention is that #1 is minimal and #2 is complete. I think people post partial code samples often enough that #1 alone is not enough - they can still just post the 1-2 lines they think is relevant without actually posting the code we need to see. Commented Oct 30, 2017 at 21:03
  • 3
    @JeffC The options aren't meant to be good, they're meant to be things users are thinking of doing, and then we can tell them "hey, no, you shouldn't do that, you should do this instead". Yes, it should be specific actionable feedback on what they selected - I thought including that would take up too much space without much benefit - I don't think it would be hard to find something reasonable to say there, if this gets implemented. An alternative is yes/no options, like "Is this a minimal example?", but it's easy to treat it like a licence agreement and just select "yes" for every option. Commented Oct 30, 2017 at 21:25
  • Problem: this wizard only works for questions asking for debugging help. Although the vast majority of questions today fit, taking a quick look at the top voted list proves that there are other types of questions.
    – clickbait
    Commented Aug 15, 2018 at 2:45
  • @sag Yes, this is only intended to be one part of the final wizard. Commented Aug 15, 2018 at 8:22
17

Random thoughts on this in no specific order (and want to get this information somewhere other than locked inside my head). Apologies if this incoherent dump is incoherent :) This is stuff that sort of applies regardless of the intent of the asker (looking to solve a complier error or wondering why a RDBMS won't install correctly).

Notes on what to capture

  • We ran a test where folks asking their first question took a short survey after asking. Turns out, the overwhelming majority of people actually did something amazing that most of us wouldn't have expected - they actually searched prior to asking. We need to ask them what they searched for, regardless of the question being asked, so we get a better idea of how they understand what they want to accomplish.

  • Most people don't realize that a good title can make or break a question as far as getting decent answers in a reasonable amount of time goes. Consider a "title-good-o-meter" that checks punctuation, length, ratio of common words to uncommon ones, if a tag is present in the title, etc. Think of a password strength meter, but apply it to a title. Use that score when thinking about what should or should not go right to the helper queue.

Notes on when to give 'wizard-y' guidance

  • User has just been rate limited for asking questions due to score or number of questions coming from the same network
  • User has just been unsuspended after being suspended for consistent low quality over time
  • User is shiny and brand new AND has fewer than [x] questions with [n] aggregate score
  • High number of previous attempts to post being blocked by simple quality heuristics (e.g. kept forgetting to add tags, or otherwise obviously having a difficult time using the standard form). You'd be surprised here, some people have to submit the page 5 or 6 times before actually getting it to post.

Goals

  • Let's primarily help people have a better experience. If we focus on that, we stand a better chance of actually getting this right, which means we stand a better chance of actually raising question quality in a noticeable way. Huge walls of text or over-complication won't help us meet the end goal.

  • Let's try to make it easy to figure out what we got wrong, which is going to happen. If we make major changes to the ask page, it's going to be difficult to understand what parts helped, and what parts hurt. We're thinking 'small, incremental changes backed by tests' once we get the basic wizard layout / concept done.

  • We test the efficacy of changes by looking at questions once scores typically stop moving, which is about 72 hours according to our data team. Hence, small copy changes need at least a week to test in order to see what (If any) difference they made. This makes less explanation through copy ideal, with more emphasis on simple flow / design.

I'm going to check with Shog and a few others that have been kicking this idea around internally for the last few years and see what else we came up with that's valuable and make some additional edits in a bit, but I didn't want folks brainstorming too hard without us putting out what we'd already come up with out where it might actually do some good.

More to come in a bit.

8
  • Thanks, Tim. A lot of things I had kicking around in my head while building this were assumptions that could go either way depending on what kind of info you had already tested/gathered yourself.
    – TylerH
    Commented Oct 30, 2017 at 16:35
  • @TylerH I've pinged folks that have been working on the idea to chime n here with edits, so that at least we've got everything that we thought could be a good idea out there for folks to chew on. It's a crazy week so give folks a day or so, but we should have some more here soon. Thanks for getting this going!
    – user50049
    Commented Oct 30, 2017 at 16:40
  • 2
    I'm very curious about what they searched for, considering how often I can find an answer to an asked question by searching the first thing I can think of, their title word-for-word or the error message as is. Commented Oct 30, 2017 at 17:16
  • @Dukeling It's going to be an interesting experiment. My prediction is a lot of stuff like folks not knowing "dependency injection" is a term, so searching for "insert class into a class" and getting stuff about DB abstraction, or folks seeing a slightly different error message and not realizing it's actually the same thing - stuff like that. But we'll see. We'll either see where the disconnect is very quickly, or it will just be very apparent that they didn't in fact search (or at least didn't read many of the results).
    – user50049
    Commented Oct 30, 2017 at 17:38
  • @Dukeling And in some cases, it might be the language itself fighting against the user... if they try to search for error messages, that is. Error messages in VBA and, to some extent, JavaScript, are essentially useless for figuring out what you did wrong, for example.
    – TylerH
    Commented Oct 30, 2017 at 18:09
  • 1
    @TylerH <shudders> That brings back the 300 or so times I've seen folks in the Android tag not know there was such a thing as logcat :) They were trying to do something simple and figured they'd just sit down, muddle through it and learn whatever they needed to learn as they went along.
    – user50049
    Commented Oct 30, 2017 at 18:13
  • 1
    "We need to ask them what they searched for". Often they searched, from my experience in the mentorship program, for all sorts of stuff unrelated to their problem because they didn't know what to look for. So instead of finding what they needed, the just asked Stack Overflow in a generic enough way so they could fish for more terms. Keep in mind, this isn't all new users, this was mostly just the bunch that had ridiculously vague questions with only 150 characters of text.
    – Travis J
    Commented Oct 30, 2017 at 19:14
  • Looks like those pings for edits were effective :)
    – user4639281
    Commented Jan 3, 2018 at 0:20
6

I like the idea of identifying on topic buckets!

While I am not sure if the buckets described are ideal enough, that is just a detail issue. In my opinion, the majority of value from this suggested feature is the on topic buckets for a wizard starting point.

Overall, I think this is a good approach. I really like that it is introducing tags right at the start. This allows for so much optimization down the line in the wizard, such as targeting tag based similar questions and perhaps tag based mentors. It makes things more accurate.

I think it is easy to use. While I personally would not want to use a wizard, I am not a new user. From a new user, or a user unaccustomed to writing questions, this is a definite improvement. There are also many places between steps to introduce suggestions or warnings based on the progress of the post.

As a side note, I very rarely ask questions at Stack Overflow. If I do there are like 4 browsers and 200 tabs open. As a result, the frequency I ask questions with is very small and I would probably not want to be hit with the wizard. In this regard, "The user has not asked a question in X months" could use some work. I don't mind the other triggers.

4
  • You say you rarely ask questions... but you have over 200 of them! :-P I initially was thinking something like 6 or 12 months for the 'not asked in X months' criterion, but figured putting in a specific number would freak someone out, so I just left it ambiguous.
    – TylerH
    Commented Oct 30, 2017 at 19:45
  • What about off-topic buckets? what if they are asking about (bad example, I know) the concept of infinte loops in general, rather than something about a specific programming language? Shouldn't we direct them to SoftwareEngineering.SE?
    – NH.
    Commented Oct 30, 2017 at 20:14
  • @NH. - I don't think this will solve all problems. So long as it makes even a small percentage of difference, that will result in a very large amount of quality improvement.
    – Travis J
    Commented Oct 30, 2017 at 20:21
  • @TylerH - I think back when I was asking questions, it was about one per every 3 months. Also, perhaps it could integrate with the Curious badge, "Ask a well-received question on 5 separate days, and maintain a positive question record."
    – Travis J
    Commented Oct 30, 2017 at 20:50
-10

I think the first goal for the wizard should be to satisfy the user's need for help before they post the question. Failing that, the wizard should extract enough information from the user to prepare a decent help-me question.

The first goal obviously cannot be satisfies with simple text search, as SO already does that.

So I propose to extract more structured information out of the user - information that allows for more targeted help, better matching of the user's situation to the existing SO knowledge.

Here is what I would suggest, in the form of some examples

Top Level

  1. My program is not working.
  2. I don't understand why this code works.
  3. What is the meaning of this code?
  4. Other

1. "My program is not working."

What language is your program written in?

1.1 C++

1.1 C

1.2 Java

1.3 JavaScript

1.4 Python

1.5 C#

1.6 Other

1.1 C++

Do you have

1.1.1 Compiler error (explain)

1.1.2 Linker error (explain)

1.1.3 The code does not print what I want it to print

1.1.4 Runtime error (explain exception, aborts, memory corruption)

1.1.3 Output of the code does not match the expected output

1.1.3.1 Please try using the debugger (ask for environment, explain how to use debugger)

1.1.3.2 Please provide the following

Exact code you're running (explain MCVE, limit the amount of code accepted, possibly use one of online compiler sites to verify) The exact output (explain copy/paste) The expected output

1.1.4 Runtime error

Please provide the exact text of the error

Perform a search on canonical questions for C++ runtime errors, present answers. If none fit, proceed as for 1.1.3

1.2 Java

1.2.1 Syntax error

1.2.2 Runtime error

1.2.3 Exception thrown

1.2.3 The code does the wrong thing

1.2.3 Exception thrown What is the exception (explain how to get both the message and the stack trace) Perform a search for canonical Java Exception questions

1.2.3.1 NPE 1.2.3.2 ClassNotFound ...

1.2.3.1 NPE

Explain how to analyse the line where NPE was thrown (notNull.call()) Explain how to track the null through the code If help is still needed, proceed with collecting the Java MCVE

16
  • 3
    The wizard shouldn't get into the specifics of the errors. That will just result in too many "mis-categorized" errors.
    – Cerbrus
    Commented Oct 30, 2017 at 15:28
  • 1
    Getting into specifics, on the other hand, allows us to provide lots of targeted help to the user and possibly answer the question before it's posted. It's a balance.
    – user3458
    Commented Oct 30, 2017 at 15:29
  • 1
    Feel free to start writing the specifics for all languages SO supports, then. Including decision trees to make the wizard work. Good luck keeping that up-to-date. (KISS)
    – Cerbrus
    Commented Oct 30, 2017 at 15:31
  • SO would have to come up with some way to community-source this, similar to canonical NPE question/answer for Java. This level of support would have to be limited to, say, half-dozen most popular languages (or rather the languages with the community that is wiling to maintain the relevant data).
    – user3458
    Commented Oct 30, 2017 at 15:34
  • Now, what if a question is tagged [php] and [javascript]? What specifics would you show then?
    – Cerbrus
    Commented Oct 30, 2017 at 15:35
  • I am not familiar enough PHP or JS. But I believe :) there is a useful way to classify the user's problems to provide more help. JS and PHP experts, please chime in
    – user3458
    Commented Oct 30, 2017 at 15:37
  • 1
    As a JS gold badge owner, let me tell you that this would be a beast to maintain, assuming SO can even get it to work decently.
    – Cerbrus
    Commented Oct 30, 2017 at 15:39
  • OK, maybe JS is not a good candidate for this approach.
    – user3458
    Commented Oct 30, 2017 at 15:39
  • 1
    So, you're excluding the largest tag, now. I'm not saying we shouldn't have these kinds of steps, but it should be more generic / language-independent.
    – Cerbrus
    Commented Oct 30, 2017 at 15:42
  • @Cerberus, the goal I have in mind is to fix the user's issue before they post a question. With that goal - is there anything you can suggest that would help with the most repetitive/most annoying JS questions?
    – user3458
    Commented Oct 30, 2017 at 15:46
  • 3
    I don't think the average asker would know the difference between compiler, linker and runtime errors. I might be able to get behind the idea of having the user post the error message and pointing to the canonical post for that error (top-voted, presumably), but even that only works very selectively, as users tend to think their question is different from the canonical post and I think SO search struggles when some of the words in the error can vary (e.g. errors including method or variable names), and the user may post the wrong part of the error or too much of it. Commented Oct 30, 2017 at 16:02
  • 1
    @Dukeling as users tend to think their question is different from the canonical post. This is very true. The amount of times we see null reference errors, mark the question as a duplicate and the author cries out telling us that it's not the same error. It is, but the problem is the author doesn't want to know about the error and possible fixes, they want to know how to fix their error because they cannot be bothered to read.
    – Bugs
    Commented Oct 30, 2017 at 16:25
  • 3
    While I sympathize with this desire, I don't think something of this scale is possible to implement for a site where literally any programming language is ostensibly fair game, let alone feasible to maintain once implemented. The system would have to be robust enough to know about every possible feature of every programming language and provide a context-specific set of questions for each. Maybe some day when we have been assimilated by the Borg. :-)
    – TylerH
    Commented Oct 30, 2017 at 16:39
  • 1
    Make it community-sourced and leave to communities to provide the desired level of details.
    – user3458
    Commented Oct 30, 2017 at 16:41
  • 1
    Just let Akinator (us.akinator.com/) take a stab at categorizing questions. Commented Oct 31, 2017 at 18:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .