79

We've been linking to http://sscce.org/ in one of the custom Off Topic reasons on Stack Overflow for a while now. I added that link because there's a ton of good advice there, but... It's not really ideal for a few reasons:

  1. It isn't particularly concise. Folks hate reading. Especially the folks who ask crap questions on Stack Overflow. Handing them a 2K word document is unlikely to be effective.

  2. It's not specific to Stack Overflow. Even the folks who want to read and understand the guidance are going to have trouble relating it to the problems they're encountering (or causing...) on Stack Overflow - there's just too much irrelevant stuff in there, and even more good stuff that could be even better if it was specifically tailored to the environment and practices here on SO. It's this concern that prompted me to raise this discussion.

  3. Folks think a failure to follow all of the guidelines is a reason to close. This is kinda depressing, but... It's true. Andrew Thompson, the author of SSCCE, called it back in July, and we've seen it crop up several times since then (example). This is related to #2, in that we can't easily call out areas where something is mandatory (including the code in your post vs. linking to it) and where it's just a good idea (self-contained isn't always necessary, although the goals underpinning it usually are).

So with that in mind, what do you think belongs in a "SO-SSCCE"?

Some helpful references:

See also: What guidance should we give people asking Unclear questions on Stack Overflow?

14
  • 2
    Perhaps it would be a good idea to make an example? </shamelessPlug>
    – Pekka
    Commented Jan 4, 2014 at 21:45
  • Maybe - if they're short and clearly illustrate the surrounding guidance. Let's not put the cart before the horse.
    – Shog9 Mod
    Commented Jan 4, 2014 at 21:48
  • 1
    When it was written, I was primarily posting to the now defunct Usenet on Java based groups (though I also dabbled in the HTML and JS groups). I'd started the document thinking 'make it very generic' but that thought got lost in the process. It is too specific to Java. Also, as you alluded to, it can seem ..ironic to someone who is referred to it, to arrive at such a long document. Commented Jan 4, 2014 at 22:10
  • Doesn't the nature of a SSCCE vastly differ from tag to tag? I'm finding it very challenging trying to come up with an all-encompassing definition...
    – Pekka
    Commented Jan 4, 2014 at 23:06
  • For the record, I think you've misunderstood me slightly - I don't think that failure to follow all of the guidelines is always a reason to close, but I do think that for each of the guidelines, there exist some questions in which violating just that one guideline is enough reason to close. I'm kind of perversely honoured that you thought my proposal was so off-base it was worthy of being used as an example of getting the close reasons horribly wrong, though.
    – Mark Amery
    Commented Jan 5, 2014 at 0:04
  • 1
    The bigger problem is simply that the SSCCE link was intended as a helpful resource for folks looking to improve their question, not as a problem description, @Mark. But, this isn't entirely clear from the close reason - and that's my fault.
    – Shog9 Mod
    Commented Jan 5, 2014 at 0:12
  • It could, @Pëkka - but I think there are some generally-applicable guidelines to be had.
    – Shog9 Mod
    Commented Jan 5, 2014 at 0:13
  • 6
    You had me at, "Folks hate reading." But then you lost me.
    – raven
    Commented Jan 5, 2014 at 5:50
  • 1
    You should know by now to stick to reading the bold bits in my posts, @raven
    – Shog9 Mod
    Commented Jan 5, 2014 at 5:51
  • 2
    Don't you get tired of having good ideas? Commented Jan 8, 2014 at 10:08
  • 2
    @Pëkka, I actually think that BAD examples might be more helpful. For instance an example for each criteria which only fails that criteria whilst respecting the others. After all, the whole art of writing a good question is projecting yourself into the position of the potential answerer: temporarily putting our 'pupils' in that position would/should help them understand what it's like to be on the receiving end. Maybe. And examples might need less reading.
    – Benjol
    Commented Jan 9, 2014 at 8:53
  • I do want to keep this here as a place to suggest improvements, @Andrew. I'll be integrating edits into this page provided they're constructive (per my subjective and often capricious perspective). Feel free to link to either.
    – Shog9 Mod
    Commented Jan 9, 2014 at 17:26
  • That - is - excellent.. I'll link to the official page. :-) Commented Jan 9, 2014 at 17:28
  • @Shog9: Would it be possible to get the URL of the help center page changed to match the new title / acronym? Commented Mar 22, 2014 at 18:08

5 Answers 5

35

How to create a Minimal, Complete, and Verifiable example

When asking a question about a problem caused by your code, you will get much better answers if you provide code people can use to reproduce the problem. That code should be…

  • …Minimal – Use as little code as possible that still produces the same problem
  • …Complete – Provide all parts needed to reproduce the problem
  • …Verifiable - Test the code you're about to provide to make sure it reproduces the problem

Minimal

The more code there is to go through, the less likely people can find your problem. Streamline your example in one of two ways:

  1. Restart from scratch. Create a new program, adding in only what is needed to see the problem. This can be faster for vast systems where you think you already know the source of the problem. Also useful if you can't post the original code publicly for legal or ethical reasons.
  2. Divide and conquer. When you have a small amount of code, but the source of the problem is entirely unclear, start removing code a bit at a time until the problem disappears – then add the last part back.

Minimal and readable

Minimal does not mean terse - don't sacrifice communication to brevity. Use consistent naming and indentation, and include comments if needed to explain portions of the code. Most code editors have a shortcut for formatting code - find it, and use it! Also, don't use tabs - they may look good in your editor, but they'll just make a mess on Stack Overflow.

Complete

Make sure all information necessary to reproduce the problem is included:

  • Some people might be prepared to load the parts up, and actually try them to test the answer they're about to post.
  • The problem might not be in the part you suspect it is, but another part entirely.

If the problem requires some server-side code as well as an XML-based configuration file, include them both. If a web page problem requires HTML, some JavaScript and a stylesheet, include all three.

Verifiable

To help you solve your problem, others will need to verify that it exists:

  • Eliminate any issues that aren't relevant to your question. If your question isn’t about a compiler error, ensure that there are no compile-time errors. Use a program such as JSLint to validate interpreted languages. Validate any HTML or XML.

  • Ensure that the example actually reproduces the problem! If you inadvertently fixed the problem while composing the example but didn't test it again, you'd want to know that before asking someone else to help.

It might help to shut the system down and restart it, or transport the example to a fresh machine to confirm it really does provide an example of the problem.

For more information on how to debug your program so you can create a minimal example, Eric Lippert has a fantastic blog post on the subject: How do debug small programs.

12
  • 4
    I quite like this. Made a few tweaks, but it remains fairly short.
    – Shog9 Mod
    Commented Jan 9, 2014 at 3:32
  • I like this as a base, but I think it's still a bit unclear at points (particularly 'valid' which seems to be the SSCCE 'correct' along with 'formatted' as it is now). Rather than edit it, I tossed my own (vastly simplified) version below. I think that this one is great for explaining how to do it, but is less clear on why it should be done. I'm not sure which is more of a source of confusion for users.
    – jmac
    Commented Jan 9, 2014 at 4:58
  • 1
    @Andrew, of course! The goal is to create a great resource for users -- not to nitpick over who created what. Please feel free to use whatever makes sense!
    – jmac
    Commented Jan 9, 2014 at 5:08
  • 1
    You know, if you switched the order of these a bit you could have some "wonderful" confusion with MVC. Not that its a good thing, just amusing to think about when people ask for an MVC and getting back the response of "but I'm asking about how to use that pattern!"
    – user289086
    Commented Jan 9, 2014 at 17:37
  • There's a convenient definition of complete. SSCCE.org captured this idea: "It is important to ensure that the code you relate to others can be 'copied, pasted, compiled, run' so that they can help you quickly and with a minimum of fuss." This draft does not, and is deficient, IMHO. I'm not interested in having to guess through which headers are included and whatever else may be missing. I'm not interested in whether the problem can be reproduced with the information provided, I want a complete reproducer which is according to the definition copy, paste, compile, run. Commented Jan 13, 2014 at 5:57
  • @RobertCrovella "..important to ensure that the code you relate to others can be 'copied, pasted, compiled, run'" You are the second person I've heard comment that they particularly like that part. My experiences suggest that perhaps we should include it in some form in the document. I write 'in some form' for two reasons. 1) The word 'compile' implies it does not apply to a compilation error, whereas I feel the example should be useful for those as well. 2) 'compile' never applies to a JavaScript problem, or an HTML layout problem. - Perhaps someone has an idea how we might fix it? Commented Jan 13, 2014 at 10:01
  • '..copy/paste, see problem?'.. Commented Jan 13, 2014 at 10:03
  • Yes, in the case of compile errors, the provided code should be sufficient with no required additions or changes, to be copy, pasted, and compiled, and generate the compile error at that point. And I agree that these points vary depending on the underlying language, but those seem like nits against the general problem of folks providing insufficient code to work with. If there is convenient terminology to address it, great. Commented Jan 13, 2014 at 14:40
  • My bigger concern is not the detail wording of this, but that the recent change to SO completely obliterates anything like this as an off topic close reason option, even though it is clearly stated in the SO off-topic examples. Why remove it/obscure it? It is the first example given for questions that are off-topic, why not make it an easily selectable option for close reasons (as it used to be)? The change was bad bad bad IMHO. Commented Jan 13, 2014 at 14:42
  • This was never supposed to be an off-topic reason, @Robert - it was supposed to be guidance for folks whose questions were closed because they forgot to include usable code. We'll be bringing back something for that eventually, but I wanted to get this nailed down first.
    – Shog9 Mod
    Commented Jan 16, 2014 at 23:41
  • 1
    Your statements don't make any sense to me. If that is, as stated, a reason why a question might be off-topic, it seems perfectly valid to have an off-topic close reason for it. Furthermore, under what reason do we close a question because someone forgot to include usable code?(excerpting your words exactly -- you are suggesting this could happen). Anyway the lack of a minimal complete correct example for many questions is a serious issue and a close reason is needed for it. It was working just fine IMHO, and I'm not sure why it was removed. Commented Jan 17, 2014 at 0:06
  • If there is no code, we can't reproduce it to figure out what they are talking about (unclear what you're asking), or they are asking us to write an entire code example to reproduce their error (too broad). In the former case this is helpful advice on how to make their question clear by providing code that focuses on that.
    – jmac
    Commented Jan 17, 2014 at 0:13
7

Minimal Complete Correct Example

  • Minimal - Make it as short as practical.
  • Complete - Include everything a person might need to recreate the problem.
  • Correct - Validate the separate parts. Use the naming conventions and code indentation your audience expects.
  • Example - Provides an actual example of the problem.

Minimal

This can be done one of two ways.

  1. Restart from scratch - Adding in only what is needed to see the problem. This can be faster for vast systems where you think you already know the source of the problem.
  2. Divide and conquer - Where you have a reasonable amount of code, but the source of the problem is entirely unclear.

Complete

If the problem requires some server-side code as well as an XML based configuration file, include them both. If a web page problem requires HTML, some JavaScript & a stylesheet, include all 3.

This is a good idea for two reasons:

  1. Some people might be prepared to load the parts up, and actually try them. The acid test of the changes they are about to suggest.
  2. The problem might not be in the part you suspect it is, but another part entirely.

Correct

Validate the HTML or XML. Run the code of a run-time problem through a compiler and check there are no compile-time errors.

Don't reduce code to a single line just to make it shorter! It is important that people who read it are able to understand what they are reading. Use the accepted naming conventions of that language for the same reason & use meaningful variable names. nameOfPerson is a lot more clear than n, and helps others keep track of what is going on.

Example

Ensure to the best of your ability that the MCCE actually reproduces the problem! Sometimes a problem might be fixed in the course of your changes, but for lack of a server restart or browser refresh, you do not see it work.

It might help to shut the system down and restart it, or transport the example to a fresh machine to confirm it really does provide an example of the problem.


That is my first thoughts on reducing the concept down to 'about a screen full' of text. The only reservation I have of it, is that ideally I'd like to have each letter different, so alternate words for one of 'minimal' (which I like a lot) or 'correct' would be welcome.

Thoughts?


In retrospect, I have decided that the following should not be included.

..& use meaningful variable names. nameOfPerson is a lot more clear than n, and helps others keep track of what is going on.

The two primary reasons for this are:

  1. I have seen many problems around SO which have text like "Say I have interface A which is implemented by classes B and C..". If the person knows how to abstract a problem, it might be solved within minutes. I don't want to exclude those types of codes from being considered as 'the rest of the abbreviation'.
  2. Many people speak English as a 2nd (3rd or 4th) language. Asking them to put 'meaningful' names might be embarrassing to them. (It never ceases to make me cringe when people feel the need to apologize for their English, mostly because of the many, many times I have received a great solution from people who speak 'my' language about as well as I speak their native tongue (I only know how to speak English). Further, I've managed to solve many problems where the class and member names are obviously not written in English, yet the problem is obvious.

So on the occasions (and there are not many of them) where we need more meaningful member names, it is not too hard to ask WTE:

Can you make a 'best effort' to translate/change that to more meaningful names? It might help me to understand & solve the problem. Don't worry too much if they are spelled wrong.

Or something like that.

In any case, by losing that part, we then have the 'space' to cover 'logical and consistent code indent'. Which I forgot to mention in this version, and is much more important. I.E. Some compiler errors are caused by misplaced 'end of code block' closing brackets, and a misplaced bracket might change the flow of the program, causing the bug.

It is much more important to focus on those. Plus, most IDEs have a shortcut to format code, it would not hurt to push that ability - it can save programmers a great deal of time and stress.


I will leave this version here, in case other people feel it is better then my new 'preferred version'. We can 'let the votes slug it out' as to which version is better, and if either version is 'best' of the many proposed ideas.

3
  • Maybe an alternate for 'correct' would be 'valid'. I'm just not sure that conveys the naming conventions and meaningful variable names.. Commented Jan 8, 2014 at 7:41
  • ..the more I think on 'valid' in place of 'correct', the more I'm liking it. Correct sounds too much like a 'temperance preacher', and people might mistake 'correct' for 'OK & working'. The only part of the two paragraphs that does not completely fit 'valid' is 'meaningful variable names'.. Commented Jan 8, 2014 at 7:55
  • I deleted this as redundant, but then realized that some of the people who knew me & the SSCCE might like some context behind why the seeming 'radical shift in words' of the abbreviation. This answer explains better. Commented Jan 10, 2014 at 2:26
5

I feel like some of the above examples were a little too focused on rewriting the original SSCCE document, rather than focusing the user on the really important part of what we want them to do -- provide code that we can use to reproduce their problem. This is a variation on Andrew Barber's attempt above that simplifies the message even further.


Providing Sample Code to Reproduce the Problem

When asking a question about a problem caused by your code, you will get much better answers if you provide code people can use to reproduce the problem. That code should be:

  • Minimal - Use as little code as possible that still produces the same problem
  • Complete - Provide all parts needed to reproduce the problem
  • Tested - Test the code you're about to provide to make sure it reproduces the problem

Minimal

The more code there is to go through, the less likely people can find your problem. Before posting your question, you should do some testing to figure out which part of your code is causing the problem. Simplify your code just to include the relevant parts that cause the problem with as little noise as possible to get better answers.

Complete

Make sure that you include all the information necessary to reproduce the problem. If your query breaks only with certain data, include the data it breaks with. If your javascript works fine without a stylesheet, make sure you include the stylesheet it breaks with. If your code relies on data from a query we can't access, include sample data so we can test the code without the query.

Tested

Make sure that what you've provided actually reproduces the problem as-is. Open up a new window, test out the code you've posted, and see if you get the same problem you are looking for help with. If you can reproduce the problem, chances are we can too, and we can start looking for the cause to get you an answer.


Resources

If you need more detail on how to do the above, there is a more comprehensive guide at SSCCE.org. For more advice on how to improve the quality of your question, here are some helpful articles:

4
  • the sentence about style sheets under the heading complete is hard to follow. is there a typo there maybe? Commented Jan 9, 2014 at 5:39
  • @Jess, not a typo, just poorly written. Revised and hopefully easier to understand now.
    – jmac
    Commented Jan 9, 2014 at 5:44
  • yes, better :-) Commented Jan 9, 2014 at 16:12
  • There's some good stuff here, particularly the introduction. Can you perhaps edit Andrew Thompson's answer (now accepted) so that I can integrate these into the live helpcenter page?
    – Shog9 Mod
    Commented Jan 9, 2014 at 17:20
3

OK, here's some random rambling. I may try to distill this into an actual draft guidance later (maybe in a new post), but for now, it's just a collection of ideas. Feel free to edit or comment.

Thoughts:

  • The meta.TeX.SE thread is made of awesome. We should try to emulate it.

    • The specific tips are, alas, specific. To TeX, that is. If we made a FAQ post about this here on meta, we could maybe have a bunch of language-specific answers in addition to the main answer.

  • There are two main ways to produce an SSCCE / MWE: the restart-from-sratch method and the binary division method. Both should be mentioned, at least briefly.

  • An example is self-contained / working enough when someone else can copy and test it in a standard environment. It does not need to include lengthy boilerplate, if that boilerplate really is standard, but it should not depend on any nonstandard stuff you have and others don't. If a non-standard dependency really is unavoidable, explain how to set up the environment needed to reproduce the issue.

Issues:

  • (How much) should we encourage the use of external services like JSFiddle or Ideone? On one hand, they provide an explicit standard environment: if you can reproduce the problem there, so can anyone else. On the other hand, we do want people to trim down their code and post it here on SE, rather than dumping a huge mess of HTML / CSS / JavaScript on JSFiddle.

  • Generating a minimal example is an excellent debugging technique — you can often find and solve the problem yourself while doing so. Great! But if you've already asked a question about it, what to do: withdraw the question or self-answer it? Both can make sense in different cases.

4
  • 2
    First issue: meta.stackexchange.com/questions/125997/… Second issue: depends largely on whether the problem was where you thought it was or not. If you were simply able to develop a solution along the way, great - post it! If it turned out the problem was somewhere else (a typo, or literally in a separate bit of code), then might as well withdraw.
    – Shog9 Mod
    Commented Jan 4, 2014 at 22:56
  • 3
    An example is self-contained / working enough when someone else can copy and test it in a standard environment. Occasionally this asks too much - like if the entire point of a question is that you're trying to track down a problem that you're seeing in the wild but struggling to reproduce. Questions like "this innocuous-looking standard library function works fine on my local machine, but hangs on a production server that has the same version of the language. What environment difference could be responsible?" fail your criteria but - IMO - this doesn't make them bad questions.
    – Mark Amery
    Commented Jan 5, 2014 at 0:28
  • 1
    @MarkAmery: Agreed, and that should be noted. More generally, SSCCE / MWE is an ideal, but not always attainable. Doesn't mean one shouldn't try to get as close as possible. Commented Jan 5, 2014 at 0:40
  • Your idea re. the two different ways to build an example was such an excellent one that I put a form of it into my answer of a proposed alternate document. Commented Jan 8, 2014 at 7:34
3

I am new to Java programming language and SO. I had a problem on this website in terms of creating a correct question.

For the full story you can check Why was my question about Java Array.length closed?

My experience with SSCCE:

  1. First time I saw the link to SSCCE it was in comments to some question without answer. The person in comments who was Andrew Thompson gave a link asking to give a code example. I briefly looked at the document and the author was Andrew Thompson. Looked like a cheap self-PR at the expense of SO community to me, so I ignored it.

  2. Encountered it several times in other topics, so got back to it and read it carefully(It is not that big, honestly you are being over dramatic about the size.)

  3. When I decided to create my own question I read help center pages as well as SSCCE again, trying to follow both.

My conclusion:

  1. As much as SSCCE is a good document that describes good practices on creating a question it is completely useless for the following reasons:

    • People who are experienced enough in writing code can localize the problem area of their code quickly and submit the question easily anyways. SSCCE is of no use to them.

    • People who are new to programming will not be able to follow all the guidelines even if they tried to read it and understood, especially that is true for low-level languages where it is not clear which part of the program will be compilable and complete and which should be excluded. It took me about 2 hours to create a suitable piece of code(still wasn't good enough).

    • It is incomplete: In the link above Shog gave me better advice on how to create the question that the whole load of documents that I've read, including SSCCE.

    • It is used as a waiver: Much easier to say "post SSCCE", then when posted code is not good enough for some reason to close the question quietly than to try and actually answer it.

  2. In its current form SSCCE should be "dis-attached" from SO and reformatted.

Answering the question:

  1. I think it would be a good idea to make SO-SSCCE language specific and include the actual examples of code that were incorrect and became correct after editing(I bet there are plenty of those in the history that might just be copied and pasted).

  2. Create a special noob-tag, that would allow more experienced programmers to skip the question without being annoyed by asker's mistakes in general and about SSCCE in particular and to close the question prematurely. This tag would allow other programmers who really don't mind helping noobs to reach the zen of SSCCE and edit their question, if necessary.

  3. If SSCCE is an attempt to automate users' questions to be of the right format without need of editing every single question it fails badly as of now. But even if it was better there is only a certain percentage of people(those, who actually would read it) to whom it might be helpful anyways. My point here is that as of now reading it, and following it did not help me much(I might be retarted).

I was not able to see what SSCCE really is before my question was edited even though I read the document a couple of times.

For now I see SSCCE.org as sort of 10 commandments of bible that are good overall, but are not applicable in a real particular situation and impossible to follow thoroughly, especially for a new user.

2
  • 5
    I spent considerably more than 2 hours trying to distill a problem down into a form where it could be presented as an SSCCE. It started at over 3000 lines of code; it was down to about 200 (and nearly ready for presentation) when I finally worked out what the problem was (stack overflow of the other sort; an attempt to recurse too deeply -- 8 MiB of stack was a little shy of what I needed, it turned out). But the symptoms (running the test with make version 1 gave different results from running with make version 2 -- running not compiling! -- were weird). A good SSCCE can be hard work. Commented Jan 7, 2014 at 1:09
  • True, but SSCCE document seems to be around for a while already. I believe during this period of time there should have been enough people to state the obvious complaints about its ambiguity. It might be hard to create a decent SSCCE, but more specific guidelines would definitely narrow down thinking and could potentially put a person on the right track with higher probability.
    – user3081519
    Commented Jan 7, 2014 at 1:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .