11

Problem

Browsing the list of unanswered JasperReports questions reveals a trend: most people don't provide enough information to answer their question, which results in their problem being ignored, down-voted, or--in time--closed. When asking questions about reports, any of the following key pieces of information would increase the odds of getting an answer:

  • The XML code for the JasperReport template that isn't working.
  • Any exceptions (stack traces) or error messages that appear.
  • Pictures of what the report is doing vs. what they want it to do.
  • A minimal working example that demonstrates the problem.

Although I have no data, it seems to be the case that ignoring, down-voting, or immediately closing the question causes the person to "go away" and not come back, which is neither supportive nor helpful, nor useful for any Googlers who hit upon similar keywords.

Feature

It would be nice if there was a way to inform the user upon submitting a question that there is probably insufficient information included for anyone to provide a helpful answer.

The easiest way would be to detect for the presence (or lack thereof) of a JasperReports template (some XML code) and when the user clicks Post Your Question, prompt them with a warning. It would be great if the warning included an example of a question that will increase the likelihood of their problem getting answered. The warning would have a Continue Anyway button for those who opt to ignore the warning or a Revise Question button that allows the user to provide more information in the question.

This could be broadly applicable.

Unanswerable Questions

It is evident that few people read the FAQ or help (and if they do, they might not understand it due to language issues). In other words, it's a waste of time for people who frequent that tag to prompt authors of questions with, "Show code. Add examples. What's the error?" when there are ways to guide people into asking answerable questions.

Answerable Question Metrics

You could probably calculate the odds of asking an answerable question based on quantifiable, weighted metrics. Here are some examples--not authoritative or exhaustive or even necessarily useful--that are meant to stimulate conversation about the types of feasible metrics that could be measured (if any exist):

  • how many questions the user has asked previously
  • how many answered questions the user has asked
  • word count
  • inclusion of code
  • inclusion of images
  • the word "error" without either substring of "Exception" or "Trace"
  • whether the user has read the FAQ and/or minimum working examples link
    • (this could be improved by adding an example MWE on a per-tag basis)
  • how well the English grammar parses

and other metrics. Given the vast amount of data in this tag alone, it'd be interesting to perform an analysis of what makes for an answerable question to see if there are any commonalities. This analysis might be possible to perform across a variety of tags so that every analyzed tag could include it's own question-to-answerable ratio metric. That would allow a fuzzily-quantifiable indicator to appear, much like a password strength measure, that informs the person of the likelihood that their question can be answered.

Machine Learning and Heuristics

Another approach to this problem, which eliminates the need for human-defined weightings based on hypothetical metrics, is to apply machine learning, similar to how Google killed spam.

Use machine learning to calculate the probability of their question being answered. If the probability falls below 40%, don't let new users submit without more details. StackOverflow has so much training data that a neural network should be adept at performing this analysis.

5
  • ooh, the beautiful jasper-report tag, I like the idea of Question Metrics its better then my old idea to improve the ask question interface. However I also think that these low-traffic tag depends heavily on the few people having fun answering, taking time to comment, reproduce problem and then answer (since not much repz), Commented Oct 9, 2016 at 19:57
  • 8
    Yeah no doubt SE needs to find a way to help new user to ask better questions, I actually now spend my time closing questions, flagging NAA etc. Not much time left to answer any question :D. Commented Oct 9, 2016 at 20:17
  • @PetterFriberg at least you can say that you are making the site more cleaner.
    – Braiam
    Commented Oct 9, 2016 at 20:52
  • 2
    Related: meta.stackoverflow.com/questions/326868/… (The team said a while ago they were "working" on this, but the silence so far is deafening. Oh well, at least we have Documentation to keep us busy shoveling more crap.)
    – Cody Gray Mod
    Commented Oct 10, 2016 at 11:29
  • Why not a question form with sections. Like Describe Problem, Describe how to duplicate it (with MVCE), Describe troubleshooting steps.
    – user6017774
    Commented Oct 10, 2016 at 12:41

1 Answer 1

3

The easiest way would be to detect

This does not seem easy. Automation (detection) with regards to writing is very difficult, and would require a very large amount of work in order to come close to determining question quality.

Moreover, many of the metrics you cite simply do not exist for the greatest offenders of low quality questions (1 reputation users). For that group, the issue becomes only measurable by (from your list) word count, inclusion of code and/or images, a yes or no for the use of error coupled with exception or trace, and a lexicographical parser.

These are the issues with those metrics.

  • Word count: Aside from noting that exceptionally long posts tend to be low quality, word count does not offer much more correlation. On top of that, there is a minimum character requirement for posting.
  • Code/Image: The existence of code is not a good metric for determining the quality of the code or if that code is even representative of the problem, for example wall o code or code dumps or code with no context. The image is slightly worse actually, as often new users will post an image of their code. Rarely are images used for their proper intended use of showing what is happening in a render.
  • if(error && exception || trace): I am not sure how accurate the results from this type of query would be, especially considering that some languages (JavaScript) typically do not dump a stack trace (without forcing it).
  • lexicographical analysis: This analysis is based on proper English grammar and has no clue about programming jargon, I would worry that there would be too many false positives on failed grammar as a result since it may be hard to automatically separate out prose from jargon.

Overall, I think that this type of automation would only be able to indicate success from a history of proven success, and not be able to indicate success/failure from a user without any history.

That said, it would be nice to get any sort of indication of low quality for the user, but again, if they "aren't reading anything" (as you indicate - and as many have also said on meta before), why would a "continue" button be any hurdle?

Perhaps the best way would be to simply add a bronze badge for inspecting the MCVE page, so that it may at least get more exposure.

6
  • "Why would a 'continue' button be any hurdle?" -- Depending on the machine learning accuracy, it could be used to prevent low-quality posts from being submitted altogether. That is, if the probability is below a threshold force the user to add details to the question. To stress, only contingent on whether machine learning can find any discernible patterns regarding answerabililty. Commented Oct 10, 2016 at 22:23
  • 7
    I've got to say, I really like that badge idea.
    – Taryn StaffMod
    Commented Oct 10, 2016 at 22:37
  • @bluefeet why? Wanting to get your question answered should be enough incentive to go and create a question that's actually answerable. Not sure if yet another badge is going to help.
    – CodeCaster
    Commented Oct 11, 2016 at 9:23
  • 2
    Why not @CodeCaster? We have badges for various things including encouraging behaviors like reading the tour page and completing your About Me on the profile. This could be another badge to encourage users to read the MCVE page and get informed about how best to ask a question.
    – Taryn StaffMod
    Commented Oct 11, 2016 at 11:12
  • @bluefeet: Write a query to determine how many one-question-askers acquire badges for the tour and about me pages. There are probably other ways to determine if there's a correlation between new users, badges, and desired outcomes. That is, settle the CodeCaster's question with data, not conjecture. Commented Oct 11, 2016 at 16:00
  • 2
    Yes I could do that @DaveJarvis and probably would before we just implemented a badge. Me saying that I like the idea doesn't necessarily mean it will happen.
    – Taryn StaffMod
    Commented Oct 11, 2016 at 16:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .