-19

Reading the post related to The Ask Question Wizard and the section about How to Ask shown in the Ask a question page, I'm thinking on the possibility of automating the detection of non-useful or bad-formed questions before being published (too short questions, requesting answers for a problem -sometimes homeworks- without providing their attempts, etc.).

I'm sure there are some basic patterns of questions that could be detected by bots, saving time to the community on maintenance tasks, and keeping the site cleaner.

Have the collaborators considered developments in that way?

13
  • meta.stackoverflow.com/questions/375365/…
    – yivi
    Commented Dec 20, 2018 at 11:57
  • 1
    Somewhat related: Can a machine be taught to flag comments automatically?
    – honk
    Commented Dec 20, 2018 at 12:11
  • 5
    There are already bots doing things like that: SmokeDetector, FireAlarm. Have a look at Sobotics
    – BDL
    Commented Dec 20, 2018 at 12:34
  • The question was not about the same than the suggested duplicate, I was thinking about detecting not useful questions before being published, but I suposse it could be similar
    – bra_racing
    Commented Dec 20, 2018 at 14:05
  • @bra_racing I don't mind reopening, if you think it's not a duplicate of your question. Want me to cast a vote?
    – user247702
    Commented Dec 20, 2018 at 14:18
  • Ok. Now I'm edditing a bit the question to avoid the misunderstanding
    – bra_racing
    Commented Dec 20, 2018 at 14:24
  • There are lots of mechanisms to either prevent bad questions from ever being posted, or to post a question but take some sort of action on it due to an automated tool having evidence it may have problems. There are already several mechanisms in place doing these things. If you have some more specific suggestion of how they can be improved, or a new metric/criteria not currently being considered, then propose it.
    – Servy
    Commented Dec 20, 2018 at 14:30
  • The only mechanism I know is be flagged by users, manually, after being published. When I talk about automating method, could imply machine learning and at the begining a training phase would be necessary in order to avoid problems.
    – bra_racing
    Commented Dec 20, 2018 at 14:36
  • 2
    I'd suggest doing more research into the existing mechanisms to deal with low quality content then, because there are lots beyond just users flagging content.
    – Servy
    Commented Dec 20, 2018 at 14:53
  • 2
    @bra_racing: Unless you achieve almost 100% accuracy, a full automatic tool is not helpful. There are bots who flag for specific reasons. FireAlarm and SmokeDetector, for example, post their findings to several chatrooms where users review them.
    – BDL
    Commented Dec 20, 2018 at 14:53
  • Ok @BDL, the aim of the question was to discuss about a full automatic solution, but the bots you talk about are a good aproach. Thank you!
    – bra_racing
    Commented Dec 20, 2018 at 14:57
  • 2
    @bra_racing as BDL was saying, a full automatic system needs to be 100% reliable. Not 99.99%. At our daily volume, 99.99% is still not reliable enough, unfortunately. The best you can hope for is a system that automatically forces more reviews (like the VLQ queue) :/.
    – Patrice
    Commented Dec 20, 2018 at 15:05
  • 1
    @Patrice There are already several mechanisms that prevent questions from being posted entirely. They're not 100% reliable, not even 99.99% reliable, but they're still reliable enough to be worth including. There have been mechanisms to prevent questions from being posted for years and years. Now yes, some of the heuristics for finding low quality content are less reliable, and so are validated by humans through the Low Quality Posts queue. Some are more reliable, and so don't require human intervention. You simply never see those questions, because they don't get posted.
    – Servy
    Commented Dec 20, 2018 at 15:18

1 Answer 1

11

We'll be talking about this more in 2019, but we're able to pretty reliably detect certain types of questions that have a history of going over quite poorly. Simple heuristics detect a lack of code in otherwise terse questions currently, along with some other basic checks, but we can also (now) pretty accurately scope what looks like a homework assignment and maybe provide some pretty strong guidance up front, so people's expectations are set accordingly.

As we dove into unwelcoming comments, one of the phrases that surfaced the most was (unsurprisingly): "homework". So setting the goal of detecting "bad quality" is the wrong way to go, detecting with a high degree of certainty certain kinds of questions that we can model can help us better set people's expectations (and help certain questions go straight to the close queue instead of clogging up the helper queue).

I can't say too much more about this because it's all highly experimental and it was only yesterday that I learned what a confusion matrix was and how it worked, so.. but we're taking some stretches in this direction, at least in 'stack labs' as I've come to call it.

Look for posts by Jason Punyon, Julia Silge, and Kevin Montrose next year, as they've got a ton of this to talk about. I'm only mentioning that we're looking in this direction because the wizard, upon breaking questions down into basic components, does strongly lend to the idea of systems that are way more superior than the current (inadequate) static analysis that we do.

3
  • Thank you Tim! I'll pay attention to the posts.
    – bra_racing
    Commented Dec 20, 2018 at 15:47
  • Sweet! Lets all sit down at the confusion table with rabbits, cats and dogs and talk about the positive predictive value ;) Why does that remind me of Alice?
    – iLuvLogix
    Commented Dec 20, 2018 at 16:51
  • That's so exciting! Hoping to know how ML will be applied in the (very) near future.
    – bra_racing
    Commented Dec 20, 2018 at 17:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .