tl;dr: there's a new review queue. It'll be getting somewhere around 1-2 questions per minute. The only thing they have in common is that the system is unsure of what to do with them. Some are great, some are awful, some are in-between. We need you to help the system decide which category these questions belong in. If this works, we'll be building a whole lot more stuff on top of it: better questions on the home page, more help for promising users, less of a close-queue backlog... So let's test it!
Background
Stack Overflow gets a huge number of questions per day. The best get answered. The worst get closed and deleted. However, there are a huge number of questions that aren't bad enough to be removed but... still aren't great. Many of these still get answered, and could be turned into exemplary questions... if anyone cared enough to fix them. But whether or not they get answered or edited is largely a matter of luck. In fact, luck has become entirely too much of a factor for even well-asked questions - the volume is too great to easily sort through, and many users depend on either the homepage's automatic filtering or complicated tag filters.
For a long time, we've been trying to find a more automated way to categorize questions when they're asked rather than requiring each and every question to be moderated. It turns out this is really hard. Ben Collins took another crack at this recently, and the results are pretty good... But not perfect. The algorithm still needs help. And that's where you come in:
The goal for Triage
The term “triage” originated in battlefield medicine, describing a process for quickly sorting the wounded into three groups:
- Those who are likely to live, regardless of what care they receive;
- Those who are likely to die, regardless of what care they receive;
- Those for whom immediate care might make a positive difference in outcome.
The goal is to make the most of limited resources. And that's what we'll be doing with Question Triage, sorting questions into three groups:
- Those that will likely be well-received by the community and obtain answers with no further structured review.
- Those that will likely be poorly-received by the community, closed or deleted regardless of how much assistance they’re given.
- Those for which additional guidance or revision might result in a positive reception and useful answers.
Behind the scenes, a "quality score" is calculated for each question based on an automated analysis of the content. Those that score well are sent immediately to the homepage; those that score poorly will now be sent to Triage. From there, they'll go to one of three places based on human input:
- The homepage, where they can be answered
- The close or moderator flag queue where they can be reviewed and eventually deleted
- A new "Help and Improvement" queue where they can be edited
The intended lifecycle for a question will look something like this:
However, first we need to make sure that works! In particular, before we start filtering questions from the home page or building a queue for helpful editors, we need to make sure the two pieces we've built already actually do what they're intended to.
So right now, there's no visibility restrictions or helper queue. After a few days, I'll come back with the results:
- If it looks like things are working, we'll start dropping questions that don't "Look OK" from the home page.
- If it's not working, we'll fix them. Change the guidance, the number of reviews required to complete a task, etc.
Once we're all happy with both the behavior of Triage and the appearance of the home page, we'll roll out a Helper/Editor workflow.
Go, review, & let us know how it works!
M*A*S*H
metaphors. Because reasons.