I think the new queue is great. Fantastic, even! I had some thoughts about the implementation, which I was going to post in the feedback thread, but this seems like a more appropriate place. (The incoming wall of text is not because this is a horrible, world-ending issue that kills the crab, but because I find the subject interesting.)
So here are the options as they currently stand:
Looks OK Should Be Improved Unsalvageable
I see this as roughly equivalent to a three-point scale assessing the overall quality of the question.
When I was in college I had a fascinating conversation with a psychology lecturer who taught survey design; one of the things she said was that people have an inherent bias away from either end of the scale. So when you want to assess whether people feel positively or negatively about something, a three-point scale tends to produce a strong bias toward the neutral response.
I don't think that's necessarily a problem in terms of where the question ultimately ends up – these questions have already "failed" the Q-score screening and Shog9 has been fairly clear (1, 2) that the expectation is for "Should Be Improved" to be a catch-all for borderline scenarios. But one of my strong impressions using this queue, which I think is echoed by some of the other feedback that's been given, is that it's not as fast as it could be. Sometimes you get a question that just really doesn't seem to fit on the three-point scale, but you do feel able to quickly and accurately assess it and you don't want to skip it. And even if you do skip, it takes some amount of extra time to decide to skip.
One of the other points from this conversation with the lecturer was that it can be easier for people to respond to questions asked on a five-point scale versus a three-point scale.1 We intuitively understand that lines have endpoints, a midpoint and regions in between, and we're comfortable mapping this concept to a five-point scale.2 Here's an illustration, using a Likert scale:
When choosing to use a three-point scale, the naive survey designer may expect that we'll simplify our model by eliminating either the in-between regions or the endpoints. Either way, you get essentially the same result:
Instead, what we tend to do is cling to that comfortable, finite-continuous model and map it to the three-point scale. But it's not as easy to do this when we have fewer options to map to:
Now we have to spend more time thinking about the scale instead of the question. According to this instructor, generally speaking, we don't want to be fence-sitters, but even more than that, we don't want to be extremists. We act as though we're being personally judged, even when the responses are anonymous, and we tend to be more concerned about nuance when taking a survey than we expect people will be when we're designing surveys because we have different motives. The survey designer ultimately wants to answer a yes or no question, while the respondent wants to express their personal opinion, and maybe also give "good" answers. In the review queues, there's definitely an incentive to figure out the "right" answer – go against the majority too often and you get the whip. Failed audits! Drama on meta! Accusations of robo-reviewing!
So the survey designer spends a bunch of time thinking about their survey, and comes up with a three-point scale that they understand, and expect other people to understand. Because they're soliciting opinions, they might want people to be very opinionated; more likely they've just developed their own "feel" for the scale and haven't given it much thought. All that response style bias will smooth itself out if the sample size is large enough, right? No big deal.
What she suggested to me was that a clever thing to do is to design the survey with more options, to maximize how comfortable people are expressing an opinion one way or the other when they're not truly ambivalent; and then map the responses after the fact back down to your desired three-point scale. In other words, drop the strongly/somewhat qualification and look only at whether people agreed, disagreed, or were neutral. Everyone has seen this happening, whether they know it or not, in PR copy that says things like "X% of customers were satisfied with the service they received." Odds are, those customers were asked to rate service on a scale with more than three points.
This was fascinating to me, because it's so counter-intuitive to think that people would take less time and be more comfortable choosing between more options.3 You do lose some apparent measure of the intensity of the response by doing this but, practically speaking, you shouldn't expect to get a consistent, stable measure of response intensity from this type of scale in the first place. Nor would the three-point scale you started with have measured intensity at all.
The analogy only goes so far; review queues aren't opinion surveys and the actions available to us aren't responses on a Likert scale.4 Also, I should point out that there's nothing special about a five-point scale – it's just the example that was used in the original conversation. We could be talking about any scale with more than three points (although even vs. odd has its own implications).
So here's the point of all this blah, blah, blah. In testing the Triage queue, I've had the same feeling that you and davidism have expressed, that the options we're given may be too limited. I've felt like I'm being asked to rate questions on a three-point scale from "Awesome" to "Average" to "Horrible" but my first impression often doesn't fall into any of those buckets. As a consequence, while I can evaluate items very quickly in the triage queue, I find myself sometimes spending an unreasonable amount of time translating my evaluation into one of the three available options. There's some overhead involved in that mapping step, that I find makes reviewing in this queue a little less comfortable than it has the potential to be.
Like you, I don't think "just skip it" is a great solution here. It takes about as long sometimes to fail to decide between the given options as it would take to choose one, when the question is borderline.
There's a good chance that this is just growing pains; as we get more comfortable with the triage queue, we'll all get better at making quick and painless decisions. But it's worth thinking about whether expanding the options could make the queue function better from the reviewer's perspective. One of the uncertainties about using "Should Be Improved" as a broad catch-all is that there's a difference in how we perceive and approach questions that need the author's intervention to fly, versus those that just need some love from a capable editor. And there's a difference in how we handle them, ultimately, based on whether the author puts in the work or not. I read this comment by Shog9 as saying that, since questions that are quickly closed rarely escape deletion, there's not a significant enough difference in expected outcome to justify distinguishing between "unsalvageable" and "author must edit" – but this isn't necessarily about changing the outcome for the author of the question. It's about the usability of the queue for the reviewer.
So, here's an example of the sort of alternate interface that some people have requested:
Looks OK Should Be Improved Needs Revision By Author Unsalvageable
Behind the scenes, this doesn't need to change where the post actually goes. "Looks OK" and "Unsalvageable" will probably still lead to the homepage and a flag/close review queue, respectively. "Needs Revision By Author" and "Should Be Improved" could both go into the helper queue, to begin with; but as we get data about how people are actually using the triage queue, it might make sense to divert "Needs Revision By Author" to flag/close review instead. Or send "Should Be Improved" to the homepage to sink or swim.
Would these four options be better than the three current options? Would any four options be better? I don't know. This is a question that requires data – and a clear metric for "better" – to answer. It may or may not be worth investigating; that's up to the good folks who Make Everything Work around here to decide.
Anyway, I hope this was at least moderately interesting food for thought. I went back and forth a few times about whether to even post it after Shog9's answer, because he's not wrong – the three options are a great idea. Personally, I think the four options are a good competing idea, and I'd be interested to see whether people use them differently.
I'll spare you the annotated bibliography for this post; suffice it to say that Google will bury you in literature about N-point scales, should you so desire.
1 Anyone who has studied survey design will start squirming in their seat when they read this; at the very least, I'm opening a huge can of worms by even bringing up the relative merits of different scales. There's tons of research on this subject and no clear consensus; some studies have concluded a seven-point scale is best, or a two-point scale, or even a 100-point or analog scale. My layperson's take on the subject is that there are different implications of different scales; keep reading, and I'll get to the point.
2 In hindsight, there may have been an unspoken cultural context to this conversation. Different cultures can have fascinatingly different perspectives and ways of thinking; in any case, it was a relatively casual conversation, so take it for what it's worth. When I sent her an email to confirm my recollection of the conversation, I also asked about this; she responded:
I have no idea if this is 'American' / Western or 'Human'. What an
interesting question. We think about Response Sets and Heuristics as
being 'humans think this way' -- but so much of the psychological
research has been done on American undergraduates, it is possible that
these are cultural not innate.
3 On the other hand, maybe it's not that counter-intuitive; you wouldn't ask, "What's your favorite color?" and provide only cyan, magenta, yellow and black as acceptable responses.
4 "Should Be Improved" is the middle option in terms of how the interface is laid out, and also in terms of judging the question on a scale of overall quality. But in terms of taking action on the review item, "Looks OK" could be considered the middle because it doesn't lead to further review, and is in that respect "neutral." And what if there's no middle option at all? Bye-bye, Likert scale analogy. It was fun while it lasted.