219

Big changes are coming with our review queues. You all already know this; we’ve been talking with you about it and have started releasing our bigger changes bit by bit. This announcement isn’t that. While we’re working on our broader overhaul of review queues, we also wanted to tackle some of the easiest-to-address issues that can improve reviewer quality of life in the interim.

With that in mind, here’s what’s changing with the Triage queue today (this change is now live):

“Unsalvageable” is renamed to “Flag.”

We’ve heard from you in the past that calling something “unsalvageable” is uncomfortable and unclear. Because that button simply opens up the flag menu, we decided it was best to be as simple and descriptive as possible.

“Requires editing” is renamed to “Needs community edit.”

Another problem we’ve heard from you is that, although “requires editing” sends to the Help & Improvement queue, when a question needs to be edited by the author, this is not the correct option. We’ve renamed this to “Needs community edit” which will hopefully make it clearer that this option is only for questions that would benefit from community cleanup.

“Needs author edit” is added as an option.

This option functions fairly similarly to the “Flag” option, except that it opens directly to the close dialog instead of the flag dialog. Our intention is that this correctly routes the questions that need further clarification from the author back to the author, instead of to the review queue merry-go-round as before.

Note: we'd initially changed "Looks OK" to "Approve," But after some great feedback and further discussion with design, we've decided to revert that change for the time being.

These changes will be highlighted with a simple banner in the Triage Queue for one week to make sure that as many folks as possible are able to see and understand the updates. (Update: this banner has been turned off!)

As always, we welcome your thoughts and feedback, and look out for more substantial changes to review queues from us in the coming months!

25
  • 171
    "“Needs author edit”" is a very welcome addition.
    – Thom A
    Commented Oct 29, 2020 at 16:06
  • 21
    Re: “Needs author edit” - "except that it opens directly to the close dialog instead of the flag dialog" What if a reviewer has less than 3,000 reputation? Does it still go to the close flag dialog?
    – TylerH
    Commented Oct 29, 2020 at 16:14
  • 5
    Do we really need both options, "Flag." and "Needs author edit"? One option should be enough and if it sends the user to the flag dialog or the close vote dialog depends on the reputation?
    – Tom
    Commented Oct 29, 2020 at 16:15
  • 8
    Yes @Tom, because the first screen of the dialog offers options that do not necessarily involve an "author edit". E.g. spam, rude, vlq, dupe.
    – yivi
    Commented Oct 29, 2020 at 16:16
  • 52
    Our philosophy here was to err on the side of surfacing more options to make it more obvious that "Needs author edit" is a possible choice, instead of giving people more room to make mistakes. Even though you can technically accomplish all you'd like from the Flag option, we want to make sure that questions that need author edit are properly routed.
    – kristinalustig StaffMod
    Commented Oct 29, 2020 at 16:18
  • 13
    The "Needs author edit" is wonderful; I've encountered a high percentage of Triage Queue questions, where with a minor clarification from the OP (IE: What operating system) would make the question easy to answer, and useful for others. Commented Oct 29, 2020 at 17:01
  • 29
    Wow! I have been asking for this for years. Makes me feel like maybe someone does care. Color me impressed.
    – Travis J
    Commented Oct 29, 2020 at 18:51
  • 1
    Are OPs notified on close votes? If they do, is it all CVs, or just the first one? I haven't asked a poorly-received question in many years, so I have no idea how that part of the system works.
    – MattDMo
    Commented Oct 29, 2020 at 19:12
  • 2
    @Catija Thanks. Wouldn't that mean, though, that the author may potentially not know if someone clicks Needs author edit and selects one of the canned reasons, such as needs debugging details? Or am I missing something?
    – MattDMo
    Commented Oct 29, 2020 at 20:27
  • 16
    Notifying users of their questions being closed has been a long-discussed issue on the network, @MattDMo :) I'm personally in favor of changing the behavior to notify users when their question is closed. If we don't, then they can't possibly improve it unless they're sitting on the page, refreshing, to see that they need to do something. There are certainly valid concerns about what would happen if we notified people about their question being closed... but it seems odd to not. Anyway, that's why I'm a proponent, if you can, to leave a comment so that the user is aware of what's needed.
    – Catija Staff
    Commented Oct 29, 2020 at 20:31
  • 2
    @Catija yes, I've seen that, but I've found it's just as easy to switch over to my text editor, find the right comment, copy/paste, and customize as necessary.
    – MattDMo
    Commented Oct 29, 2020 at 20:50
  • 2
    I know there's a ton of backstory here, and I understand why this is important, but on a surface level, am I the only one here amused by all this hype over renaming four buttons? On a more serious note, I do think this will at least improve the review queue situation, and I appreciate the work put into it. Commented Oct 30, 2020 at 1:49
  • 3
    You are ambiguously overloading "flag". Plus independent of other SO/SE uses the word means almost nothing & little in context. Continuing a SO/SE lack of clarity in terminology & documentation.
    – philipxy
    Commented Oct 30, 2020 at 4:09
  • 2
    The two buttons that reviewers (should) use the most in Triage were: Skip and Unsalveageble. One of the things I found most confusing when I started reviewing was all the options in the flagging dialog. Adding a button that, for the most part, effectively does the same thing as another button is likely to cause additional confusion. I've removed the "Needs author edit" completely from my own user interface and instead named the Flag button "Flag / Needs author edit". I think that makes it less confusing and it also makes the button wide in size and easier to "hit".
    – Scratte
    Commented Oct 30, 2020 at 8:48
  • 3
    As somebody who has been getting a bit more involved in moderating recently, one thing that would be quite helpful is having an official chart or other documentation of how the different queues and dialogues interrelate to each other. It can be a bit confusing how the system is meant to work, although this post actually clarifies things somewhat.
    – Stephen
    Commented Nov 1, 2020 at 13:56

12 Answers 12

60

Long time coming, but it's a massive improvement

Not a huge fan of the Triage queue in general, but these small tweaks should help it work at least a bit better.

I hope this communicates effectively that Looks OK and Needs community editing are very similar options: they both apply to the same kind of questions: those that we want.

To complain about something, and conform a bit more with Meta tradition, my fear is that Needs community edit looks closer to Needs author edit than to Approve, while I think is conceptually closer to the latter.

At least the button grouping puts the similar options closer together, ordered from "better" to "worse", which is good and appreciated: Approve, Needs community edit, Flag, Needs author edit, and Skip.

One thing I would change, is to put Skip all the way to the left, being that it should basically be considered the "default" option for the queue, separate from all the other options.

10
  • 35
    ... Oh, we're not through yet. :P
    – Catija Staff
    Commented Oct 29, 2020 at 16:02
  • Do you mean "but it's not a massive improvement" or "it's a massive improvement"? Based on your second paragraph I assume the former.
    – Wai Ha Lee
    Commented Oct 29, 2020 at 16:06
  • 1
    @Wai, I believe it's a massive improvement. Truth be told, "improvement" was not the hardest target to hit. The soon to be "old" labels are a bit of a disaster.
    – yivi
    Commented Oct 29, 2020 at 16:09
  • 6
    Maybe grouping the "good" buttons together and grouping the "bad" buttons together with a thin green/yellow/red outline (respectively, or whatever) would be a further improvement. E.g. "hey I'm giving this a kind of passing grade by clicking this button." or "hey, I'm clicking a red/bad button here which indicates this post is doomed".
    – TylerH
    Commented Oct 29, 2020 at 16:12
  • 5
    Hold on, you're telling me skipping is the worst option? /s
    – code11
    Commented Oct 29, 2020 at 20:55
  • Skipping isn;t the worst option. It should be encouraged to do, but not too much. who wants a cheeky reviewer spamming skip?
    – 10 Rep
    Commented Nov 3, 2020 at 3:46
  • 1
    @10Rep "but not too much". What's too much? Why? Do you prefer a suspension? If someone want's to Skip 1800 posts, what's wrong with that?
    – Scratte
    Commented Nov 3, 2020 at 8:43
  • @Scratte Whats the point of that? They might as well not review.
    – 10 Rep
    Commented Nov 3, 2020 at 15:08
  • 4
    @10Rep It's better to not review or to skip everything than to get suspended because one feels pressured to skip "not too much".
    – Scratte
    Commented Nov 3, 2020 at 15:10
  • 1
    @10Rep It's waaay better to skip questions you feel unqualified to review than to give mediocre reviews just because you don't want to skip it. The former creates more work for others, while the latter merely leaves existing work undone. If a person feels qualified for only 10% of the questions, so be it. Let them be.
    – jpaugh
    Commented Nov 5, 2020 at 23:01
58

Thinking about it longer, I'm not sure changing "Looks OK" to "Approve" is a good change. In my opinion, it is less clear what action you are taking by clicking the button that says "Approve" here. It is not a pending post, it's an already-posted post... so you're not approving anything. I understand the sentiment behind the change, but I think "Looks OK" gets the message across way more clearly than "Approve". If you're dead set on changing it, maybe "No changes needed" would be better wording.

Take this with a grain of salt, since I avoid the Triage queue in general

2
  • 18
    I think this is a really fair point. That combined with the added complexity of the LQP queue using the same option on the backend means that, for now, we're going to change it back to "Looks OK." Trying to get that change out today, I'll update my post when I do. Thanks for the feedback!
    – kristinalustig StaffMod
    Commented Oct 30, 2020 at 15:16
  • 18
    I've made this change and am now updating documentation accordingly.
    – kristinalustig StaffMod
    Commented Oct 30, 2020 at 17:41
32

This is a big improvement, which should hopefully help new reviewers better understand the available options. Thanks for the work on it.

However, one thing does stand out for me: on moderately-sized browser windows (e.g. I keep mine around 1200 pixels -- I know, I'm a heathen), the actions area is wider than the content area, pushing the "Skip" button off the edge of the visible page:

Triage header wider than triage question body, such that "Skip" button requires horizontal scrolling

While the "Skip" button is a mere horizontal scroll away, in my opinion that should be the button with the absolute least friction to use. It seems that there's enough open space there to scoot the buttons left a bit so that they align to the right side of the visible page rather than triggering overflow.

1
23

Hooray! This is an excellent change.

I think the differentiation between community edit and author edit will be the most effective change listed here. Of course, I think they're all good changes (except maybe the "Looks Okay" problem TylerH pointed out, but that's not a huge issue IMHO ). But that one (back to the author and community edit buttons again) will be particularly helpful, especially for H&I reviewers (as you have mentioned).


Edit: And you guys have actually took the feedback and reverted the change back to the original. So now my sentence seconding TylerH's point is now moot.

I'd like to take this opportunity to say thank you guys for taking the time to listen to us, accepting our constructive feedback, and adjusting if necessary. You've made major progress from a year ago. Keep up the good work!

12

These new changes sound like great improvements. The triage queue needs a lot of work, but as Catija mentioned, you're not through yet, so I look forward to the upcoming improvements as well.

I have a question here:

“Needs author edit” is added as an option.

This option functions fairly similarly to the “Flag” option, except that it opens directly to the close dialog instead of the flag dialog.

The close votes privilege is given at 3,000 rep. Many of the triage reviewers have much less reputation than that. Is there going to be some sort of adaptation here for those reviewers?

3
  • 8
    I imagine they mean it opens the "Flag -> Needs Improvement" dialog directly, instead of simply the first screen of the flag dialog.
    – yivi
    Commented Oct 29, 2020 at 16:12
  • 3
    @Yivi That makes sense, I'll leave this here in case someone else is confused my that wording too.
    – Davy M
    Commented Oct 29, 2020 at 16:13
  • 3
    To get a more solid answer than "I imagine", as a <3k user I can confirm yivi is correct
    – rh16
    Commented Nov 4, 2020 at 23:41
10

Thank you for the great improvements!

It would be also very helpful if the Triage queue showed more questions related to the "watched tags" of the reviewer.

I understand that we want to triage all questions regardless of whether they have tags that are watched by the users. But the queue should not be entirely random. Instead, it should be enriched up to a certain percent (between 0 and 100%, TBD) by the questions with the watched tags. Spiking in questions into the Triage queue is not a novel idea. You occasionally get "test" questions designed to check "if you are paying attention" currently anyway. I simply suggest another type of spike-in questions.

SEE ALSO:

This has been suggested many times before, see below. But the previous questions ask mostly about using plain filters (which would limit the Triage queue to 100% watched tags). I suggest a level of spike-in less than 100%, so that rarely watched tags or untagged questions get enough reviews as well.

Filter the Triage review queue by subject/expertise
Why is there no filter in triage review?
I only want to see items related to my favorite tags in the review queues
Review triage questions accordingly to my favorite tags

1
  • 2
    I really like this. While I think that I can triage every kind of question, I'm sure that I can triage questions in my tags faster and more reliable, which would be a good thing. Commented Nov 1, 2020 at 9:42
10

The flag/close dialogs are hidden behind the instructions section (which is now expanded by default):

Dialog hidden

See it in action:

Dialog hidden - GIF

1
9

It's definitely working. The H&I queue is dead. The queue activity was already on a decline (undoubtedly moderator and curator action that led to many suspensions of poor Triage reviewers was having an impact), but the number of Help and Improvement reviews is now averaging single digits and even down to zero on some days, as can be seen in SEDE.

enter image description here

This does, however, render the Help and Improvement queue quite useless. If I were to persevere at checking the H&I queue multiple times a day in the hope of finding something to review, I estimate it will take more than a year to complete the 172 additional reviews I would need to get the Steward badge; so there is a good chance that reviewers will abandon the H&I queue altogether. It was always a fairly frustrating queue when it was busy (for precisely the reasons that you have modified the Triage options), but it has gone from being mostly chaff with some (malformed) wheat to having neither chaff nor wheat.

Essentially, having the community edit button seems like a good idea as it makes a clear distinction between an author edit and a community edit, but, to be honest, the community edit button might as well just record this flag in the question's history and then close the Triage review without any further action. There really is no point to the H&I queue any more - anyone deciding to pursue a gold badge from scratch (at my current rate of progress) would take about 6 years to get it.

3
  • 5
    Mission accomplished.
    – Samuel Liew Mod
    Commented Nov 28, 2020 at 11:43
  • 1
    @SamuelLiew Absolutely. The days of going into H&I and using 30 close votes in a single pass through the queue are history. Your actions brought the numbers down hard, and the (finally) change to Triage has finished the job.
    – David Buck
    Commented Nov 28, 2020 at 11:46
  • 2
    Next: meta.stackexchange.com/q/357180/165483
    – Samuel Liew Mod
    Commented Nov 28, 2020 at 11:59
7

Are the "Flag" and "Needs author edit" options counted as two separate review types for the purposes of dismissing the review task with an outcome, or the same?

Under the previous system, three reviews of the same type would dismiss the Triage review task as "completed" with that outcome. For example, three "Needs Editing" reviews or three "Unsalvageable" reviews would dismiss the review task with that respective outcome (sending the post to Help & Improvement if the former, per the official flowchart that documents the old system).

Since then, the old "Unsalvageable" option has been split into two, "Needs author edit" and "Flag".

Are those two new review types counted as the same for the purposes of completing the review task, or are they counted as two different outcomes? In other words, if a review receives two "Needs author edit" reviews and one "Flag" review, will that dismiss the task, or will it be that one more "Needs author edit" review or two more "Flag" reviews will be required to complete it?

If the former, this will be consistent with the behavior of the previous system, as all three of those reviews would have been counted as "Unsalvageable" in the old model and the review would have been completed, but if the latter, those both would have two separate uses despite the same ending outcome.

6

I like the changes, eventho things can still be improved as pointed out in other answers. For me there is one thing that hasn't been mentioned that I would like to add.

Right now any decision on the queues is definitive, there is no space for the slightliest mistake, for example I have been suspended for a period of 1 Month from the triage Queue for clicking on the wrong button. It was a mistake that I was aware almost immediately after triaging, but there was no way of correcting this, and I think it is quite harsh measure after changes on the triaging buttons.

I believe that adding a time period were your actions can be ament is as important for moderators as new actions, and more if SE wants to keep moderators doing queue jobs. Changes on the interface can be confusing and will create quite some conflicts with new people in the queues (and Triage is one of the first queue contributors get access to)

5
  • 1
    What do you mean no way of correcting it? If the post is flaggable you can flag it outside the queue. If it's closeable you can flag/vote to close it outside the queue. Commented Nov 12, 2020 at 9:55
  • After clicking "Needs community edit" instead of "Needs author edit". I was not able to correct it outside the queue. Maybe this is my mistake and just wasn't able to amend it eventho it is possible
    – SirPeople
    Commented Nov 12, 2020 at 10:03
  • Why not? Just find the post via your review queue history and do what you need to do with it? Commented Nov 12, 2020 at 10:08
  • 2
    @RobertLongson Correcting the action is different than correcting the review- since reviews are immediate, there's no way to amend your review itself if you misclick. Like you mentioned, there's recourse you can take, in the form of flagging a post anyway or bringing it up in chat, but you can't "undo" a bad review, even the split second after clicking.
    – zcoop98
    Commented Nov 12, 2020 at 16:11
  • @zcoop98 The point is you can show the moderators that you recognised the mistake and tried to address it as best you can within the limits the system imposes. Commented Nov 12, 2020 at 16:56
4

Thank you for the effort. However, as these issues were raised by the community literally on day one of the triage queue, forgive me for not being especially enthusiastic about a fix that's more than half a decade late - especially as two of the three changes are simply changing the text on a button.

To me this is yet another sign that something is or at least was incredibly wrong in the innerworkings of SE. That the issue was resolved now shows that SE acknowledges that it is important. But then, what happened in the almost 6 years between the time the issues were reported and the fix? Why was this issue sidelined for so long and is being worked on now? How are you going to make sure issues are fixed in a more timely manner in the future? I think a honest answer to these questions would be a good first step to re-establishing trust with the part of the community interested in improving the platform.

3
  • 6
    I can't speak to the last six years - I think it's clear that there was a period of time where the company's focus was on building revenue and there wasn't an ability to build or maintain the tooling for the sites. That's changed. We have a team - dedicated to this now and that's where the changes like this are coming from. We're also committing to addressing issues like these moving forward through our status-review process - meta.stackexchange.com/questions/348642/…
    – Catija Staff
    Commented Nov 2, 2020 at 17:38
  • @Catija "we have a team dedicated to this now" - well three years ago you had Team DAG which on the surface was dedicated to the same things ("It's also the DAG team's job to make sure that Q&A gets what it needs, approximately when it needs it [...] They also ensure that community requests get scheduled, discussed and many times ultimately completed or at least deferred"). Didn't work out so well and I think meta does not remember them fondly. I wish you the best of luck for your current attempt but surely you'll understand I'm not holding my breath.
    – l4mpi
    Commented Nov 10, 2020 at 10:20
  • 3
    I don't think anyone is asking for that... but I'm curious what you think you're impacting with answers like this one? We can only control the future. DAG wasn't focused on this and it was an extremely small team. Even the current team is small and has to focus on specific projects but there's a big expansion for this team over the next year to better meet the needs... but short of hiring 50 people, most of the stuff that needs attention will still take a long time to get to. There's a lot of stuff that's been ignored over the years. We're working on things, gradually.
    – Catija Staff
    Commented Nov 10, 2020 at 13:02
2

I was about to do my part to the never ending queue “help and improvement” today. But I got extremely surprised. The queue was empty! Had I activated any filters? Nope! In the past, stepping into that queue was basically alternating “skip” and “flag for vlq” a hundred times before you found a post that you really could save by some editing.

Obviously this change did exactly what it was supposed to do. Very good job!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .