180

Problem statement

A large segment of the triage "Requires Editing" votes are simply wrong. The people voting don't understand the essential difference between requires editing by OP and requires editing ... by some random other user to fix minor glitches. As a result: a large number of entries on "Help and Improvement" simply don't belong there.

Which is bad for everybody:

  • instead of giving the OP immediate feedback (by closing/putting on hold), the questions sit around for hours, days; before somebody shows up and makes a pointless edit, that makes the question "nicer to read", but still leaves it "unanswerable".
  • people taking "help and improvement" seriously ... spend a lot of time not editing, but downvoting questions, and doing the reasonable close vote.

A simple solution

Reorder and relabel the 4 buttons for the triage review:

  1. Requires Editing by Questioner (was "Unsalvageable")
  2. Requires formatting improvements (was "Requires Editing")
  3. Looks OK
  4. Skip

The new "labels" are just a proposal, better wording is possible. The new order of buttons does (somehow) (imho) reflect the likelihood that the buttons should be used, "Requires editing by questioner" to be the almost default to click.

Why will that work?

While working on my last steward badge on H&I, I did something bad: I occasionally directly contacted triage reviewers who voted "Requires Editing" for questions that were really obvious "too broad" or "no mcve" cases. After doing that like 30, 50 times, I noticed the following:

  • most such triage reviewers are relatively unexperienced (new members, reputation between 500 and 2000, rarely more)
  • most of them: really didn't know. "I voted edit because, obviously, the OP has to edit the question". "But that is not what edit should be used for!" "Ups. I didn't know that." ... over and over and over again

It doesn't matter that the help text is right there. It doesn't matter that there is a link to more help text explaining that thing. They see the buttons, and click the wrong one.

But please note: 99% of the folks I spoke to, they are eager to improve and do better. The majority of people doing the reviews wants to contribute in meaningful ways. Thus we don't need to look into complicated changes, just make it easier for the average reviewer to do the right thing!

Beyond that

Of course:

  • In a reasonable world, the UI team would carefully draft different A/B experiments, and measure how different UIs affect the outcome. But hey, nobody is measuring the effectiveness of these queues in the first place. It even seems to be impossible to remove the pointless VLQ link on the H&I page. So hoping that a real, costly UI change happens here isn't realistic (at least to what I observe: there is tremendous activity around code of conduct, and "being welcoming", but the quality stuff doesn't matter, because that doesn't result in money coming in)
  • As said, for me one problem with wrong triage vote is lack of experience. Honestly, I think that only people who have done 50 or more edits (ideally on H&I!) should be able to vote there. And looking a bit deeper, I would even claim that maybe we talk about a 10K reputation limitation on top of that.

Sincerely, GhostCat 8th Steward of H&I

29
  • 26
    for 1. we could consider Requires Action by Questioner as that would be applicable for every close reason that is behind that option.
    – rene
    Commented Nov 8, 2018 at 9:38
  • 24
    I still wake up randomly in the night thinking of the times I pressed "Requires Editing" back when I didn't know what it meant...
    – Jesse
    Commented Nov 8, 2018 at 10:25
  • 8
    @JessedeBruijne The "system" caught me, too ... and then I learned my lessons, too. But beyond that, honestly: the really real answer is to throw out the whole system. The major purpose of the system is for people to acquire these badges and give them the feeling of doing something useful. Just look at stackoverflow.com/review/helper/history ... scroll back in history, and randomly open questions. The majority of them:still in a bad shape, no answers, few comments, no accepts Just a lot of people who devoted their time to triage them and edit them. For no outcome. Waste ;-(
    – GhostCat
    Commented Nov 8, 2018 at 10:30
  • 44
    Triage is broken by design. SE have repeatedly failed to adequately address this problem. Remember, closing the questions of new users is unwelcoming and makes Jaydles sad. Those unwelcoming established members must be trained that closing the questions of new users is evilbadwrong.
    – Raedwald
    Commented Nov 8, 2018 at 10:40
  • 12
    One problem with "Requires (Action|Editing) by questioner" is that it doesn't cover the cases where the question really is actually unsalvageable (e.g. it's completely off-topic). Why don't we label that button what it really is: "Close or Flag"? Those are the actions the user is performing. Doing so has the advantage of using the same keywords as are used throughout the site for the actions that the user is performing (flagging or voting to close), rather than introduce some new terminology for those actions. [The current "Unsalvageable" has the same issue wrt. introducing a new term.]
    – Makyen Mod
    Commented Nov 8, 2018 at 16:39
  • 20
    SE in general seems to have this awful culture of misleading labels for moderation buttons. I would love to see steps made to clarify this on any front.
    – user2201041
    Commented Nov 8, 2018 at 17:54
  • 5
    Maybe 'requires editing by author' should push it back to the question asker with wording such as 'please edit your question to address reviewer comments'... such as, add debug output, missing code, etc.
    – Peter L.
    Commented Nov 8, 2018 at 18:54
  • 2
    I often see users upload a question that is totally answerable but they are missing one peice (e.g Data, Error printout, etc.). In this case I would be likely to press requires editing, but based on the current help text and what you all are saying here, this should only be used for edits YOU can do. At this point should we really just remove the question from the site? It just seems like we are missing a middle ground option.
    – Chabo
    Commented Nov 8, 2018 at 19:09
  • 22
    We could also strongly consider mandatory training to access the queues, I am new to reviewing and I have passed all of the "tests" but have definitely been falling into some of the mistakes mentioned here. We could have ~25 question test with the first couple being explanations of when to use each button. Unless there already is training and I am forgetting, this seems necessary.
    – Chabo
    Commented Nov 8, 2018 at 19:15
  • 4
    Strongly related: Please rename the “Should be improved” button
    – Travis J
    Commented Nov 8, 2018 at 19:28
  • 3
    @Chabo I thought about such "education" too. The problem with that, as with most other ideas: cost. As said: that stupid "low quality" link on the H&I queue sits there since years. People have complained multiple times. The link is still there. So, removing a stupid link is already asked too much. Therefore my conclusion is that developing a UI for an education program, and having people do that ... is a great idea, but would be considered too expensive, therefore not going to happen.
    – GhostCat
    Commented Nov 8, 2018 at 19:39
  • 2
    @Makyen I agree. But on the other hand, the essence of my proposal here is: we need to emphasize that difference between "OP action required", and "somebody else can fix it". Maybe ... more buttons would actually help here.
    – GhostCat
    Commented Nov 8, 2018 at 19:40
  • 9
    "How to fix triage ?" With a key. The del key. Commented Nov 9, 2018 at 6:48
  • 5
    @Lundin When everybody agrees that triage and HI are dysfunctional. then they shouldn't be ignored but terminated. People spend their time working these queues. If we all agree that this is, on average, a waste of time, then lets stop that wasting!
    – GhostCat
    Commented Nov 9, 2018 at 8:10
  • 5

4 Answers 4

39

In a reasonable world, a reasonable suggestion like yours would be actioned promptly.

Stack Overflow the company is no longer a reasonable world; I'm not certain that they even have devs employed there anymore, except Marc Gravell. There's no other reasonable explanation for the fact that numerous useful changes to the Help and Improvement queue have been suggested since it was launched three-and-a-half years ago, yet almost none of them (see the answers in that question) have been implemented - even incredibly basic things like text changes, for crying out loud.

I poked about the H&I queue almost exactly a year ago to draw attention to the fact that it either needs TLC or a bullet to the back of the head, and nothing was done; I'll let you guess what the fate of your suggestions, and indeed the fate of any and all questions related to bettering H&I going forward, will be.

tl;dr your intentions are good, but you're wasting your time.

3
  • 5
    Maybe if we go on twitter and cause a ruckus...
    – Chabo
    Commented Nov 9, 2018 at 19:17
  • 3
    Related: Shog's brief history of Triage. (Note the years mentioned.) Commented Nov 9, 2018 at 22:15
  • 6
    The other day I thought if anyone ever went on "strike" here. Would things change if a large group of experienced community members would cease all activities for a day, or a week?!
    – GhostCat
    Commented Nov 10, 2018 at 6:14
35

You're overlooking the elephant in the room:

close votes review queue at 8.7k questions vs. 142 for reopen votes and less for everything else

The close vote queue: where "unsalvageable" questions go to die, (unless they're spam which gets deleted directly from flagging). In the past 30 days, over 63.7 thousand questions ended up there... And just over 8.7 thousand were completed. The rest aged away. There's no point in throwing more stuff in that queue.

The purpose of Triage was to sort out the worst of the worst questions, to at least give them a shot at getting closed. Here's the outcome for all questions entering Triage between 30 and 60 days ago, broken down by how they were triaged:

Outcome            Tasks MinsToReview PctTotal PctUpvoted PctDownvoted PctPosScore PctNegScore PctClosedOrDeleted PctAnswered PctEdited PctError 
------------------ ----- ------------ -------- ---------- ------------ ----------- ----------- ------------------ ----------- --------- -------- 
Looks Good         10630 106          47.85 %  25.63 %    0.33 %       30.23 %     15.11 %     14.70 %            36.96 %     60.06 %   20.87 %  
Should Be Improved 7742  102          34.85 %  0.15 %     0.30 %       8.12 %      45.66 %     36.17 %            21.98 %     88.58 %   35.69 %  
Unsalvageable      3844   94          17.30 %  0.05 %     2.06 %       1.90 %      71.02 %     62.62 %            10.98 %     40.92 %   4.16 %   

You can ignore most of those columns; this is just a little query I use to keep an eye on how the queue is doing. The important ones for this discussion are PctClosedOrDeleted and PctEdited: a majority of questions triaged as Unsalvageable do get closed or deleted, while a majority of questions triaged as Requires Editing (referred to internally as "should be improved", since that was the original name) do get edited.

I wish both numbers were higher, especially the first one. Frankly, I wish deletion was a heck of a lot easier. But for the purpose of review, it doesn't have to be: the purpose of triage is to maximize the value of a limited resource, not stretch it ever-thinner.

All that said... I'm sorry the Help & Improvement queue hasn't turned out better than has. We knew from the start that there'd be a lot of questions ending up there that couldn't be improved - that's where the "help" bit comes in:

help and improvement sidebar guidance: "If you were unable to fix everything wrong with this post then please try to fix as much as you can - then identify what additional information is needed from the author."

The idea was, questions that needed edits by their author could still be helped by someone taking time to point that out to them... Before they hit the homepage and got downvoted to hell. Unfortunately, this particular use is, uh, a bit buried: it's not hinted at in the queue instructions, or the action buttons, or even the initial guidance in the editor - unless you're watching the sidebar after you've finished editing, it disappears. You've probably seen it; I kinda doubt most reviewers have though. And that kills a lot of the potential utility here, since just editing an unclear question doesn't really set the author up for success. In fact, here's the outcomes for questions going through the H&I queue over the same time period as the triage stats above:

ReviewActions                    Tasks MinsToReview MinsEdited Total   Upvoted Downvoted PosScore NegScore ClosedOrDeleted Answered 
-------------------------------- ----- ------------ ---------- ------- ------- --------- -------- -------- --------------- -------- 
Edit Body                        2202  256           2         46.11 % 3.72 %  5.27 %    10.85 %  29.75 %  24.89 %         8.40 %   
Edit Body, Edit Body             4      30           3         0.08 %  0.00 %  0.00 %    0.00 %   50.00 %  50.00 %         25.00 %  
Edit Body, Edit Tags             75    342           3         1.57 %  21.33 % 0.00 %    20.00 %  26.67 %  22.67 %         10.67 %  
Edit Body, Edit Tags, Edit Title 1     ***           3         0.02 %  0.00 %  0.00 %    0.00 %   100.00 % 100.00 %        0.00 %   
Edit Body, Edit Title            677   399           2         14.18 % 6.20 %  3.99 %    10.04 %  33.68 %  26.29 %         8.12 %   
Edit Body, Edit Title, Edit Body 2     281           3         0.04 %  0.00 %  0.00 %    0.00 %   100.00 % 50.00 %         0.00 %   
Edit Body, Edit Title, Edit Tags 53    282           5         1.11 %  39.62 % 7.55 %    24.53 %  22.64 %  24.53 %         3.77 %   
Edit Tags                        9     366           1         0.19 %  0.00 %  0.00 %    33.33 %  22.22 %  33.33 %         33.33 %  
Edit Title                       15    218           2         0.31 %  20.00 % 13.33 %   13.33 %  40.00 %  46.67 %         6.67 %   
Edit Title, Edit Tags            1      15           1         0.02 %  0.00 %  0.00 %    0.00 %   100.00 % 0.00 %          0.00 %   
not edited                       1737  189          null       36.37 % 0.00 %  0.00 %    8.41 %   43.41 %  41.16 %         13.36 %  

A couple of notes there:

  • "not edited" means the review didn't actually complete - either it aged away, or the question was deleted while the task was still in the queue.
  • the numbers don't match the ones above, as not every question that goes through Triage as "Requires Editing" ends up in the H&I queue (questions that get answered, closed, deleted, or score < -1 are excluded).

And, a few observations:

  • clearly most of the edits observed in the Triage stats aren't coming from the H&I queue. There are less than half as many edits as are indicated for questions triaged as Requires Editing.
  • the outcomes for these questions are... pretty terrible.

Does that mean you can't do well in the queue? No, but... Here's your personal H&I stats over that same time period:

ReviewActions                    Tasks MinsToReview MinsEdited Total   PosScore NegScore ClosedOrDeleted Answered 
-------------------------------- ----- ------------ ---------- ------- -------- -------- --------------- -------- 
Edit Body                        216   318           1         68.79 % 6.02 %   53.70 %  38.43 %         3.70 %   
Edit Body, Edit Tags             2     311           1         0.64 %  0.00 %   0.00 %   50.00 %         0.00 %   
Edit Body, Edit Title            81    326           1         25.80 % 6.17 %   48.15 %  44.44 %         4.94 %   
Edit Body, Edit Title, Edit Tags 9     298           2         2.87 %  0.00 %   22.22 %  33.33 %         0.00 %   
Edit Title                       4     106           1         1.27 %  0.00 %   75.00 %  100.00 %        0.00 %   
Edit Title, Edit Body            2     177           2         0.64 %  50.00 %  50.00 %  50.00 %         0.00 %   
15
  • 5
    It's a nice post, but if you could simply rename "Requires Editing" to something better, that would make reviewers already a little happier.
    – Cœur
    Commented Nov 9, 2018 at 4:20
  • 4
    Honestly... I think I'd rather keep the button... And get rid of the Help & Improvement queue, @Cœur. Roughly the same results, less work for everyone.
    – Shog9
    Commented Nov 9, 2018 at 4:28
  • 1
    "Incomplete or unsalvageable" / "Looks OK, but could be cleaned up with an edit" / "Looks OK"
    – Cœur
    Commented Nov 9, 2018 at 4:32
  • 10
    I understand you may want to remove Help & Improvement. In the mean time, we could have "Requires Editing" changed to "I should Edit it", it would clarify the meaning of this button as who can perform the edit.
    – Cœur
    Commented Nov 9, 2018 at 4:44
  • 5
    note: I'm giving jungle animals drawing lessons on saturday. I'd strongly suggest that you enroll. Commented Nov 9, 2018 at 6:50
  • @Shog9, seeing that you can pull some string around here, how are you going to tame the elephant?
    – Luuklag
    Commented Nov 9, 2018 at 8:24
  • 4
    Give people unlimited steward badges on the close queue, one every 1000, and that number will go down. Maybe not all these votes will be great, but there will be more votes...
    – GhostCat
    Commented Nov 9, 2018 at 8:48
  • @GhostCat I turned your comment into a Question of its own: Award additional badges for above 2,000 review tasks. Looks like it gets downvoted quickly.
    – Cœur
    Commented Nov 9, 2018 at 9:40
  • 2
    the purpose of triage is to maximize the value of a limited resource -- if you are serious about that: In close queue, prioritize questions triaged as unsalvageable to unconditionally stay on top "I expect quicker close reviews to help leverage the obtained strong signal about the question quality and prior efforts involved in detecting and reviewing of said triaged question..."
    – gnat
    Commented Nov 9, 2018 at 9:46
  • @Cœur I know, that is why I dont make such suggestions on META ;-) but you get my upvote
    – GhostCat
    Commented Nov 9, 2018 at 9:48
  • 3
    shouldn't the freehand elephant be red?
    – user5940189
    Commented Nov 9, 2018 at 12:08
  • 1
    Also I see a lot of good observations in your post Shog, however what are the conclusions to draw from this, and how to move forward, asuming we want to?
    – Luuklag
    Commented Nov 9, 2018 at 12:14
  • 1
    Why don't we just automatically close questions marked as unsalvageable? Commented Nov 9, 2018 at 12:19
  • 18
    The elephant has been there so long it's basically part of the furniture now. I think some people started using it as a couch at some point. Commented Nov 9, 2018 at 12:28
  • 1
    I don't know why there's such mockery of that image - it's a perfect depiction of: "What do you get if you cross a sausage dog and an anteater?". The thought of also showing it holding up a red ball is pure genius... I'll open the bidding for this unique Shog9 masterpiece at $5! :)
    – Jon Clements Mod
    Commented Nov 9, 2018 at 12:48
34

I think it's a really good idea to have a clearer distinction between Requires Editing and Unsalvageable.

The only issue I have with your current proposal is replacing the word Unsalvageable with the phrase

Requires Editing by Questioner

which is, as you stated, open for change of words, so I hope a middle ground can be found.

I feel like new users wouldn't associate a thought process of "This needs editing" with a closure flag, which is the case here. If you were to click "Requires Editing by Questioner", a dialog pops up saying "This question doesn't belong here because:", which I think will make users think: "Oh no, it belongs here, it just requires a few edits".

Furthermore, there are different flag options (Duplicate, spam, rude) that simply do not fall under the category of "Requires editing by questioner", whereas the word Unsalvageable covers all these grounds.

5
  • 8
    I agree, that proposal of "Requires Action by questioner" would be less confusing. Of course, the real answer would be that this button gives you a slightly different next window, that explains your options and so on...
    – GhostCat
    Commented Nov 8, 2018 at 12:36
  • 3
    "Not answerable" might work...
    – user4639281
    Commented Nov 8, 2018 at 18:56
  • 4
    Or, one more button. Action by user, Needs closing, Edit by other, OK, skip.
    – GhostCat
    Commented Nov 8, 2018 at 19:41
  • I’m not a native speaker, but wouldn’t “asker” instead of “questioner” be more idiomatic? In the context of SE the abbreviation “OP” might even be clearer.
    – 11684
    Commented Nov 11, 2018 at 0:32
  • 1
    As someone somewhat recently enlightened by a comment from @GhostCat, I agree there needs to be a distinction between the labels, but think that unsalvageable is still a pretty clear description of the current state of the question/answer. Perhaps incorporating formatting/grammar into the button label for 'requires editing' would do a better job of conveying its purpose.
    – Matt
    Commented Nov 11, 2018 at 10:11
-5

I would assert that the problem here is that the options do not reflect all possibilities. Describing a post that requires additional information from the asker as 'Unsalvageable' is plainly incorrect. It can be salvaged - by being improved by the person asking the question. Required editing, to anyone who hasn't read the instructions doesn't describe what is meant, which is what is called a minor edit on other sites. Instead the options should be something along the lines of:

  • Looks Okay
  • Requires minor edit
  • Needs to be rewritten
  • Unsalvageable

So that questions that do belong on the site once they have the relevant information can be improved and then answered, and the spam etc. separately put in a bin and disposed of.

4
  • 2
    "Describing a post that requires additional information from the asker as 'Unsalvageable' is plainly incorrect." That depends on who is doing the salvaging. The purpose of these queues is to find things that our curators can salvage. If it "needs to be rewritten" by the post owner, then it is completely unsalvagable by us, and should be closed.
    – Davy M
    Commented Mar 13, 2020 at 10:49
  • The purpose should be to remove unhelpful content from the site or improve it. I don't know where the conversation was had about how to do this but the fact that it is not intuitive to reviewers is a hint that the process is wrong. It is not efficient to keep telling new users after the fact that terms don't mean what a normal person would expect them to mean.
    – RET
    Commented Mar 13, 2020 at 10:54
  • 2
    I agree with the first part, but I don't see the need to differentiate "needs to be rewritten" from "unsalvageable". If a question would be entirely rewritten from the ground up, surely an "unsalvageable" post could become a decent one. So, just condense that into three options: "fine as is", "could do with some cleaning up", and "needs to be substantially improved by original author".
    – deceze Mod
    Commented Mar 13, 2020 at 15:41
  • If a question can be rewritten to become a different one, then it is not unsalvageable as that word is commonly understood. It would be taken to mean that it cannot be salvaged, not that it requires a particular person to salvage it. The triage just appears with no explanation on peoples accounts when they reach 500 reputation, so they will assume it means what it means in normal usage.
    – RET
    Commented Mar 13, 2020 at 15:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .