72

I have noticed quite a bit of very low-quality questions in the Help and Improvement review queue, such as this and this.

Most of the blame is that triage fails to properly explain to new reviewers on the use of the Requires Editing selection.

To a new reviewer, Requires Editing seems like a viable option to most of the questions asked; after all, if the OP would edit the question to make it on-topic, then that's better then saying the question is unsalvageable, right?

Wrong.

Unknown to most new reviewers - really not to any fault of their own - is the very vaguely tied-in support information on how to pass judgment on low-quality questions.

My Proposal

For the first x amount of times (let's say 20) that a new reviewer clicks Requires Editing, a notification will appear and prompt the reviewer with something along the lines of:

Notice: You are saying that this question is on topic and in an answerable state. This question has minor issues that you can see yourself being able to correct by editing. If you believe that you wouldn't be able to edit the question yourself, then you may need to select Unsalvageable instead.

Continue with Requires Editing Go Back Skip

I do enjoy the idea of community moderating, and while the community should be able to value its own opinion on what is considered on-topic or not - I believe this should still have a general consensus as to what the baseline expectations are and make that clear in the review queues.

15
  • 5
    Related: meta.stackoverflow.com/questions/319128/… Commented Feb 10, 2018 at 9:51
  • 7
    Part of the problem is that the guidance in the review queue itself says that if "edits by the author or by others" can fix the post, use "Requires Editing". See: meta.stackoverflow.com/questions/321895/…
    – gparyani
    Commented Feb 10, 2018 at 23:20
  • 1
    @gparyani Yes, the meaning I think is "Requires Editing for questions where edits by any user could result in a question that is clear and answerable", specifically mentioning the author is the problem Commented Feb 10, 2018 at 23:25
  • I think 20 is too much; maybe 5 is enough.
    – gparyani
    Commented Feb 10, 2018 at 23:29
  • @gparyani the number can even be dynamic. Saying a minimum of 5, but if your responses tend to lean more towards the Requires Editing while others lean more towards Unsalvageable then you may be lucky enough to see the prompt more than others. Commented Feb 10, 2018 at 23:50
  • 2
    99 views and already 25 up-votes and no down-votes. I think you struck a nerve...
    – Stephen Rauch Mod
    Commented Feb 11, 2018 at 1:13
  • 17
    While I support doing this, the real first step is to change the text that gives guidance for "Requires Editing". Currently, reviewers are doing exactly what the guidance is telling them to do. Expecting them to do differently isn't realistic. The real problem is that SE hasn't bothered to change that text in the 2 years since the change was requested. It's unconscionable that a change which would take such a low level of effort, but would at least start guiding people in the right direction, hasn't happened in the last 2 years.
    – Makyen Mod
    Commented Feb 11, 2018 at 3:34
  • 9
    I was one of those clicking "Requires editing" way too often. Only after almost 20 reviews did it become obvious to me that I was doing it wrong. And I only discovered that because I was randomly browsing Meta, not because the queue taught me how to do it. So we definitely need some better guidance for new reviewers.
    – Graipher
    Commented Feb 12, 2018 at 8:34
  • 2
    Suggestion: "you can see yourself being able to correct by editing" should be "you can see yourself being able to correct by editing without involving OP". I know the sentence already says that enough; but being explicit/verbose is better.
    – Amit Joshi
    Commented Feb 12, 2018 at 15:19
  • There's a full guide linked from the top of every Triage review that is a heck of a lot more nuanced than the single-sentence descriptions shown to new reviewers, @Makyen.
    – Shog9
    Commented Feb 13, 2018 at 2:20
  • 4
    @Shog9, Thanks for mentioning that, I was not remembering that the link was there. However, it really doesn't change how I feel about the short description being directly contrary to what's actually desired, and the fact that something so simple hasn't been changed. It's just text. Sure, it may need to be localized/translated for other sites, but that can just be put into a queue for accomplishing localization (or however, it's done at SE).
    – Makyen Mod
    Commented Feb 13, 2018 at 3:11
  • 2
    @Shog, If we're getting into the Triage interface, it's unclear to me why there's an "Unsalvageable" button. What's actually being asked of users there is to either flag, or vote-to-close (if > 3k rep). Calling it "Unsalvageable" just creates an additional, confusing semantic which is different than elsewhere on the site. IMO, a much better UX would be to just call it what it is: "Flag", "Flag or Close", or even better: two buttons "Flag" and "Close". This would keep, and reinforce, the semantics which are already used throughout the site, which is what users are already familiar with.
    – Makyen Mod
    Commented Feb 13, 2018 at 3:21
  • You're assuming more familiarity than is warranted, @makyen. Most folks don't use either of those functions often (or at all); one of the lessons we learned when rolling out the close review queue was that an awful lot of people would click "close" regularly just to remind themselves of what options were there. Similarly, we redesigned the flag dialog for Triage to summarize the options; the distinction between various flags or close reasons isn't as obvious to many people as it is to folks whi have been using them for years.
    – Shog9
    Commented Feb 13, 2018 at 3:45
  • 2
    @Shog9 I can understand that. The flag dialog appears to be the same between Triage/questions. I assume it was redesigned and used for both (looks good). Due to character limitations, I left out the benefit of using the same semantics that using such results in people becoming more familiar with them, rather than training them on some unique semantic which is only used in that circumstance. I would assume that it's better to have people overall more familiar with the site. Using the same semantics, to mean effectively identical things, is helpful in that regard.
    – Makyen Mod
    Commented Feb 13, 2018 at 4:09
  • As a new reviewer, I just need to say I totally missed all the signs and made some base assumptions about what the "needs editing" button was for. I sometimes still hit it by accident, followed by a loud expletive. It seems so native to say to the OP "this needs editing"
    – Chiperific
    Commented Oct 29, 2020 at 3:22

2 Answers 2

8

I think the pop-up would just be annoying. Worse, there's a very real chance it would train reviewers to click through pop-ups without reading them - a dangerous approach given that the other pop-ups in review are attached to things like deletion.

But, I like the approach you took with wording. It's long, but it side-steps the problems inherent in asking reviewers to guess at what others are able to fix by focusing on what the reviewer themselves believes is fixable. So, as of today, the guidance now reads:

Requires Editing for questions that you can make clear and answerable by editing

...and also includes a second link to this meta answer as the last line in the expanded instructions (same answer linked to in "help separating questions" in the short explanation at the top).

0
15

So...

We had this idea, back on about three years ago, that the Help & Improvement queue would be sort of a mentoring system for new users. Not, "hey, here's where you fix code formatting" but rather "hey, here's where you add clarity and apply some socratic method to get folks to think about the questions they're writing"...

...That may have been a bit ambitious.

As a result, most of the stuff in the H&I queue is crap because most of the stuff going into Triage is crap. It's not the worst of the worst, but it's certainly not gonna be stuff that you'd seek out ordinarily. Also, we don't actually tell anyone that their question has been identified as borderline and in need of serious help - so they're not exactly primed to cooperate with the process either. We intended to do that, but kinda ran out of resources before we could agree on a method of doing so; there's probably a nice way of telling someone "hey, your question is in purgatory", but it's beyond my skill.

Aaaanyway, I like your idea. Your proposed wording is too verbose, but admonishing the reviewer to use their own judgement is a good approach: one thing I've repeatedly observed is that folks tend to overestimate what askers are able to accomplish on their own and underestimate what 3rd-party editors can do - often severely. Asking the reviewer to apply their own experience and mark the question for editing if they could imagine themselves fixing it perhaps would side-step this problem.

27
  • My experience: Ten H&I ~ Five CV.
    – iBug
    Commented Feb 11, 2018 at 0:46
  • 1
    I recall that when H&I was new, a review there was hard work. It had to satisfy several criteria. Nowadays, even the tiniest edit counts as a review there. I suppose those hard criteria were softened for a reason. But if we want it to be a mentoring system, the bar should be a little higher. Commented Feb 12, 2018 at 8:51
  • I think there's been a prior proposal to just rename the button from "requires editing" to (something along the lines of) "I could edit this." They wouldn't need to edit it, but it would be clear that it wasn't authorial edits but rather community edits could salvage it. Commented Feb 12, 2018 at 19:46
  • 6
    -1 for supporting this suggestion without even acknowledging Makyen's objection. Users who mark close-worthy posts as "Requires Editing" do so because the instructions they are explicitly given in the review queue require them to. The solution to users following bad instructions is not to then immediately scold them for not magically knowing that they should've disobeyed you; it's to stop telling them to do the wrong thing in the first place.
    – Mark Amery
    Commented Feb 12, 2018 at 19:47
  • 1
    @MarkAmery In Shog's defense, he answered this @ 2018-02-10 23:40:32Z, whereas Makyen's objection didn't come in until 2018-02-11 03:34:24Z (approx 4 hours later)
    – user6576927
    Commented Feb 12, 2018 at 22:50
  • @TheNotSoGuru Oops. Yeah, complaining that Shog didn't answer a comment that didn't yet exist is, uh, perhaps asking a bit much. The underlying issue still stands, though (and has been complained about on Meta over and over for the last two years).
    – Mark Amery
    Commented Feb 12, 2018 at 22:52
  • @MarkAmery Regardless of when the comment came in, you and Makyen have a valid point and they probably just need to overhaul the entire guidance of the review queue altogether... It's all pretty vague (if not contradictory to what is actually expected) imho. Commented Feb 12, 2018 at 22:57
  • 1
    To be clear, I think K.Dᴀᴠɪs's strategy could work just as well as a replacement for the current guidance, @Mark. The issue I've had with previous suggestions for altering that guidance is that they badly want to ask reviewers to predict whether or not the question is complete in its current form - that is, they must be able to say with confidence that the question can be corrected by a 3rd-party; in keeping with the triage metaphor, this is akin to demanding that a hospital admit patients only when they can guarantee those patients will not die.
    – Shog9
    Commented Feb 12, 2018 at 23:35
  • 1
    @Shog9 the reasoning you have just given are new to me. Maybe this could have been better communicated in the past ;). I am glad, we finaly found a good solution to the Triage problem. Commented Feb 13, 2018 at 7:37
  • @Shog9 The issue you describe - people demanding perfect judgement in triage, when triage is meant to be quick by design and need not be perfect - is orthogonal to whether we should tell users to mark things that specifically require additional information from the author as salvageable. To mutilate your metaphor further, if the burns unit is swamped with non-burned people who've been shot because all the medics have been taught to send them to the burns unit, that's a problem that needs fixing, and is independent of how good the medics are at assessing patients' survival chances.
    – Mark Amery
    Commented Feb 13, 2018 at 15:28
  • 2
    @Shog9 And even if you think the two issues are not orthogonal - perhaps because you think reviewers are likely to underestimate the often-impressive psychic powers that editors can bring to bear when improving questions, and that "only improvable by the author" posts are usually actually improvable by anyone - that's still not an argument for introducing explicitly contradictory guidance. What's a new reviewer meant to think when we give them instructions, wait for their first review, then say (effectively) "NOTICE: You should be completely ignoring our instructions." It makes no sense.
    – Mark Amery
    Commented Feb 13, 2018 at 15:34
  • 1
    I've been thinking about this more, @gnat... I kinda wonder if something that just prompts for a description of what is needed might be more effective than a nag - "Please describe what is needed from an edit: "
    – Shog9
    Commented Mar 1, 2018 at 16:12
  • 1
    No, I'm thinking something akin to what we do for custom OT reasons, @gnat: please leave a comment for potential editors. This post could use some editing to fix... [ ]
    – Shog9
    Commented Mar 1, 2018 at 22:31
  • 1
    Very, very few, @gnat. Depends on how you define it, but the simple definition - "scores > 0" - gives about 6-7%. OTOH, ~47% get closed and/or deleted, which is pretty solid for an option that doesn't expressly feed into the close queue.
    – Shog9
    Commented Mar 1, 2018 at 23:11
  • 1
    @gnat absolutely I can agree with that. This way it doesn't always have to prompt the reviewer, but it will let the reviewer aware that the question has received negative feedback and their "Requires Editing" may or may not be the best option. Regardless, the standard instructions language need to be updated regardless of any outcome, as it's currently deceptive to the reviewers. Commented Mar 4, 2018 at 5:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .