8

I was reviewing this question while it was in staging ground.

This question exhibits features that commonly lead to closure specifically for the tags and topics listed, but isn't the point of staging ground to provide feedback to OP to improve the quality of the post before it is released?

Should the level of standard be raised so that a question is forced to stay in this staging ground for longer or require votes before it is published?

Or do we improve or provide a forum for raising disputes or conversing with other reviewers? Should multiple votes be required to publish? I feel like the process this question went through has resulted in no net benefit to the question, OP or the community.

0

4 Answers 4

4

I feel like the process this question went through has resulted in no net benefit to the question, OP or the community.

I'm not sure we can reasonably expect a net benefit for every question individually. If we end up seeing a net benefit for newcomer questions on aggregate, I think it'd be fair to call the SG a success.

Should the level of standard be raised so that a question is forced to stay in this staging ground for longer or require votes before it is published? ... Should multiple votes be required to publish?

The limiting resource is reviewer count. If we had unlimited (good) reviewers, we could tighten the publishing requirements without sacrificing time-to-publish. But given a realistic number of reviewers, we can't make the publishing requirements too strict without risking:

  • either auto-graduating too many unreviewed questions (thereby publishing with zero intervention anyway)
  • or delaying auto-graduation and forcing questions to wait for reviews (thereby increasing time-to-publish)

From what I remember of the beta test, the SG team is actively monitoring/calibrating these factors based on aggregate stats.


Re: comments

I for one don't particularly see a problem with "increasing time-to-publish" ... a newcomer with an interesting question can wait a few more days

For interesting questions, the problem is that time-to-publish is strongly correlated with question/platform abandonment. If we let time-to-publish get too high, we'll just lose those promising contributors.

and a newcomer who wants "urgent" "help" can be properly turned away as someone who has flatly refused to engage with what the site is

For bottom-of-the-barrel questions, those are low-hanging fruit in a (gamified) review context. I wouldn't expect many to slip through the cracks.

The ones that will be hurt by extending the time-to-publish parameter will be somewhere in the middle — questions that are potentially useful but require extra work from reviewers. Those are more likely to be skipped/delayed, but those middle-of-the-pack questions are also the ones that would be most greatly benefited by SG reviewers. Those askers can become valuable contributors if guided in the right direction, but we'll lose them if time-to-publish gets too high.

8
  • 3
    I for one don't particularly see a problem with "increasing time-to-publish". If 15-year-old questions can still be improved by editing, a newcomer with an interesting question can wait a few more days - and a newcomer who wants "urgent" "help" can be properly turned away as someone who has flatly refused to engage with what the site is. Commented Jun 6 at 7:29
  • 2
    @Ðаn which is indeed something I've advocated for in the past ;) Commented Jun 6 at 13:58
  • 1
    Eh, I don't know about that. There is definitely more functionality here, and it looks useful and actually thought out (template comments and improved categorization/organization of close reasons). Commented Jun 6 at 14:02
  • @KarlKnechtel Yeah, but in a review queue context, I don't expect low-hanging fruit to slip through the cracks (either on the bad end like "urGeNT HeLp" or on the good end of exceptional questions). The ones that will be hurt by extending the time-to-publish parameter will be somewhere in the middle -- questions that require extra work from reviewers and will be more likely to be skipped.
    – tdy
    Commented Jun 6 at 14:40
  • 1
    Those middle-of-the-pack questions are also the ones that can be most greatly impacted by the SG -- they have high potential to be guided in the right direction and become useful contributors. Time-to-publish is highly correlated with question abandonment, so those useful contributors are the ones we risk losing if we let time-to-publish get too high.
    – tdy
    Commented Jun 6 at 14:40
  • 1
    The issue I had as a reviewer was that there was not enough time for me to engage with OP. I read the question and immediately recognise it is a very common issue, so I thought to try to write some helpful feedback and commit to working with OP to turn this into an answerable post, but in that time the question was published. So I guess for me, more time to publish in this case would have helped, or if it was reactive so that there is a cooling off time triggered by the chat process... Except I never got to start my chat... Commented Jun 7 at 7:02
  • "I read the question and immediately recognise it is a very common issue, so I" - started searching for a duplicate to vote to close? "thought to try to write some helpful feedback and commit to working with OP to turn this into an answerable post" Oh. But why? If you already know what the issue is, and you know it's a common issue (therefore a canonical should already exist), what do you hope to gain from making it "answerable"? It won't be suitable for receiving new answers anyway, because it's a duplicate. Commented Jun 7 at 17:25
  • I get Chris's point about someone else publishing a post while you're literally reviewing it, but I'm not sure how/if that can be "fixed" (assuming it's even something to be fixed). I had that happen to me even in the beta when the publishing reqs were higher (multiple approvals).
    – tdy
    Commented Jun 7 at 19:15
2

This question exhibits features that commonly lead to closure specifically for the tags and topics listed, but isn't the point of staging ground to provide feedback to OP to improve the quality of the post before it is released.

It comes across that you see some kind of conflict or contradiction here, but I'm not sure exactly what you think you're seeing, and I doubt it's actually a real issue.

Yes, the point of SG is to provide feedback so that a post can be improved before it's published - in the cases where meaningful improvement is possible.

However, the point of closing a question, similarly, is to provide feedback so that a post can be improved before it's reopened - again, in the cases where meaningful improvement is possible.

It's the same either way. The question should not be answered in its current state; if it looks fixable, the next step is to encourage OP to fix it (or cooperate in fixing it) to the point where it meets standards. Closing a question prevents it from being answered, just like holding it in SG does.

But SG is improving on this, by at least consciously trying to distinguish between fundamentally fixable ("minor edits" and "major changes") problems and fundamentally unfixable ones (duplicates and "off-topic" - the latter currently incorporates the "not reproducible or caused by a typo" close reason). I would argue that sometimes duplicates should be published (because they help find the canonical), but overall it's a significant improvement.

Should the level of standard be raised so that a question is forced to stay in this staging ground for longer or require votes before it is published?

I think our standards are fundamentally the same as always. We're just now being confronted, much more brutally than usual, with the prospect of actually consistently enforcing them.

That said, there are many unanswered questions remaining about how many votes should be needed for each "state transition";ld mean exactly the things that could be fixed by third-party editing, and that this option should only be selected as an apology for not directly making the edit; but...); etc. etc. But I think it will take several posts on Meta to disentangle that.

7
  • 1
    "...our standards are fundamentally the same..." But with some variation. To reduce the variation we included some averaging in the past. If we want to go without that averaging, we can try to be more consistent or live with the occasional erratic result (potentially confusing question creators) or go back to averaging. Commented Jun 6 at 8:58
  • 1
    (It seems incomprehensible near "ld mean".) Commented Jun 6 at 15:54
  • 1
    @PeterMortensen I genuinely don't know what happened there. The post seems to be missing at least an entire line (perhaps multiple) from what I saw and thought I was submitting for the post. The "ld", I think, is the end of what was supposed to say "should", but several more words are missing and I don't recall exactly how I phrased it before. Or even exactly what I said. Commented Jun 6 at 21:09
  • It comes across that you see some kind of conflict or contradiction here Yes, that is exactly it. Staging ground should be a "safe" space to have opinionated discussions with OP with an aim to create either a higher quality question or for them to leave with a better understanding of the process. The sort of user who asks this sort of question ends in a frustrated user who doesn't get an answer at all and doesn't understand why it was closed... You can smell this getting closed a mile away, very common closed question for combinations of [C#] and [SQL] tags. Commented Jun 7 at 7:08
  • 1
    Closing a question in practice doesn't provide the sort of feedback you are expecting, or at least the majority of low rep / new users will not receive that way and experienced users understand the reality of the re-open queue. Closing a question is generally received like a stern warning "Please do not do that again." or perhaps a more jovial "Try again next time". It is probably the worst feedback mechanism SO has but I think Staging ground has the potential to fix a lot of that. Commented Jun 7 at 7:13
  • "Closing a question in practice doesn't provide the sort of feedback you are expecting" - it puts a giant shaded box at the top of the question that tells the user the reason for closure and provides links to the relevant Help Center articles. What other feedback do you imagine could possibly be offered instead? Commented Jun 7 at 17:22
  • "The sort of user who asks this sort of question ends in a frustrated user who doesn't get an answer at all and doesn't understand why it was closed" - if we tell them what the site is about, that they aren't entitled to answers, and that questions are expected to meet specific standards here, a large fraction of users will still appear not to understand. But there isn't any other explanation we can offer, because it's the truth. Commented Jun 7 at 17:23
2

This question mostly discusses whether one reviewer getting something out of the staging ground is a good idea rather than the concrete example but I thought I'd chime in as having been the reviewer who caused the linked question to leave the Staging Ground.

It was not robo-reviewing (because I would have nothing to gain from sabotaging what I believe to be the best feature to happen to this site in a while). I remember what I thought when I reviewed that question. I believed it was answerable by someone knowledgeable in the area (stating a problem, where it is, the code looked like a reasonable length for an MRE, no major things I would edit, etc.). It was certainly better from a quality standpoint than most of what I skipped/marked as major changes beforehand.

Now I can also see why someone would think otherwise and seeing 3 close voters and 4 commenters thinking otherwise would be silly to argue against - I was wrong.

I just today wanted to go through questions I had approved to see how those questions fare now to adapt my reviewing - I feel like I hardly approve anything in the first place so I probably should follow up on what I do approve as I would in other review queues. I guess this is a good time to do just that.


While I'm writing on Meta instead of passively reading I'll put in my two cents about this in general: I think it's generally a good idea that one reviewer can approve a post/get it out of Staging Ground.

Mistakes will happen (q.e.d.) but I believe even somebody approving a question in the Staging Ground incorrectly will still be a net positive result compared to if the question auto-graduates (which might happen more if more eyes are required than are available) or does not see the Staging Ground at all as it was seen by at least one more person than otherwise before being posted publicly.

2
  • Hello and sorry for dragging you under the bus! You are right, it's about the process. This common question is one I field a lot on SO and in the workplace. I will often Dup or close vote them, this time for whatever reason I wanted to help and shield OP from what was about to come next. I was hoping to get some time chatting with OP to teach them both about what they posted and how as well as how SO works. I might start the chat quicker next time but even then you wouldn't see it unless you refreshed the screen (that was the timing we're talking about here) Commented Jun 7 at 7:23
  • I hope that SG makes a net positive experience for new users, this is just an SQL 100 level issue that most new C# devs fall into and SO isn't a supporting environment for those users. It wasn't even obvious to me who to reach out to when my submit failed because the post was published. There's just too many faceless critics out there and in this case I had some spare time and was hoping to help. Commented Jun 7 at 7:28
1

The question was approved by a single user and later closed by three others. Probably just means that single persons vary a lot in their estimation of what is good/bad and that some sort of aggregation, if possible, might fare better. Especially automatic approvers could theoretically destroy the purpose of the staging ground by simply bypassing it for many questions.

We could think about increasing the threshold for approval, for example from one vote to maybe two consecutive approval reviews. Approval reviews probably can be done faster than other reviews. Or do that only if someone else required revisions or otherwise close voted.

And we need to pay attention to automatic approving reviewers. If people approve lots of content that ends up closed, they shouldn't continue being reviewers there.

If you ask me, an even better idea would be to rather dissolve the sharp separation of staging ground and normal ground. The sharp separation requires us to think about how and when to transfer questions between one ground and the other and decide what to do with past reviews. The ideal case would be that a question leaving the staging ground is without obvious problems, but reality is different. Approval may not be a strong guarantee and the staging ground is just another way we can interact with content. Maybe this interaction could be integrated a bit more with what follows.

7
  • 1
    mods have a dashboard that surfaces stuff that could be robo-reviewing. hopefully flagging can sort of handle slightly more sophisticated robo-reviewing with random review actions and comments...
    – starball
    Commented Jun 6 at 6:39
  • you seem to misunderstand the purpose of SG? (or I'm misunderstanding you). it's not to make every question that comes out perfect. that's not even the goal. read stackoverflow.com/help/staging-ground-reviewer-guidelines. I don't think the distinction should be reduced. SG is essentially starting questions closed by default (think about it. the closure state effectively means no answers, and in some cases, eventual roomba). in my eyes, this is a pretty great system achievement.
    – starball
    Commented Jun 6 at 6:42
  • @starball Sorry, perfect may have been a bit too much. Replaced it with "without obvious problems" which is what I meant. Commented Jun 6 at 6:57
  • 2
    @starball "I don't think the distinction should be reduced. SG is essentially starting questions closed by default..." That's right. They start closed and I also didn't want to change that. Still the distinction feels a bit like parallel structures to me and I would integrate it more. For example, by automatically starting new questions closed, forbidding to vote on closed questions and auto-reopening of initially closed questions a certain time, we could have had most of the staging ground much earlier at relatively low effort. Comments were always there. Commented Jun 6 at 8:53
  • 3
    @NoDataDumpNoContribution: Re "by automatically starting new questions closed, forbidding to vote on closed questions and auto-reopening of initially closed questions a certain time, we could have had most of the staging ground much earlier": Yes, but it would probably have been deemed way too radical and never accepted. The only feasible way to reform the system is small isolated sand boxes that nobody feels threatened by. Commented Jun 6 at 15:45
  • 1
    @PeterMortensen Also a real life lesson for me. Changing existing products is difficult. Calling something a new name and saying it's an addon often works. Commented Jun 6 at 17:13
  • 1
    I like this sentiment. The chat space in Staging Ground I think is a good idea, so we can have an actual conversation with OP without the dup and close happy robots interfering, so should OP perhaps get a say when the question gets published, should they be able to gatekeep if they think the question is good enough to release to the masses? Keeping in mind that many SG users will not have sufficient rep to participate in normal chat sessions. Commented Jun 7 at 7:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .