93

In a previous announcement, we outlined our commitment to giving back to the community in the upcoming year. Staging Ground was among the initiatives we discussed. We are excited to announce that Staging Ground is transitioning out of beta and will officially launch to all Stack Overflow users in the coming months. While we do not have a set date for this launch, we will notify the community when we have a more concrete timeline available.

What is Staging Ground, and how can I learn more?

Staging Ground is a place where new question-askers can get help and feedback from experienced reviewers on Stack Overflow. We previously unveiled this feature in early 2022.

While we previously paused the development of Staging Ground to prioritize other projects, we’re excited for Staging Ground to return and expand to everyone on Stack Overflow. You can learn more about what we learned from the Staging Ground beta here.

Staging Ground is leaving closed beta

Previously, participation by reviewers in Staging Ground required an opt-in. It was in beta testing and was therefore limited to users who were both aware of and interested in trying it out. With the public launch, any user who has earned 500 reputation points and has access to review queues can join in the efforts to improve the quality of new questions on the platform.

One of the best ways to turn prospective curators into experienced ones is to offer them ways to help teach and connect them directly with the outcome of their advice. We’re hopeful that setting a relatively permissive threshold will not only streamline the content review process but help develop the next generation of content curators. We don’t want to set a barrier so high that it keeps interested users out, nor do we want to set a threshold so low that it risks leading to a rougher new user experience. For this reason, we will be using the standard review queue privilege of 500 reputation points as the threshold required to perform reviews in Staging Ground. It strikes a balance that we’re hoping is fair - and matches existing review queues as well.

Improvements before launch

We are making a few quality-of-life tweaks and improvements before the public launch. You can look forward to the following changes:

  • A few new badges to recognize the contributions of active Staging Ground curators.
  • Additions to the notification system for significant events, such as when a question is published or submitted for re-evaluation.
  • Improved moderator tooling to help uncover curators who are misusing Staging Ground and take actions analogous to existing review queue restrictions.
  • We are taking some initial steps to build out onboarding resources for both question authors and reviewers.
  • Rate-limiting for askers who post questions that consistently take significant work to prepare for public Q&A.
  • A few user experience updates and bug fixes to clean up common points of confusion and other less common/minor incorrect behaviors.

If there’s anything you think is critically important leading up to Staging Ground’s launch, please let us know - the earlier, the better! However, we will continue to iterate and address requests, and bugs post-launch as well.

Get involved

The quality of content on Stack Overflow has always been thanks to the active participation of all of you - the curators of this site. The success that we hope Staging Ground sees will ultimately come from you.

We look forward to seeing the community's engagement with Staging Ground following its public launch. We have a lot of work ahead of us, especially as we seek to expand Staging Ground to cover the needs of as many question-askers as possible. We will be actively looking for feedback as we iterate in the coming months. Until then, stay tuned for more updates as we approach the launch date.

3
  • 18
    Good to see that things are finally progressing on this again; I'm sure the community will be more than appreciative on this. Hope that this does arrive sooner rather than later, it has been sorely missed, and I look forward to seeing it in its released form. If there's anything that feel feel is obviously outstanding still from the team should we post about that here now? I have noted that one of the PMs has been actively "mopping up" some of the posts there.
    – Thom A
    Commented Mar 27 at 14:21
  • 3
    Alright, that's a good news...! // And don't forget to update the Help Center (which should already have been edited in January 2024 and still mentions "Currently, the Staging Ground is in a limited beta release; only a small portion of new askers will be randomly sampled to go through the Staging Ground."), to avoid some endless discussion again... :idea:
    – chivracq
    Commented Mar 27 at 18:02
  • 3
    It would be nice to get some statistics about the performance of the staging ground, say 3 months after the launch. Something like what you used to evaluate the performance of it. A comparison how questions fare that have been acted upon in the SG versus those that haven't, what percentage of questions could be processed. That would be interesting. Commented Mar 28 at 19:04

6 Answers 6

72

First, great. I'm happy to see this come back. As a beta reviewer, I don't think I've ever felt more empowered to tailor a new question into a successful one.

There were a few standout points from when the SG's last beta round ended that developers for the SG recognized needed some thinking through. I made an experience report post on the Teams instance for beta reviewers and identified a number of them, and I also made a "From the Asker's POV" post there, which identified some potential pain points for askers. I'd like to resurface these problems as well as some other feedback and ask if you guys have come up with solutions for these areas.

I'm going to focus on the reviewer portions first, then the Asker's POV. I'm going to put a lot more effort into the Asker's POV feedback because I think that, if there's anything we need to tailor the Staging Ground to, it's the way the asker's experience feels.

Scale

Okay... Let's get the big one out of the way. I'm not going to mince words: The beta reviewers got burnt out. This lead to questions entering the Staging Ground, sitting in privacy for 24 hours, and then automatically graduating. That sucks for a question author.

What considerations have you guys made about scale? Stack Overflow already suffers from a lack of reviewer activity, how can we improve on this for the Staging Ground?

"I fixed it!" questions

When reviewing in the SG, I would semi-frequently come into a situation where I'd give some feedback about a question and submit it as requiring major changes. In response, the question asker would leave a comment and say that they found the solution (and, sometimes, even edit that solution into the question!)

This is a very tricky scenario. What sorts of solutions do we have for this? On the main site, I would edit the answer content out of the question body (if applicable) and encourage the user to post their solution as an answer (or, just flag/vote for closure if the question fits closure criteria). In the SG, that isn't an option, and I'm not going to hit the "Approve and publish" button just because the user found an answer... There are still issues with the question that could lead to it being closed on the main site!

Edits apply immediately at 500 reputation in the SG. Is that alright?

Anyone who reviews in the SG can have their edits applied immediately in the Staging Ground. Now... I personally love this. I'd honestly review on SO a lot more if I had full edit privileges on the condition that I have to be in the review queues to use them (hence why I reviewed in the Staging Ground so much), but I absolutely do recognize the danger that this can pose to a new user's question-asking experience if their question is edited by a new and potentially inexperienced user.

Has thought been put into the ramifications for allowing newer users full edit privileges when handling new questions? I'm not suggesting that you should change this, I'm merely bringing it up as a potential concern. The full edit privilege is available at a somewhat-high 2,000 reputation for a reason: It can be a bit destructive in the wrong hands.

Closure: Available at 500 rep and threshold is at 2 votes

The ability to vote to close is available to all SG reviewers and the closure threshold is at 2 votes (gold badge-wielders (when closing as a duplicate for their related tag badge) and moderators make it 1, still). This is quite the reduction in requirement for closure. That said... Reopening a question is quite easy for a question asker, as editing and submitting their question for re-evaluation will reopen it. (Don't worry though, reviewers: The option to decline re-evaluation will re-close it)

Is this reduced rep requirement and reduced close vote threshold something we're comfortable continuing with?

Asker's POV: "Major changes" review = comment replies instead of editing

This is taken from my "Askers POV" post. When an asker receives a "This question requires major changes before being published" review, they receive a notification. Clicking that notification (or following the link in the e-mail), they're taken to the reviewer's comment rather than prominently shown the feedback at the top of the post.

gif showcasing following a notification to the SG popping the user down to the comment

This causes a situation where focus is diverted from the top panel, which tells the user to edit their question in response to feedback, down to the comments section, where any reasonable person would assume they'd need to respond to that feedback with a reply, which is precisely what we don't want. I cannot emphasize enough how important it is to encourage users to edit their question instead of replying to reviewer comments, because it happens all the time (example is from when I was writing the "Askers POV" post).

Have some considerations been made to add some guardrails against replying to reviewer feedback in the form of comments, and instead encourage them to edit their question? My recommendation was to add a panel or tooltip of some kind when a question author begins writing a comment or comment-reply on a question that is in any state other than New or Re-evaluate.

Asker's POV: Could we get some more handholding?

There are a number of places where I think we need to hold askers' hands more.

The first point is right when you start asking a question that's going to be funneled into the Staging Ground. You get this fancy diagram:

image of 3-step Staging Ground lifecycle shown to asker

That's great, but it crucially ignores an important step (y'know... The whole reason the Staging Ground exists?) - Responding to feedback! It doesn't mention that reviewers can ask for you to make changes and in order for you to get your question posted publicly, you've gotta make adjustments! My recommendation was to add an extra step:

Same diagram but with an extra step explaining that you may need to respond to feedback

Furthermore, when an asker posts their question, they're not given any information about what just happened.

image of newly posted SG question

I know, I know... We just told them what was going to happen, but I think we can do better. Could we add a panel at the top, reusing the ones you get when you receive "Major changes" feedback?

example new panel explaining that their question is now under review

This does a lot for us:

  • It explains what's just happened
  • Explains what happens next
  • It establishes that this space is where new information about the state of their question will be held

Lastly, when a question asker edits their question in response to a "Major changes required" review, they receive the following options when they go to save their edits:

save changes/submit for reeval/cancel

This is confusing, because hitting "Save changes" doesn't prompt the user to submit for re-eval. They'd need to open the edit page again to be able to do so. Further, if you hit "Save changes", that helpful panel at the top is gone. Yaakov Ellis (a previous developer here at Stack Overflow who we do dearly miss) mentioned in a response to my experience report that the recommendation to simply have a "Save changes" button with the in-your-face option to submit for re-evaluation was under review (ticket number 794). Did anything come of that?


In all, I'm still quite happy to see that you guys are finally coming back around to new user onboarding and reviving the Staging Ground. I'd also like to take a second, as I do whenever I can, and sing the praises of all the developers (whether they're still working with Stack Overflow or not) who were involved with the Staging Ground project. Their constant response to our beta reviewer feedback was amazing. It often happened that bugs/feature requests beta reviewers gave were immediately responded to by developers, ticketed, and implemented within a couple days. That's fantastic, and I'd love to see more of it with future adjustments to the site.

17
  • 5
    "gold badge-wielders and moderators make it 1, still" - Users with a gold tag badge can only single-handedly close questions as duplicates, not for other reasons.
    – dan1st
    Commented Mar 27 at 17:11
  • 23
    Thank you so much for your feedback I am taking notes and we will be looking at each of these points internally.
    – Bella_Blue StaffMod
    Commented Mar 27 at 17:24
  • 3
    "This is a very tricky scenario. What sorts of solutions do we have for this?" Reject the question. If OP did something other than fix the problems reviewers identified, rejection is the appropriate response.
    – TylerH
    Commented Mar 27 at 18:14
  • 6
    @TylerH In some cases, the OP might want to self-answer the question. When I came across this situation, I typically commented saying I would publish it if they wanted to self-answer it and use the Request Major Changes to ensure it isn't published. If the OP wants to self-answer, they can.
    – dan1st
    Commented Mar 27 at 18:58
  • 2
    @dan1stmightbehappyagain OP wanting to self-answer is a red herring. OP is allowed to self-answer on-topic questions. Spevacus' scenario is about where OP has an off-topic question and wants to answer it instead of fixing it so that it is on-topic. The solution when your SG question needs revisions is to fix the specified issues, not post an answer (especially not as an edit to the question).
    – TylerH
    Commented Mar 27 at 19:06
  • 3
    @TylerH They said it would be in Requires Major Changes which does not mean off-topic at all. If I remember correctly, most of the questions with an I fixed it comment were in Requires Major Changes and the questions were typically at least somewhat ok (though there were some missing details in which case I typically indicated that in my comment as well - e.g. "if you want to self-answer, please fix XYZ").
    – dan1st
    Commented Mar 27 at 19:07
  • @dan1stmightbehappyagain Here "off-topic" means "close worthy" (that language is used because that's what the closure banner says, even if the topic of the question is about programming. "needs focus", "opinion-based", "needs details" etc. are all labeled as "off-topic"), not that the question is not about programming as defined by the help center.
    – TylerH
    Commented Mar 27 at 19:08
  • 1
    @TylerH If I think a post needs details and the OP says they'd have fixed it, I'd tell them what they need to change. In most cases, the OP doesn't actually want to write a self-answer but just says they would have fixed the problem in the comments (in my experience).
    – dan1st
    Commented Mar 27 at 19:11
  • 2
    @dan1stmightbehappyagain I don't see how that situation is relevant. Again, Spevacus' scenario is when they didn't fix the issues raised by the reviewer, but claimed they did. The appropriate response there is to say "no, this isn't fixed." in whatever mechanism the system allows.
    – TylerH
    Commented Mar 27 at 19:17
  • Note that tooling for "I fixed it" situations is deferred.
    – dan1st
    Commented Mar 28 at 7:38
  • 1
    "Scale...how can we improve on this for the Staging Ground?" More badges or reputation points maybe? But I would not expect any miracles there. Commented Mar 28 at 18:53
  • 1
    Given your eloquent prose and insightful points, it's clear you'd make meaningful content editing contributions. However, five minutes in the Suggested Edits queue trenches would give some insight into why edits should remain restricted. In part, I believe the volume of low quality edits is due to the +2 rep incentive. That's not to say higher-rep users only ever submit perfect edits either. However, I would be more comfortable knowing that SG edits submitted by <2k rep users 1) do not garner rep, 2) remain subject to edit ban criteria, 3) do not immediately go live (did they during beta?).
    – L Tyrone
    Commented Apr 2 at 22:21
  • 1
    @LTyrone These edits went live during the beta but these editors did not gain any rep from (and I don't think this was ever planned).
    – dan1st
    Commented Apr 2 at 23:47
  • 4
    One small thing I noticed while reading your post: first, you say "we precisely don't want them to respond to feedback with a reply", and then your suggested new step 3 is titled "respond to feedback". Given that within our daily lives "Respond" often mean "Write a response", I'd suggest to change step 3 title into "Act upon feedback" or "Improve your question" with the text explaining where to find out what needs improvement.
    – LWChris
    Commented Apr 11 at 11:17
  • 1
    @LWChris Yeah, that's a great clarifying change to make. Absolutely agreed.
    – Spevacus
    Commented Apr 11 at 14:02
27

How is AI going to be "leverage[d]" to scale this to all askers?

Staging Ground - a space for new askers

In 2022, we tested a concept in beta called Staging Ground—a sheltered environment where new askers could get help from experienced users to iterate on and improve their questions before posting to the broader community. This test was received positively by new askers and reviewers alike. We saw higher rates of successful questions making it to the platform. Askers felt supported and welcomed by more experienced users as they learned the ropes of how to ask a good question. Reviewers were happy to see their contributions lead to higher-quality content on the platform. It was a win-win for everyone. However, while the initial tests were positive, it wasn’t a scalable solution. Now, we can explore how to leverage AI to scale this feature to all askers—something that wasn't possible just two years ago. We are excited that we will be revisiting this concept and believe Staging Ground is a multifaceted approach to supporting community growth and content health on the platform.

15
  • 18
    I don't see how AI is relevant at all to scaling a project like this. The only AI that has "come around" in the last two years is generative text large language models... we are not using AI to write questions (that are allowed on SO) or to review them. So... I have to say "huh?" to that blog post claim.
    – TylerH
    Commented Mar 27 at 14:29
  • 6
    Exactly. I could see it being leveraged in the form of ML powered search or semantic search... but neither didn't exist 2 years ago. Neither did grammar correcting solutions or spell checking or code linting, etc.
    – Kevin B
    Commented Mar 27 at 14:30
  • 6
    Inb4 formatting assistant v2 Commented Mar 27 at 14:36
  • 23
    The first rule of Community Manager Club is that we do not over-promise, so let me say that what I am offering here is highly speculative. But two early proposals include a better means to detect duplicate questions, and methods to surface questions to appropriate subject-matter experts. However, these examples are not exhaustive.
    – Bella_Blue StaffMod
    Commented Mar 27 at 17:22
  • 11
    So, things that definitely were possible before 2 years ago
    – Kevin B
    Commented Mar 27 at 17:25
  • 3
    @Bella_Blue Would these things use generative AI, or something else? Commented Mar 27 at 19:05
  • 6
    @KevinB To play marketer's advocate: just because the technology was known to some humans doesn't mean Stack Exchange was capable of deploying it two years ago.
    – wizzwizz4
    Commented Mar 28 at 9:41
  • @TylerH "I don't see how AI is relevant at all to scaling a project like this." You could take all those questions that got closed and later reopened and train a generative AI to produce the comments that appeared below those questions as well as the close reasons as well as the close decision. Why should AI not be able to curate (in principle)? It's not totally out of the question. Commented Mar 28 at 18:48
  • 1
    @NoDataDumpNoContribution You can do that with people, or with non-"AI" code already. The company clearly isn't interested in addressing issues with questions before they are posted, otherwise we'd have regular expressions that block posting of questions that include things like "best practices" in the title or body. More effort would have been put into the ask question wizard. Ad infinitum.
    – TylerH
    Commented Mar 28 at 18:56
  • 1
    if by AI you mean LLM's, they by design generate text based on existing text it was trained on, they don't curate content. If by AI you mean some non-existent general AI, then, well, it doesn't exist.
    – Kevin B
    Commented Mar 28 at 18:58
  • 1
    @KevinB Do you mean me? Yes LLMs could maybe be trained to give "helpful" comments. And other AI, could for example be used to classify a question (predict the final score, predict open/close) and then auto-close. This is not so complicated, it may well be just now developed by the company. We could simply wait and see what they mean by that. Commented Mar 28 at 19:01
  • 1
    generating "helpful" comments based on past comments to different questions is just silly. At best it'll generate comments that would be helpful if not for being irrelevant to the question they're posted on.
    – Kevin B
    Commented Mar 28 at 19:03
  • 2
    You're effectively describing a non-existing solution using a tool that literally can't do that. It's certainly possible some ML solution could eke out potential issues with a given post, similar to how the current ask wizard does so with rudimentary regex, but no solution that doesn't actually understand what the asker is asking for will be able to replace reviewers.
    – Kevin B
    Commented Mar 28 at 19:09
  • Well how can I do else but describe a non-existing solution. It's all about planned features. Another thing where AI could help for example is duplicate detection. We have so many recognized duplicates, using some contrastive learning it might just be possible to identify duplicates to existing questions automatically. Again, this doesn't exist yet and may never exist or it may. Let's see what the company comes up with. My general guess would be that they think they can automatize (parts of) curation. Commented Mar 28 at 20:14
  • 3
    Duplicate detection can definitely be improved by ML, but that's exactly the point i'm making... ML isn't a new thing that was invented in the past two years, they could have been using that for the past 10 to improve duplicate detection.
    – Kevin B
    Commented Mar 28 at 21:51
21

Thank you

I want to take this opportunity to thank you for doing this.

From my experience as a beta tester, the Staging Ground can greatly improve questions that would otherwise just be downvoted and closed (which results in negative experiences of askers with SO).

There has been a lot of work put into the Staging Ground (both development work and beta-testing) and it would be a shame if that would go to waste. Thanks to the development team and everyone who was making this happen within the company (e.g., by getting other people to make it a priority). I am happy I had the privilege to beta test the Staging Ground.

Getting reviewers

That being said, I do worry about one thing. The Staging Ground needs a lot of reviewers in order to keep up with all of the questions. What are the exact plans on getting sufficiently many reviewers into the Staging Ground?

You mentioned badges in your post but what badges are planned? There were discussions about engagement of reviewers/scaling e.g. in here, this post and don't forget about that one (these links are only accessible to beta testers; sorry). Are there any plans on doing something about this? If so, please tell the Community (in public) what these plans are! Can we expect anything to happen in this area?

When?

Finally, what's the planned timeline? When will the Staging Ground be activated?

12
  • 7
    We are currently in the process of finalizing our launch plan, which makes it challenging to commit to a specific release date at this moment. However, I am hopeful that we will be ready to launch in the next few months, especially as we focus on enhancing scalability. You're right to point out that scalability will be a challenge for Staging Ground, and we are likely going to have to run some experiments before we nail the final design.
    – Bella_Blue StaffMod
    Commented Mar 27 at 17:26
  • 4
    Note that when saying scalability, I am mainly talking about making sure sufficiently many reviewers are actively reviewing questions. Of course, technical scalability is important but that should be doable. Getting reviewers is still a huge open question IMO.
    – dan1st
    Commented Mar 27 at 17:28
  • 1
    @dan1st Absolutely - three angles to approach from. First is how much work there is to get done. Second is how many people are available to do that work. Third is how much work each person can reasonably do. All three are valid directions from which to approach the problem - and likely all will need some amount of attention for the solution to really click.
    – Slate StaffMod
    Commented Mar 27 at 18:11
  • 2
    I'm guessing the badges will be similar (if not identical) to the ones you already get in review queues; bronze, silver, and gold, with gold repeatable for every 1000 (or whatever number) of questions you review in the SG. In a perfect world, nearly all close vote reviewers could shift over to the SG and just review there, because the SG in a perfect world would mean orders of magnitude fewer close-worthy questions get posted. Those that need edits before posting can be told so in time, and inherently off-topic questions would never make it to the main site.
    – TylerH
    Commented Mar 27 at 18:12
  • 5
    One of the simplest strategy to increase SG reviewers would be to push SG posts to them on the questions page itself.Given it's possible to search for them currently this should be possible as well. One of things I liked the most about the SG experience was that unlike a review queue I could see the list of SG questions and pick amongst them to review. Commented Mar 27 at 18:38
  • 3
    @AbdulAzizBarkat That's an interesting idea. However, I think most people would be interested in answering the questions (and getting the rep for it) and not reviewing posts. These people would probably just use the SG to FGITW questions by preparing an answer, publishing the post and immediately posting the answer without reviewing/improving any questions. While it is OK to answer SG questions when coming across ones you are able to answer (this happens, especially since people think about the posts reviewed), specifically using the SG in order to be the fastest gun in the west is abuse IMO.
    – dan1st
    Commented Mar 27 at 19:00
  • 1
    @dan1stmightbehappyagain that was the biggest caveat I thought about this as well :( BTW I noticed you've changed your nickname :) Commented Mar 27 at 19:03
  • 1
    I changed my display name as soon as In saw the activity in Teams ;)
    – dan1st
    Commented Mar 27 at 19:04
  • 3
    "What are the exact plans on getting sufficiently many reviewers into the Staging Ground?" In my opinion there are no possible plans to get sufficiently many reviewers. It just doesn't scale in no possible scenario. But the original plan simply said that a question can only stay for a maximum time in the staging ground and then if there aren't reviewer for it, it just goes online as is. Or in other words: the staging ground is optional and can adapt to the number of available reviewers. You get what you pay for. How else could it be. Commented Mar 28 at 18:51
  • 1
    @NoDataDumpNoContribution As a beta-tester, I do think it's possible to make it so that most questions get reviewed. However, that wouldn't be easy in any way and I don't expect that to happen (though I do certainly hope so). However, even if 10% of questions get reviewed in the SG, this would be a huge win.
    – dan1st
    Commented Mar 28 at 19:25
  • 2
    Exactly, that's what I think too. 10% is better than 0% and having such a framework/toolset is definitely how high quality Q&A is meant to be. I'm skeptic about the availability of reviewers and think that unfortunately people ask too many (not well researched or thought-out) questions for the amount of available reviewers, but that is a topic for another day. This is definitely a step in the right direction. However, people should not expect wonders from it. Commented Mar 28 at 20:10
  • 1
    getting reviewers is one hurdle. Them not burning out is another.
    – Gimby
    Commented Apr 23 at 12:45
14

What will happen to the "First Questions" review queue? After all, it's pretty pointless for questions to first be reviewed in the Staging Ground and then go through "First Questions" (SG questions skipped that Review Queue during the betas).

Will it be deprecated/removed completely in favor off the Staging Ground?

Assuming it doesn't get removed/deactivated completely, if questions are automatically published due to nobody reviewing them, will these questions still go into the First Questions queue?

2
  • 16
    The plan had been that first questions would only get items that auto-graduated from SG without getting an SG review (and items that are SG-approved skip the FQ queue). Commented Mar 28 at 14:11
  • 4
    It's like a fallback chain. Question is first in staging ground, but nobody looks at it -> it goes to first questions review queue, but nobody looks at it -> it simply goes live, but nobody looks at it -> it is forgotten. Commented Mar 28 at 18:58
7

Will every question by new users go through Staging Ground? If yes, how do we get enough reviewers? If not, how will questions be selected for SG?

FWIW, during beta testing, a lot of questions went unreviewed which led to frustration for some users who expected immediate answers to their questions (I remember one comment asking how to bypass SG to get more eyes on their question).

3
  • 2
    Especially if there are already more questions in the SG that can be reviewed, it may be a good idea for some questions to skip it.
    – dan1st
    Commented Mar 28 at 8:17
  • 4
    if no reviewer gives a review that action by the asker is necessary within a given period, the item is auto-published. if such a review is left and no asker action happens in a given period, the item goes stale or something. it's described in more detail in meta.stackoverflow.com/q/423469/11107541 I think
    – starball
    Commented Mar 28 at 8:51
  • 6
    Isn't this the general problem of a Q&A platform? Somebody has to answer the questions and/or quality check them and/or improve on them. If there are no such people willing to do that work, it simply won't work. And from the other side, if I'm willing to do that work, how do I find worthwhile questions? I don't want to search for a long time to find a question I can review, improve or answer. I always had this problem, right from the beginning. Commented Mar 28 at 18:56
5

A few thoughts about how Staging Ground needs to work to be effective:

Questions need to start closed

According to every description I've heard so far, questions that sit in the Staging Ground for a certain amount of time, and aren't explicitly acted upon to close or otherwise reject them, will simply pass through onto the site as is. This is unacceptable and misses the point of having this filter. The underlying presumption of the Staging Ground is that almost everything that would be submitted, at least needs work in order to actually meet the site's standards. (Daily reminder: the large majority of existing open questions do not meet the site's standards, and the site's overall quality greatly suffers for the fact that we did not get this in check a decade or more ago.)

If staff imagine that a default-permissive attitude will make for a friendlier user experience, they are being hopelessly naive. What will happen is that a huge fraction of questions will be ignored by the "experienced community members" (after all, they have little incentive to participate, and are few in number to begin with), or receive a single bit of feedback that is then ignored by the OP. The question ends up on the site, and then immediately gets downvoted and closed by the broader community. Or worse: a poor question that should be closed instead gets answered - except this doesn't even have the tiny value that it normally would, because OP has lost interest by now.

What needs to happen instead: Hold the question indefinitely until at least one person interacts with it; at that point, treat it as closed (and not upvoted), and subject to the usual "abandoned closed" Roomba rule. That is: if OP doesn't edit or interact for 9 days, delete it.

Keeping OP in the loop

I want to echo Spevacus here. It needs to be crystal-clear from the start that using the Staging Ground is interactive and that asking a question will potentially require multiple iterations of responding to feedback and not just receiving it. But going further than that, I don't think that trying to show a linear "flowchart" here is useful, and simply adding a step to it visually is a band-aid that largely misses the point. The existing UI:

comes across to me as window dressing that's insulting the reader's intelligence. Please instead use prose to explain the concept, and also set proper expectations - including the inescapable fact that some questions are fundamentally unsuitable for the site despite being nominally on topic. This can still be done in an upbeat, "promotional" manner - something like:

So you're ready to ask a question about programming? Great! Even the simplest questions can help contribute to our library - as long as they're on topic and well asked. The community is here to help make sure you have a suitable question and to refine it as much as possible, so that everyone can benefit.

If you have a question about a different topic, click here to see the other sites we offer on the Stack Exchange network. You can also visit the Help center to read more detailed information about the Staging Ground, as well as about Stack Overflow in general.

The next form will give step by step instructions on how to write a high quality question that's clear, focused, searchable and - most importantly - properly answerable. The better your question is to start, the more likely it is to be accepted onto the site and answered.

Once you submit your question, it will be reviewed by experienced community members. As long as the question is potentially useful, they'll be able to propose improvements so that the question and its answers can be useful for everyone. Some changes may be edits that the community proposes directly, while others may require additional information from you. Please respond to feedback as quickly as possible, to have the best possible chance of getting your question published.

Even if your question is found unsuitable, you might still get help in the form of someone pointing out a typo in the code, or an existing duplicate Q&A that answers your question.

Don't use comments!

The comment UI is far too clumsy for giving the kind of detailed feedback that most people need. It also leaves a massive clean-up headache, unless you're committed to automatically removing all comments when publishing a question out of the Staging Ground. (Seriously, the bulk of blatantly NLN comments on popular old questions is beyond absurd to begin with.)

To communicate properly about fixing questions, the OP and the "community experts" really need to be using chat, in nearly every case. (More than that, they need a fully-powered version of chat, that actually lets them workshop edits to the question and view them side-by-side with the chat going on.) Real-time interaction will keep the OP engaged - and allow for interjection when necessary (e.g. "please stop trying to post the code here; edit the question instead"). Finally, using chat means that the comment section on the posted question is clear, but there's still a reviewable transcript.

Aside from that, it needs to be much easier to highlight issues using canned comments as previously discussed. These should be templates, so that important details can be filled in; and they need to be far more comprehensive than simply generating one per standard close reason - as I already explained in my answer there.

This is training, not just for the OP

It's become abundantly clear that a huge fraction of long-time users don't actually understand the site at all, and are in fact actively incentivized to do things that are harmful to the site in the long run. The Staging Ground would be a massive help with training such users to close questions, to recognize issues with questions (i.e., what we consider to be issues) and just to have some expletive-deleted standards in general.

It would also mean that when people with the FGITW instinct get to see published questions (where the site permits answers), they're high enough quality on average that those answers aren't harming the site merely by existing. Heck, we could even incentivize them to participate in the Staging Ground by making it easier for them to get the first crack at those questions (by streamlining the UI, I mean, not by any explicit lock-out or anything).

Of course, this entails that there are systems in place to review Staging Ground feedback and approval actions. If there's one user who's constantly approving questions where others consistently find issues, that needs to be detected and handled.

It also entails that people need to feel empowered to help out with the Staging Ground. Anyone who wants to put on the "curator" hat needs an easy way to get access to multiple Staging Ground questions and choose the ones where they're best able to provide feedback (typically, due to having subject matter expertise). The queues don't work. The concept is flawed; the underlying principles of queueing theory would only make queues a good idea if a) every question needed to be deterministically handled in a "fair" amount of time and b) everyone reviewing could be treated as interchangeable. Neither is true.

10
  • re "starting closed", the design tradeoff is how long the user has to wait for their question to get published, which there was a goal to keep low. if no "needs changes" reviews are given after a period of time, the post auto-publishes (and I think can go into the "first questions" review queue). if a "needs changes" review is given and the asker doesn't do anything about it, after some time, the item goes into a stale state and doesn't publish, and no longer shows up for SG reviewers (or something along those lines). I think it's a fair tradeoff and has semblance to "default closed".
    – starball
    Commented Apr 26 at 20:13
  • 4
    I don't think it's fair to the asker for their question to get deleted in SG just because nobody reviews it.
    – starball
    Commented Apr 26 at 20:15
  • 1
    comments in SG do not get carried over to main. (re: "unless you're committed to automatically removing all comments when publishing a question out of the Staging Ground"). "canned comments" in SG are templates (re: "These should be templates")
    – starball
    Commented Apr 26 at 20:20
  • re: "If there's one user who's constantly approving", I asked about this during SG beta and Yaakov said "we have plans for mod-only sections of stats that will highlight outliers in terms of successful and open rate for exactly this purpose". I don't know whether that ended up getting implemented.
    – starball
    Commented Apr 26 at 20:23
  • re: "choose the ones where they're best able to provide feedback" there are tag filters and status filters in the SG search interface. re: "deterministically handled in a "fair" amount of time", that's what auto-publish and auto-stale are for.
    – starball
    Commented Apr 26 at 20:24
  • given recent new feature launches, i wouldn't expect the mod stats to exist for the first year or two
    – Kevin B
    Commented Apr 26 at 20:28
  • 1
    "I don't think it's fair to the asker for their question to get deleted in SG just because nobody reviews it." - hence, "hold indefinitely until at least one person interacts with it". Thanks for additional information about how SG currently works. Commented Apr 26 at 20:32
  • I strongly disagree with the idea of posts being "closed by default". There will likely be posts nobody looks at and it's better if these posts are published - it us not acceptable if a potentially good post is kept from being published just because nobody had the time to review it.
    – dan1st
    Commented Apr 26 at 21:17
  • For "Please instead use prose to explain the concept, and also set proper expectations": People don't read. Regarding comments: This worked fairly well during the beta IMO and yes, they are intended to be templates (and the user needs to first fill out the template and then submit the comment so it actually is a template. I don't think that chat is needed here at all (and I do actually have a user script adding my own templates)
    – dan1st
    Commented Apr 26 at 21:17
  • Regarding "Anyone who wants to put on the "curator" hat needs an easy way to get access to multiple Staging Ground questions and choose the ones where they're best able to provide feedback (typically, due to having subject matter expertise).": The main page of the SG is a list of questions where the reviewer can choose from (but they also get a new question after submitting a review which is perfectly fine because reviewers don't need to be SMEs).
    – dan1st
    Commented Apr 26 at 21:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .