50

We have previously mentioned our plans for a new section on the site called the Staging Ground, which is aimed at improving the new user onboarding experience and increasing the quality of first questions. We have also given an overview of the extensive research behind, and motivations for, this initiative.

This post is the first of four posts that will be going through the proposed workflow for the Staging Ground in detail. The topics of the four posts are:

  1. Staging Ground Workflow: Question Details & Actions (this post)

    • General introduction and main operating principles
    • Question page
    • Commenting overview
    • Actions & related workflow
  2. Staging Ground Workflow: Listings, Filters, and Notifications

    • Question listings in the Staging Ground
    • Volume and quality control
    • Publicity of the feature on the site
    • Notifications and Emails
  3. Staging Ground: Reviewer Motivation, Scaling, and Open Questions

    • Exploring Reviewer motivation
    • Scaling concerns
    • Talk about other open questions
  4. Staging Ground on the Network (an MSE post opening up a discussion about how the Staging Ground could be effective on other sites on the Stack Exchange network)

There is a lot to cover here. In order to keep things organized, we ask that questions and comments be posted on the relevant post (so if you have concerns about scaling, please wait until the third post, etc).

So without further ado…

TLDR: What is the Staging Ground?

  • The Staging Ground will be a new limited-access area on the site where first-time question Authors can interact with experienced users and receive guidance towards improving the quality of their questions before they go live.

  • The inspiration for the name is that of a staging environment, commonly used in developer parlance to denote a limited-access area where code can be reviewed and tested before it goes public.

  • The general goals of the Staging Ground will be to:

    • Address quality issues that exist with first questions by lowering the close/deletion rates and improve their overall quality
    • Take the pressure off of the First questions queue
    • Improve the question-asking experience for new users by creating a lower-pressure environment where users can get more guidance related to asking questions. Encouraging new users to iterate on their questions in a more collaborative way should improve their overall experience and increase the likelihood and quality of future contributions to the site.
    • Our exact test criteria will be discussed more in a post immediately before the test goes live.
  • Our initial plan is to build out an MVP (Minimum Viable Product) and release it in a controlled A/B test on Stack Overflow, after which we will evaluate feedback and results, and make a determination for future iterations and whether it will graduate (to both Stack Overflow as well as to other network sites).

  • We do not yet have a date for when this test will begin, though based on planning, it will probably be sometime during Summer (June–August) 2022.

  • Qualifications:

    • For questions: same qualifications as the First questions review queue (the Author’s first question on the site, or their second/third question if their first question(s) didn’t do well).
    • For Reviewers (this is how we are referring now to the experienced users who will be able to review questions and provide guidance in the Staging Ground): same qualifications as review queues (at least 500 rep).
    • We haven't yet decided what percentage of eligible First question Authors and Reviewers will be included in the initial A/B test. We may start small and increase participation as the test proceeds, with the goal of trying to ensure a good experience for both new Authors and Reviewers.
  • Comments and questions will be "private" (only open to those who have access to the Staging Ground as Reviewers). There will not be any votes or answering. Search engine indexing will be blocked.

  • While closing and flagging will be offered as options (more info on that in the next post), Reviewers will be encouraged to find ways when possible to offer advice to users on how to improve their questions, using a structured workflow to help guide the process and allow for efficient use by Reviewers.

  • Questions can be published on the site from the Staging Ground right away if they are approved by Reviewers (as described below). Questions that are not approved by Reviewers will be auto-published after a defined period of inactivity, except for questions that have received a close vote or flag, or where a Reviewer has asked for major changes that were not made (this will be covered more in the second post)..

  • While it will bear similarities to the First questions queue (and will of course overlap in purpose), the Staging Ground is not another review queue. It will function differently in a number of key ways (as will be highlighted in this and the next post).

(Note: final copy on all screenshots in this post may differ from what you see here.)

Entry into the Staging Ground & General Structure

Ask Wizard

All questions will enter the Staging Ground through the new Ask Wizard, after which they will be created as new Staging Ground questions with their state set to Pending review (more on Staging Ground question states in the next workflow post).

Some refinements to the Wizard for when it is being used as the feeder for the Staging Ground (in addition to how it is operating today):

  1. A different notice at the top of the screen letting the user know about the process that they are entering into:

    "Writing a good question notice" that will appear in the Ask Wizard. Subheader: "You’re ready to ask your first programming-related question and the community is here to help! Looking to ask a non-programming question? See all topics here to ask on a different site." Includes three steps: "Propose a question", "In review", and "Approved".

  2. When users click on a similar-question suggestion from the wizard and the potential duplicate loads, this notice will be shown at the top – giving the user a clear path forward for the case where they have found an answer to their question, and giving us a positive signal that this has occurred as well:

    "Does this post answer your question?" notice, with more explanation and options for "Yes, this answers my question" and "No, return to asking a question".

    We are hoping to apply this interaction on all uses of the duplicate posts suggestion box (not only in the Ask Wizard). We will post about this separately closer to the release of this specific feature.

  3. Before submitting the question, the Author will be able to provide a "private" comment that will be added as a comment to the question after it is created in the Staging Ground. This can be used to provide more context or ask questions of Reviewers who will be assisting.

Welcome Notices

Authors seeing their question in the Staging Ground will be shown a similar notice to the one shown above (highlighting step 2 of the process). Meanwhile, Reviewers will see the following notice on their first visit:

"Reviewing proposed questions" notice shown to new Reviewers. Includes "What and why?", "How?", and demarcates sections for "Leave feedback", "Use filters", and "Skip questions". Also includes a Review opt out.

With this notice, we want to set a tone and expectation for Reviewers that the Staging Ground is different from review queues in its focus on leaving feedback and guidance for users, and in its structure (which we will talk about more in the next post).

The Question View

Questions will be displayed as follows:

View of question body in the Staging Ground

Unlike regular published questions, in the Staging Ground we will not include any voting, views, bookmarks, or answers. This should help first-time askers to feel more comfortable, and Reviewers to be able to focus on things like edits and comments that can better teach/inform Authors. Users will still be able to edit, close, delete, flag, or follow the question as normal. The question’s revision history and timeline will be available. We will talk about the ways that closing and flagging will work on the Staging Ground in the next post.

Commenting

Unlike on the main sites, where the purpose of comments is to serve as “temporary Post-It notes”, comments on the Staging Ground are the main venue of interaction between Reviewers and question Authors — the place where Authors can ask questions about how to improve the question itself and Reviewers can provide guidance and feedback.

As such, we are expanding comment functionality from what is offered on the regular site, in order to help facilitate more helpful interactions:

New comments layout for Staging Ground

Some things to take note of:

  1. Nested Comments (!). We are really excited to try this out here, with the hopes that it can help to make the conversations easier to navigate, especially when multiple points are being discussed. We are going to start out with only one level of nesting as we explore this dynamic.

  2. All top-level comments (and nested comments within their own thread) will sort chronologically; all comments will load by default.

  3. There will be comment flagging. We have not made a final decision yet about voting on comments.

  4. New UI, which includes profile images and a separation of the user name and comment date from the actual text of the comment. We think that this will help to humanize the interaction between Reviewers and Authors and promote a more collaborative environment.

  5. When comments are left relating to specific Actions (see below), these will be noted in the comment line itself (see the “Minor edits requested” label above).

(To preempt your questions: nested comments is something that we have been thinking about for a while, and we would love to see them someday on the main site. This platform will serve as our first foray into this area, and this control might eventually be modified such that it can also be used for published posts. However, that will need to be a separate project, as comments on published posts have a number of additional requirements that we will need to address in order to allow them to function optimally.)

Though not shown here, there will also be a number of Canned Comments that Reviewers will be able to access and use as the base for the comments that they leave. These will operate similar to the Share feedback option on the First questions/answers review queues, except that here there will be no anonymous commenting, and selecting a Canned Comment will just populate the text in the comment box (always leaving the user with the option to modify what it says). We are still working on the comment types and text for this feature.

Actions

A number of Actions will be available for Reviewers on the question detail page. These will be laid out similarly to Actions in review queues, but with some big differences in the workflow that proceeds from different Actions.

Actions menu

The Actions and their resultant workflows are as follows:

  • Good to go: Indicates that the question can be published as-is. When this Action is taken (by a single Reviewer), the question will be published right away on the site.

  • Suggest minor edits: Indicates that the question looks good, but requires small changes to be made before it can be published. The Author will be notified/emailed, and any subsequent edit made by the Author will result in the question being published right away. The UI will require that a comment be left by the Reviewer (explaining what edits are needed) when selecting this Action (canned comments can be used here).

  • Suggest larger changes: Indicates that the question requires significant edits and changes before it goes live. The Author will be notified/emailed, and any subsequent edit made by the Author will result in the question being put into Pending re-review status in the Staging Ground (notifying the Reviewer to follow-up). The UI will require that a comment be left by the Reviewer (explaining what edits are needed) when selecting this Action (canned comments can be used here).

  • Edit post yourself: Allows the Reviewer to perform edits (this also applies to users who do not have full edit privileges but still qualify as Reviewers in the Staging Ground). When they are ready to save the edit, they will be provided with the same list of Action options to perform (minus the Edit option, of course). So a Reviewer can edit the post, and then approve it, ask for subsequent minor edits, or ask for larger changes.

  • Also add an upvote when published: When this is checked, it will note a provisional upvote by the Reviewer on the post. This means that when the post is published, it will receive an upvote right away by the Reviewer. If the Reviewer found the question here to be worthy of an upvote, we want to facilitate making sure that that upvote can be applied once the post is published.

When an action is submitted or the current post is skipped, the Reviewer will be redirected to a new Staging Ground item that is Pending review (or an item that they have already interacted with that is Pending re-review). The Return to listing button will return the Reviewer to the main Staging Ground listing, where they can select another post to review. We will talk more about how these will work in the next post.

If you would like to see how the page will look for Reviewers with all of the components at once, go here (I’m leaving the full image out here to save space).

Author view & notifications

The Author view of Staging Ground questions will be very similar to the Reviewer view. The Author will not have access to Canned Comments or Actions.

There will be a special Call to Action notice at the top of the page that will be displayed whenever the Author loads the page:

Call to action notice for Asker. Heading: "Changes have been suggested". Includes instructions for how to proceed, highlights recent comments, links to revision history, and gives a link to edit the question.

This notice will list any Actions taken or comments left by Reviewers since the last edit or comment by the Author. If any edits were made, a link to the revision history will be given. And if the question is in a state that requires an edit from the Author in order to proceed, a link to Edit your question will be provided as well. The goal will be to make it easy for the Reviewer to focus on the newest feedback that was given to them, with easy access provided for next steps.

Any Action by a Reviewer will create a notification for the Author. If the notification isn’t seen in a very short window (5 minutes), an email will go out to the Author. The exact text and format of the email is not yet defined, but its content will resemble the Call to Action notice mentioned in the previous section. We will go into more detail in the next post.

Post history and going live

Staging Ground questions will be stored in our database as Posts (with PostTypeId=1, for those of you familiar with our schema). However, they will have different PostState values to indicate that they belong in the Staging Ground (and what their state is in the Staging Ground). Whenever an Action is performed on the post, a new PostHistory entry will be recorded on the post to give an audit trail of who performed what Action. These will be available in the timeline of the post within the Staging Ground.

Staging Ground questions are published if one of the following conditions is met:

  1. The Good to go Action is selected by the Reviewer.
  2. The Author made any edit in response to the Suggest minor edits Action.
  3. The question is auto-published after a certain amount of time (to be determined) has elapsed with no Review or Author activity on the post.
    • This will only happen if the question (a) does not have any pending close votes/flags and (b) is not in a state where larger changes were requested by a Reviewer and the question was not subsequently re-reviewed.
    • If it is auto-published in this way, the newly published question will be added to the First questions queue upon creation.

When a Staging Ground question is published, the new question will use the final Title, Body, and Tags of the Staging Ground question, and will list the original Author of the Staging Ground question as the Author of the post.

The Staging Ground is designed to have big differentiations from the public site in order to foster an environment with lower stakes, where Reviewers will feel less of an urgency to respond (with downvotes, comments, and edits) in manners that can at times add pressure upon new users. As such, neither revision history records nor comments from Staging Ground questions will be transferred to the newly published questions. The only indicator that posts originated in the Staging Ground will be a PostHistory entry relating the newly published post back to the original Staging Ground post, which will only be visible in the Timeline to Reviewers.


As was noted above, we are still in the early stages of this project and are still several months away from being able to commence with testing. We are happy to receive questions and feedback below about the content of this post. Answers are preferred to comments.

As we noted at the top, we have a few further posts planned going through the workflow in more detail, through open questions, and through issues relating to the Stack Exchange network as a whole. If you have questions or thoughts relating to those areas, we request that you save them for the relevant post.

28
  • 17
    Now we are talking! Cautiously optimistic about that. A nitpick, though: the sample post is quite horrible: lack of proper formatting, it is a mega-dupe, it is not well-written as well, however, the sample (I hope) comments below are lukewarm and praising the author - can we please use something at least of decent quality as a sample?
    – 0Valt
    Commented Mar 28, 2022 at 10:03
  • 9
    On a positive note - can we please have comment replies everywhere some time? :)
    – 0Valt
    Commented Mar 28, 2022 at 10:05
  • 12
    @OlegValteriswithUkraine nitpick noted. And status-declined. I have better things to task our designers with than redoing the question that they used in our prototypes. The point here is to see how things look. If it is possible, please try not to get caught up on those particular details (your feedback on all the other details is welcome, of course).
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 10:20
  • 9
    Will reviewers be able to filter to certain tags like they do in the current review queues? Would it support Wildcards if so (this would be ideal), which reviews don't? Also, will there be any repercussions to users that, for example, vote the question as "Good to Go", and the question is then shortly after closed and downvoted because it isn't good?
    – Thom A
    Commented Mar 28, 2022 at 10:38
  • 19
    You say (with emphasis) that, "the Staging Ground is not another review queue". However, in your "go here" screenshot, that's exactly what it appears to be - even the "16/20 daily reviews" progress bar near the top-left. Maybe I need to wait for the next post, but how/why is this not another review queue? Commented Mar 28, 2022 at 10:52
  • 6
    @AdrianMole that part of the prototype was copy/pasted from review queues, and can definitely change. The second workflow post (that I am writing right now) will highlight more differentiators from review queues.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 10:53
  • 7
    @Larnu there will be lots of filtering options, much more so than review queues. This will be detailed in the second workflow post.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 10:53
  • 7
    @Larnu "will there be any repercussions to users that, for example, vote the question as "Good to Go", and the question is then shortly after closed and downvoted because it isn't good?" that is the type of thing that we can explore more after the initial test, when we need to make decisions about these types of improvements, quality enforcement issues, suspensions, etc. Good point though.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 10:55
  • 5
    @AdrianMole yes, badges are in the planning, exact breakdowns TBD. And potentially more (trying to think about non-badge rewards that can be given - will explore this more in the third post planned above).
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 10:55
  • 4
    @OlegValteriswithUkraine the magnitude of changes requested could be different (major vs minor, still need to work out how the copy differentiates). Also, there could be no "changes requested" (no action was made) but comments were left for you, in which case the notice at the top wouldn't give you any action guidance (someone wants you to edit, click here), but would still highlight the newest comments that were left. The notice at the top can also highlight if someone suggested a duplicate post.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:21
  • 21
    (Wow, threaded comments on this post would be so nice.)
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:22
  • 6
    I have this feeling that opening the SG to the same users who have access to the First Questions queue is not going to be a significant improvement over... the First Questions queue. If a new question suffers from issues that anyone can point out, it's probably unsalvageable. The big differentiator IMO is feedback from tag experts (esp. w.r.t. debug questions or dupes) who are those who'll be answering it. Hence I'd see this better restricted to bronze tag badge holders, but this would even further reduce the reviewers pool, which brings us back to the root issue: lack of reviewers.
    – blackgreen Mod
    Commented Mar 28, 2022 at 11:24
  • 4
    @blackgreen "If a new question suffers from issues that anyone can point out, it's probably unsalvageable" There is another possibility: there might be many first questions that would be closed in a review queue, while in a place like the Staging Ground could receive feedback that can lead to a question that is of higher quality. I dont agree that all questions where folks can point out issues are unsalvageable.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:29
  • 4
    Well FINALLY. "How about we put new users' questions in a different place so we don't have to put up with so much garbage while they learn how to ask", i.e. tiered SO is one of those proposals that come up now and then and is invariably and consistently downvoted to death. Let's see how this new variation of the same concept plays out.
    – walen
    Commented Mar 29, 2022 at 18:13
  • 6
    @walen I disagree that this is a variation of a "tiered SO". The most significant flaw in those proposals (that gets brought up again and again) is that it segregates newbies from experts in some fashion, to the detriment of both. The system outlined here does nothing of the sort– the sandbox doesn't answer questions, it only helps refine them before sending them to the main site. This means that all questions still always get answered on the same old main site, while (hopefully) helping newbie askers produce better questions, which ultimately benefits newbies and experts alike.
    – zcoop98
    Commented Mar 29, 2022 at 19:37

22 Answers 22

51

Please, at least let us upvote comments.

There is no point in anyone writing a second comment that communicates exactly the same thing as an existing comment.

+1 to what @Dharman said. You are wide open to SQL injection.

Otherwise, how else will the question author know that multiple community members feel the same way?

16
  • 19
    Would downvoting comments make sense as well? E.g. if someone makes a suggestion that other reviewers disagree with? Commented Mar 28, 2022 at 11:56
  • 4
    Yes, this is our main consideration for adding comment voting.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:57
  • 1
    @YaakovEllis Ah, I guess I misunderstood. If the plan is to definitely allow upvoting and maybe allow downvoting comments, then if you add that clarification to the question, I can delete this answer. Commented Mar 28, 2022 at 11:58
  • 4
    @IanCampbell the plan is to probably allow comment upvoting (still not clear if this will fit in the roadmap for the initial release or will have to wait for later). No plans at all at the moment for downvoting on comments (a case for this could be made in a different answer). No need to delete the answer.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 12:00
  • 7
    @MisterMiyagi "Would downvoting comments make sense as well?" Just adding another comment saying the opposite and then upvoting that other comment, is effectively very similar to that. Commented Mar 28, 2022 at 15:45
  • I don't see much use in being able to upvote comments by reviewers in this view since it only takes one reviewer to approve it. Either everyone's criticism has been addressed in the post edits, and it gets approved, or it hasn't, and you can just leave a comment saying "this still needs to be addressed". An upvote will not convey anything other than "I agree with this comment" which is unnecessary.
    – TylerH
    Commented Mar 28, 2022 at 19:00
  • 5
    @TylerH I disagree that conveying agreement is unnecessary. From the screenshot above, we can see that the company expects multiple reviewers to be interacting with the same question. If you get 5 recommendations to fix your question and 3 of those 5 things are upvoted 4 times, I suspect you're going to focus your attention on those 3 suggestions. Likewise, reviewers may be less likely to hit the "good to go" button if highly upvoted recommendations remain unaddressed. Commented Mar 28, 2022 at 19:06
  • 1
    @IanCampbell Nested comments covers the ability to add your thought to an existing remark far better than an upvote does. It might be good to have a 'completed' option to click on comments here since there specifically/exclusively for getting a question into shape. RE: focusing on 3 of 5 comments, that's actually bad because we don't want users to ignore those other two comments. Any comment that is posted ought to be acted upon here, based on how I see the process working.
    – TylerH
    Commented Mar 28, 2022 at 19:19
  • 3
    @TylerH If we could get all reviewers to agree and follow those recommendations, I agree that it could work that way. But I guess I'm more cynical of how easily people might push that "Good to go" button. Commented Mar 28, 2022 at 19:21
  • 3
    @TylerH You have a point that if comments need to be addressed, upvoting doesn't increase that requirement. But what if the OP gets comments that pull in different directions? Wouldn't it be nice to know that one direction is more popular?
    – Lawrence
    Commented Mar 29, 2022 at 17:46
  • @Lawrence Consider a few things: it only takes one reviewer to 'approve' a SG question, and SG questions have comments removed once they leave the SG. There's not really a benefit to know that one direction is "more popular", because "popularity" isn't the right thing to consider when we are talking about questions that have substantive or formative problems. These aren't answers of a different flavor that people can choose between based on a preference.
    – TylerH
    Commented Mar 30, 2022 at 18:32
  • 1
    @Lawrence If you think Reviewer A's comment is good, and should be acted upon, your course of action is: do nothing. Just skip the question and move onto the next one.. OP will see the comment eventually and have an opportunity to address it. If you think Reviewer A missed something/you have a separate concern with the question altogether, post a separate comment at the top level. If you think Reviewer A's comment is wrong or bad for some reason, leave a comment reply under Reviewer A's comment. None of these workflows need or benefit from comment voting.
    – TylerH
    Commented Mar 30, 2022 at 18:35
  • 2
    @TylerH I get your point - comments are visible to the OP regardless of votes - but that's not what I'm talking about. Suppose someone asked a question containing some spaghetti code. One comment is "Tighten the code: 5-lines test", and another is "Ditch the code. Draw a UML interaction diagram". If the OP could decide between the two, they'd probably not need to ask the question in the first place. But if they saw "5-lines" with +3 votes and "UML" +15, it would be a useful signal as to which way would communicate their problem/question better.
    – Lawrence
    Commented Mar 31, 2022 at 18:27
  • @Lawrence That would be a situation where educating the reviewers is necessary, because those are contradictory pieces of advice. The person who gave the second piece of advice (to draw a UML diagram) should have replied under the first, instead, and explained why they should do that instead of applying whatever the '5-lines test' is. Explanations and discussion are far more useful for OP than upvotes, especially because votes are not mutually exclusive (what happens if I am a 3rd reviewer in that scenario and I decide to upvote both comments, or neither?).
    – TylerH
    Commented Mar 31, 2022 at 20:26
  • 2
    @TylerH Sure, you can propose a methodology for reviewing, and a series of exchanges can provide insight into what best to change. I'm not arguing against that. My point is that there is a place for voting in review comments. It's the long-standing Stack Exchange way of compactly saying, "This comment is useful".
    – Lawrence
    Commented Apr 1, 2022 at 5:25
40

The "Actions" dialog is only following the happy-path

After years and years of seeing this pattern repeated in some form or other, I'm not sure if the SO staff tasked with desiging this stuff is just that naive, or if they're actively ignoring the fact that there are questions that are simply not suitable for SO. Yet again, you're not accounting for the possibility that the question might simply be off topic and not salvageable (e.g. recommendation questions).

As evident by the triage queue, users are more likely to choose an existing action than simply disengaging from bad content. They are presented with a prominent list of actions, and naturally they will gravitate to choosing from that list. The reviewer welcome notice also doesn't say anything about closevoting, so you're not even briefing reviewers that closing is available. You basically bank on users finding the tiny close link and casting a close vote (or flag) without any guidance to do so, and then disengage from that post. This is bad design and an explicit option should be added to close/remove/not-post the question. It could also help with cleaning out bad content from the queue faster, which ties into my second point:

A single user is enough to push garbage onto SO

This is also another repeating pattern, yet again the actions of a single user that doesn't care about quality can invalidate (or at least frustrate) the actions of people who try to curate content.

Thus, please rethink the threshold for the "good to go" action. A few people who simply approve most bad content will make your current design worse than useless(1), as it would invalidate the positive effect of showing less garbage on SO main and waste the time of any reviewers taking their tasks seriously. I'd recommend something like a threshold of 5 users, with potentially a higher weighing for tag badge holders (e.g. counting gold badge holders as +2). This would ensure that questions even have a chance to be closed / adequately curated before someone simply presses "good to go" because they're farming a badge.

(1): If you doubt that this will be an actual problem, just look at the last few hundred suggested edit reviews. You should see quite a bad few edits ranging from just being useless to being actively harmful, which still managed to get approval votes - sometimes even enough to get accepted.

6
  • 8
    Actions path: Thanks, you make some very good points, and we are already discussing this quite actively on our project Slack room. It would be good to be able to provide a more explicit close option as a top-level action. But we also want to avoid a situation where Reviewers find it to be too easy to quickly close questions where some guidance could raise the quality to meet our site standards, provided some guidance. (And I agree with you, that some questions are irredeemable). The discussion on this will continue.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 17:14
  • 9
    Single user: we're going to stick with the single-user approval during our testing phase and then evaluate based on the usage data if this is an area that requires further consideration. It is important to us that while we are trying to achieve the other project goals, that we do not at the same time hinder posts from going public more than is necessary. Today posts go public and have the potential to be seen and answered right away. We want to be conservative in adding additional boundaries to going public. Thus our current plan (again, subject to change based on actual usage).
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 17:18
  • 15
    While I agree there's room for improvement, the drastic tone of this post is somewhat uncalled for, given that Yaakov's post above does explicitly mention (I think several times) that users can still vote to close a question from the queue. So they did not simply 'forget to account for this situation'... at best they did not afford it the same weight/importance as you would like them to have done.
    – TylerH
    Commented Mar 28, 2022 at 19:02
  • 2
    @TylerH I guess we all want the same more or less, just differ a bit on the magnitude of the problem. When Yaakov says that he agrees that some questions are irredeemable then he may be understating the magnitude a bit. We will see. But surely there is an optimum somewhere between effort that is put in and outcome of salvaged questions. Commented Mar 29, 2022 at 7:11
  • 1
    @TylerH re "tone" - if you think it's drastic, think of it as a counterpoint to SO's rose-tinted view. They might not have forgotten, they just once again chose not to address it appropriately, as has happened many times in the past decade even with specific warnings and feedback. For example, triage and the "needs improvement" fiasco - we've been arguing since day one that it was a bad design, it took more than 5 years until SO finally listened. Disregarding those lessons for SG is just bad design, especially as a single less-informed reviewer who doesn't read meta is enough screw it up.
    – l4mpi
    Commented Mar 29, 2022 at 9:02
  • 2
    @YaakovEllis I can think of many ways to resolve the close issue. For example, show a reduced set of unsalvageable "close" options (lib-rec, migration, general off-topicness) and a message that describes that the other options should be resolved with the SG specific options. Or simply show the normal close dialog but have special handling for the salvageable choices (e.g. implicitly interpret a "needs debugging" vote in SG as a "suggest large edit" action, or explicitly show a dialog after choosing that close reason which gives the user the choice to re-route to "suggest edit" instead).
    – l4mpi
    Commented Mar 29, 2022 at 9:10
30

What is your plan to deal with people answering questions in comments? I am not talking here about pointing out typos, but about potential solutions to the stated problems.

2
  • 12
    We hadn't yet considered this, but it is a good point to bring up. We probably wont include a provision for this in the first release. And then will see if it actually manifests itself as an issue, and if it does, decide on how best to address it. It could be with a new flag type (repeat violators could be banned from the SG), and/or potentially something else.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 12:40
  • 2
    Hopefully we can still flag those comments, and moderators can still delete them. Super happy to hear Yaakov say repeat violators can be kicked out of SG. Enforcement options will be needed for sure.
    – TylerH
    Commented Mar 28, 2022 at 19:06
26

On the "qualifications" required for reviewers, maybe there is here an opportunity to experiment with another aspect of curation that has been discussed (at least, in chat, if not formally on Meta.SO/Meta.SE) in recent months (notably, by Catija – I'll look for some relevant links …).

What I'm referring to is, rather than a reputation-based system, why not use previous curation activities (i.e. review history) as the criteria by which to grant the "Staging Ground Reviewer" privilege (if, indeed, we can call it a "privilege")? Or maybe some combination of the two?

You are, potentially, asking reviewers to commit significantly more time and effort into this new form of curation, and it would be unfortunate if that effort were somehow thwarted by inexperienced curators who have recently crossed a reputation threshold (which, for some, can be a very quick achievement).

There is also the problem (to which you've hinted) about what any reputation threshold should be: too high, and you won't get enough users on board; too low, and you end up clogging other review queues (e.g. a 500-rep user can't make an edit without adding to the Suggested Edits queue; a < 3,000-rep user can only offer a close flag, rather than a full-blown close vote).

I don't know how difficult such a curation-based privilege would be to implement; however, if something like that is already under consideration (even if only as a vague concept), then maybe this project could be seen as a way to kill two birds with one fell swoop1. Just a thought.


1 Note: This mixed-metaphor is in no way attempting to target Slate, or any other avian-oriented CM or developer. 😉

6
  • 7
    I like the idea, and it is something that we will talk about more internally. Thanks!
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 17:07
  • 4
    Be careful here. I wouldn't necessarily use activity in the current queues (close queue, edit queue, etc...) as a criteria because those are different activities. The people you want in this staging area queue are those willing to work with newbies to fix/edit their question and who have the proper patience and tolerance. I don't know if it's possible to "find" who that is with SO data, but there's probably a history of back and forth with newbies in comments that eventually result in an edit and answer. And, it's probably less likely to be people who closevote or downvote without commenting.
    – jfriend00
    Commented Mar 29, 2022 at 22:02
  • @jfriend00 Agreed. It's not going to be easy to determine what those criteria should be. Commented Mar 29, 2022 at 22:27
  • Would this include users who have spent a significant amount of time assisting OPs in chat?
    – QHarr
    Commented Mar 30, 2022 at 7:15
  • 1
    @QHarr That may be nice - but very difficult to assess in any system-automatic way. Commented Mar 31, 2022 at 10:14
  • @jfriend00 Agree that people who comment when closing would be good candidates, but I think upvoting an existing comment should count too.
    – tgdavies
    Commented Apr 14, 2022 at 22:43
24

A concern/question I have here is what about first posts that are spam or rude/abusive (i.e. deserve red flags).

Will these be 'visible' to the Smoke Detector bot? If so, will they be 'accessible' for users to flag, when reported by Smokey in chat-rooms?

And what happens to a post that, when reviewed in SG, is red-flagged by the reviewer? (If that will even be possible.)

I understand that most spam posts are answers but there are a significant number that are questions (including "spam seed" questions). R/A (in my experience) occurs equally in Qs and As.

2
  • 19
    This will be covered more in the second workflow post. But to answer your question here: I have already talked with the Charcoal team to discuss the access that they need, and the plan is to make sure that they have this access (including relevant API routes) from day one of the MVP release. We will try to make sure that all regular users of these chat rooms have a way to get access to the SG as Reviewers. Flags on SG posts will follow the normal flag flow to get mod attention (and will show up in the relevant mod dashboard). More on the next post.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:52
  • @YaakovEllis That's amazing; thank you for that consideration, Yaakov!
    – TylerH
    Commented Mar 28, 2022 at 19:03
20

I have several questions:

  1. Will the close votes used in the Staging area be taken from our normal quota?

    YE: This hasn't been decided yet. Leaning towards not counting it, or having it be a separate quota. We'll talk more about close votes in the next post, your feedback there will be good.

  2. Will the question views in Staging ground be counted at all? I.e. when the question gets posted will it already start with several views from the staging ground? Can the question view count be reset to start at 1 as usual?

    YE: We will still count them. Not planning on displaying them. The newly published post will start with a view count of zero.

  3. What happens when the question asker's account is deleted before the question leaves SG? Will it be posted as a community wiki or anonymous?

    YE: Good question. We hadn't thought of that. TBD.

  4. How can askers indicate to the reviewers that they would like to self-answer the question immediately?

    YE: That is the type of thing that the Author should leave in a comment.

  5. Will these questions be available just like the rest in the API calls? Will they be excluded initially and only show up in API once they are posted on main?

    YE: Will discuss API stuff in the next post. Short version: the MVP release will only include enough API calls so that Charcoal can work, and future releases will include more complete API coverage.

  6. Once the question is posted, will the revision history persist from the SG? I don't think the viewers should see the history of the edits done in SG. Can it be restricted or removed?

    YE: The SG post revisions and comments will not be removed or deleted. Users who can see the SG will be able to see a link to the SG post on the timeline/revisions of the newly published post, so there will be a way to get back there. They just wont be carried on to the new post. If there are abuse reasons to remove or restrict access to revisions on the SG post (say: it becomes mod-only after publishing, or after X days from publishing) then that is something that should be raised in a different answer.

  7. Will there be an entry in the question history that a post came through SG? Preferably with a link to the completed review in SG.

    YE: Yes, but only shown to users with SG access (or the Author)

  8. Will migrations be possible? If so, how will that work?

    YE: We hadn't planned on it, though now that you mention it, we can consider it for the MVP (since there can definitely be a need for it, though might make the most sense to publish the post and then immediately migrate it)

  9. Can non-first-time-askers manually add the question to SG? That would be very helpful when asking a question that you are unsure of if it is clear enough. I would love to have that as an option.

    YE: This is something that would be great to enable in the future. Won't be in the MVP though, and will only be able to do it if we have enough Reviewer capacity to handle it.

I have hopes for this feature. It might really improve the situation and I looking forward to the next posts in the series.

4
  • 17
    I'll answer inline with blockquotes. Please add more follow-ups in comments or answers.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 11:29
  • 1
    Great set of questions -- I was concerned with there not being any indication in the timeline/revision history that a post came through SG, so it's great to see all that info will be there for users with the reputation to access it.
    – TylerH
    Commented Mar 28, 2022 at 19:05
  • 1
    Nice @YaakovEllis, at first, because of your initials, I thought you were answering with "partial yeses" :). Commented Mar 28, 2022 at 22:34
  • 1
    @HereticMonkey I thought Kanye West got a job at SO :)
    – Marijn
    Commented Mar 29, 2022 at 10:54
15

It would be better to avoid putting the "earn badges" remark in the first sentence of the Reviewer greeting. While extrinsic motivation is important for maintaining participation levels, making it so prominent undermines the emphasis on helping new users to start up.

It also gives an impression that earning badges is of higher priority than providing actionable feedback to those new users. As we've seen from the existing review queues, extrinsic motivation like earning badges has a detrimental side-effect where reviewers prioritize badges (read: quantity) over the quality of their reviews.

Deemphasing badges should help alleviate that appearance. For example, the remark can be moved to the end of the paragraph and read something like "Also, you can earn new badges for participating".

1
  • 6
    Yup, we've slated this as something to revisit. Thanks
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 15:26
15

Will the minimal viable product release include the ability for moderators to suspend reviewers from Staging Ground?

Providing helpful guidance and fostering a welcoming environment may be skills that some users must develop over time. Some users may require a review vacation to brush up on their skills.

1
  • 9
    The plan right now is for review queue bans to serve double-duty as Staging Ground bans, and if/when there is a need to build in a new type of SG-only ban (i.e.: if it is not uncommon to find users who are good at RQ and bad at SG, or vice-versa, such that a ban that encompasses both won't work), we can consider doing so.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 12:44
15

I could imagine that askers realize that their potential question already has an answer somewhere else while being in the staging ground, for example after including research. Give them an easy ability to retract their question (abandon the question asking process). Maybe feature the "delete" button more for that (and rename it to "retract" or something similar).

Also include that in the statistics to gather. I'm not sure if a large retraction rate is desirable or not, so gathering numbers there seems important.

Other reasons to temporarily derelict a question could be that the asker realizes that he cannot make it clear or focused enough or that it's impossible to deliver debugging details within a few minutes. Maybe in these cases the question asker could then also have a time out (i.e. indicate that he is working on improving the question but needs more time until further feedback can be processed).

Special consideration should also be given to the evaluation metrics of the staging ground. Less closures/deletions and higher scores are surely nice, but they must be in relation to the increased effort. In particular, the fraction of reviewers to askers in the experiment should be the same as for the whole of SO (or the conditions wouldn't be fair). Also typically, after some time, initial enthusiasm wears off, so the experiment should at least run for four weeks.

1
  • 9
    The current workflow would allow them to close their own question as a duplicate, or delete it. Admittedly, new users who are not so familiar with site mechanics might not find this intuitive. We'll take it as a point to consider further. Thanks!
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 17:08
14

A question about the "publish" workflow.

When I'm reviewing a question and I've taken the time to read it and understand it and have decided that it is now an appropriate question or I've worked with the OP to make it an appropriate question, for some portion of those questions, I'll already have an answer in mind.

If it was a question I picked to engage with, then I'll already have an affinity for the content. And, if I think it's now a good enough question, that means I understand what is being asked. So, there's a good likelihood that I may know how to provide an answer.

So, if I approve the question for publishing will I be able to immediately provide an answer to it without having to go find it again on the open site?

This might perhaps be one of the incentives for people to participate in this queue as it give you early access to questions before the masses as long as you're willing to work with people on improving questions that are unclear.

1
  • 2
    Not sure how many people have to click on "good to go" but if it's more than one, maybe have a notification (optionally) about when it goes live and that way you have a direct way of answering it. However, I remember from my reviewing times that I often enough just reviewed without wanting or being able to answer the question. Just wanting to say that it should be completely optional. Some people might just want to review. Commented Mar 30, 2022 at 8:28
11

Will there be a way to participate in the Staging Ground as a reviewer, but only for certain tags?

I've found the other queues troublesome in this regard (like reviewing edits) because I'm willing to participate in this concept for posts where I know the content, but not for other content areas.

3
  • 4
    Yes, definitely! More details on that in the new post. And all review queues (including Suggested Edits) include a tag filter that you can use there to restrict the items shown to specific tags.
    – Yaakov Ellis StaffMod
    Commented Mar 29, 2022 at 9:35
  • 3
    @YaakovEllis - Good, but hopefully, you don't have to re-enter the tag filter every time you go in.
    – jfriend00
    Commented Mar 29, 2022 at 19:48
  • 1
    On the other review queues, selected filters remain in place across sessions. Commented Mar 31, 2022 at 10:16
9

What I would like to see is that questions don't get automatically released from purgatory. I don't know what time frame you were thinking of but I think even a couple of days would not be enough. Questions should only be posted on main if the reviewers select the "good to go" option.

Certain periods see less activity on the site. For example, weekends or holidays see less activity. What I observed is that question quality drops during these periods. It might be related to the fact that people are off-work and dabble with some programming at home. It might also be related to the fact that there are fewer reviewers available as they tend to take some time off too. We don't really want to see questions released from SG just because there was nobody to review them.

4
  • 13
    This is the type of thing that we will want to evaluate as part of our examination of how initial tests go. Auto-publishing for items that dont have a strike against them (close/flag/needs-edit), even without a "good to go" will be no worse than we have today, where everything is auto-published. And there is a big downside of never auto-publishing unapproved items: unlike on the main site, they will never be seen by most users who are browsing the site, which seems like an unfair limitation to enforce on new users.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 12:42
  • I don't think it's fair to a new poster for their post to have a chance of getting stuck in the staging ground through no fault of their own. There has to be some default behavior that doesn't lead to posts stuck in purgatory if no reviewer acts on it. I'm sure this is a complicated issue with lots of things to consider, but reviewers are volunteers. If they happen to skip over a post, it shouldn't get stuck forever. I agree with Yaakov that letting it auto-publish after some time if no action was taken on it by a reviewer is no worse than we have today.
    – jfriend00
    Commented Mar 28, 2022 at 22:11
  • 3
    Based on the reply from Yaakov, I disagree with my own suggestion now. But I am still curious to see how it will actually play out and what percentage of posts will leave SG automatically.
    – Dharman Mod
    Commented Mar 28, 2022 at 22:12
  • @Dharman Depends how many reviewers are willing to work on these questions. Already now with the First Posts queue not every first post can get the attention of a reviewer (IIRC) because some posts simply age away. Something similar could happen here. But one could probably argue that doing something is better than nothing. Giving questions from new users time to mature is surely the right direction, even if we can do that only for a fraction of them. Of course, without reviewers willing to do unpaid work, even the best staging ground idea will fail. Commented Mar 29, 2022 at 16:33
9

Will there be an audit system for Staging Ground reviewers? And, if so, will it be as {cough} good as the current audit system for the other review queues?

1
  • 3
    That will not be included in the initial release, though will definitely consider it for the future.
    – Yaakov Ellis StaffMod
    Commented Mar 31, 2022 at 10:28
7

In reading through the possible "actions" you can select for a question in the queue, I'm confused about the "Suggest Larger Changes" action. I was assuming that you would be commenting on the question and giving the OP feedback in a comment on what needs to be modified in the question.

For example, a frequent comment of mine for newbie questions is to add the relevant code (actual code, not pseudo-code) that relates to the question so we can see the precise problem and advise exactly on how to fix it. Often this is all that is needed to clarify the question.

I would normally do that with a comment and wait for the OP to respond. How does leaving a comment like this on a question in the queue relate to the "Suggest Larger Changes" action?

When do you do one or the other? Or, do you always do both?

4
  • 2
    The subtext of the action seems to indicate that minor changes are changes that the asker can be trusted to do alone and the question can be published automatically afterwards and major changes are changes that need to be reviewed and a separate go has to be given afterwards. Adding relevant code would probably be a major change in this regard because you would probably want to check that any code that is added is really the relevant code. Maybe Yaakov can give examples of meaningful minor changes. Commented Mar 30, 2022 at 8:30
  • 2
    Suggest larger changes = the question is not at all ready to go live, but here is what is missing that you need to fill in. Relevant code being missing is something that could be here. Or "please explain what you tried". There will be canned comments to help with some of the common cases.
    – Yaakov Ellis StaffMod
    Commented Mar 30, 2022 at 13:29
  • 2
    Minor changes = the post could go live as it is, but there are some improvements that the author can make that will make it better and more likely to succeed. Could be edits or formatting (that the Reviewer could make themself, but just dont want to). Could be a request for additional details (the absence of which dont disqualify the question, but the presence of which would increase its chances of getting a good answer and helping others).
    – Yaakov Ellis StaffMod
    Commented Mar 30, 2022 at 13:31
  • 1
    Both of the "Requires edit" actions will require the Reviewer to leave a comment.
    – Yaakov Ellis StaffMod
    Commented Mar 30, 2022 at 13:31
6

The new wizard bypass some check for the title.

For example, in this question: EXC_BREAKPOINT when add value on 2 dimension arrays, the title was containing "error".

When I tried to edit the body of question, it failed because of the invalid title. So, people can post question with "bad" title, but can't edit it.

3
  • I've also come across this problem - very frustrating. The 'filter' for question titles should at least be consistent. Not sure that this post really belong here, though. Maybe you should post it as a "bug" on Meta.SE? Commented Mar 31, 2022 at 10:19
  • @AdrianMole It's only because of the new question system, so I don't think it should be on MSE
    – Elikill58
    Commented Mar 31, 2022 at 10:31
  • That title has 40 characters, though. Note that a full stop at the end is also automatically removed. Maybe the order in which these quality checks are performed caused this title to pass. Commented Jun 1, 2022 at 14:04
6

From the post, it is less clear about

What's in it for reviewers?

Are there any rewards and recognition for the "experienced users" for spending time in reviewing? In my belief, currently there are low quality first time posts because of lack of motivation (through limited R&R) for "experienced users" to go to queues and do the review. Even if you plan to bring new tools, the lack of gamification may still keep "experienced users "disinterested because not everyone have time or audacity to help selflessly all the time; unless they have something to get back in return.

1
  • 5
    This was deliberately not discussed in the current post. This is planned for a future post (that is being written right now) that is mentioned at the top here "Staging Ground Workflow: Open questions for the test and the future", which includes as its first main topic "Exploring Reviewer motivation"
    – Yaakov Ellis StaffMod
    Commented Apr 6, 2022 at 10:43
6

Proof of Concept

Focusing on the following sentence:

If this question helped you, make sure to upvote the question and answer.

This is for the Staging Ground, which is for "the Author’s first question on the site, or their second/third question if their first question(s) didn’t do well".

Given this criteria, I wouldn't expect most authors who qualify for the Staging Ground to yet have 15 reputation. Accordingly, I wouldn't expect them to have earned the "Vote up" privilege. If that's true, this guidance doesn't make sense.

Will you be hiding that part of the prompt for contributors who don't yet have 15 reputation?

+1 on the verbiage for all other users.

1
  • 5
    Good call on hiding that instruction for users who can't upvote.
    – Yaakov Ellis StaffMod
    Commented Apr 10, 2022 at 8:15
5

I've gone through a number of questions in the current "First Questions" review queue and am trying to imagine how that fits with the proposed UI described here. Here are my thoughts.

  1. Filtering seems sticky so that's good. I can set it and it appears to stay so I can only wander through questions associated with certain tags.

  2. When you hit "Good to Go", the question disappears from view so there's no easy way to then write an answer for it. Since it's a question I understood and in my area of expertise, I'm likely to be able to write an answer for it and I think that should be easy.

YE: the plan is that when you hit Good to go and the questions is published, a notice will show on the screen confirming the success of the operation and giving a link to the new post. This link can be used in order to go and answer the question. A user will also be able to review their Staging Ground activity in their User Profile, and should be able to access published questions from there as well.

  1. Personally, I'd rather see a list of subjects and pick which items from that list (the way I pick regular SO questions to engage with) that I'd like to explore rather than be fed questions one at a time. Picking the subjects that seem relevant to my expertise feels more like I'm selecting the content I wish to engage with vs. being fed them one at a time where many I have to just hit "Skip" on feels more like a chore. I really this is a repurposed interface from other queues that you already use, but I'm just telling you how it feels to me.

YE: In that case, I believe that you will be very happy with the options that will be presented by the Staging Ground listing that will be highlighted in the next workflow post.

  1. If I leave a comment suggesting what needs to be edited in the content, will I be notified if they do edit the content? If not, it seems like I should.

YE: If you leave a comment accompanied by an action, you will auto-follow the question and will receive notifications on any subsequent actions taken. If you leave a comment without performing an action, you will only receive notifications if you are pinged directly or if you elect to manually follow the question.

  1. It seems like the actions list should be more complete, including things like Duplicate and Close. I guess you can directly just click on Close under the question, but that makes for a bizarre UI where the "happy" actions are in one place and the "unhappy" actions are elsewhere.

YE: This was already raised in a previous answer. We are discussing internally if/how we may address this for the initial release.

  1. Does it just take one "Good to Go" action for the question to go live? Or more than one?

YE: for the initial test, one "Good to go" vote will be sufficient. This is subject to change in the future based on how things work with real questions and Reviewers.

  1. Why is "Edit" an action? Wouldn't you just use the edit link to edit the post and THEN pick an appropriate action? There are plenty of times where I immediately edit the question (usually to fix code formatting) just so I can read the question and then after that I decide what to do with it.

YE: It is an action to promote it as a primary activity that can be done. The edit screen will give the Reviewer the other actions as follow-up options. This can of course be done the same way by using the edit link and then performing an action. For this reason we are weighing removing the edit link for all but the author.

1
  • 5
    I added responses inline
    – Yaakov Ellis StaffMod
    Commented Mar 30, 2022 at 18:59
4

TBH I'm pretty excited about this feature! My thought on this:

  1. Many users want their question to be answered fast and so don't want to spend much time creating an elaborate question. They certainly don't want to wait until questions pass review. This is both a good thing and a bad thing.

    • Good: I don't have the metrics, but I can imagine that questions like these make up a good chunk of the overall question volume, so not publishing them immediately will automatically improve quality of questions on the main page overall

    • Bad: This could prevent first-time users to ask questions at all. The time it takes for a question to be published and answerable might be not feasible for users.

  2. What about the number of reviewers for a single question? Can anybody review any question at any time? Should the number of reviewers be limited? I'm just thinking that having 10 people review a question at the same time could lead to problems.

    • Too many cooks in the kitchen, which might overwhelm the author

    • What if two or more reviewers want the author to make edits but those suggestions conflict each other?

  3. I think a staging ground may impact the overall quality of questions massively

    • Less low-quality questions, except for those that slipped through because of the timeout

    • Less spam! Bots probably won't be able to pass the ask wizard in an easy way. Also spam can now be eradicated before it is visible to the public.

  4. I love the "Upvote when published" option

    • The first upvote is a hard thing to achieve

    • This will definitely motivate users to post better questions, when they can start their visit on SO with a +1

3
  • 2
    We're not looking for "elaborate" here; we're just looking for questions that meet our minimum quality standards and contain enough information to be answerable. If that slows down some people on firing off incoherent questions, then that's all in the "advantage" column. This won't slow down bots in any way, but we've already got a good system for dealing with that. And I've never seen that an upvote is hard to achieve. A good 20% of the spam and gibberish that gets posted, I also see getting upvotes... Automatic upvotes are a terrible misfeature.
    – Cody Gray Mod
    Commented Apr 7, 2022 at 9:09
  • 2
    Re: concurrent reviews, please check out the second workflow post where I go over how we will try to avoid concurrent reviews, as well as what happens if they occur.
    – Yaakov Ellis StaffMod
    Commented Apr 7, 2022 at 9:33
  • 7
    "Upvote when published" - I am also hopeful about this. It is not an "automatic upvote". It is an earned upvote. We dont allow voting in the SG, so it just let's the reviewer indicate "I have seen this question and think that it should go live deserves an upvote, so please give it one from me when it goes live" - saves Reviewer from context switching to load up the question later and vote.
    – Yaakov Ellis StaffMod
    Commented Apr 7, 2022 at 9:35
3

So the remark about this being a queue was surprisingly spot-on.

What are you going to do with questions which would fall into this category, be enqueued, but not have enough people around to look at it (including the OP who decides to not stick around after getting put into this workflow)? Would those questions still just go on the main site, or would they be unaskable?

20
  • 1
    Hopefully they would be unaskable; while unfortunate because it means some questions will have to be re-asked later if OP wants an answer, the opposite choice (just posting them anyway) would defeat the entire point of trying to catch bad content early.
    – TylerH
    Commented Mar 28, 2022 at 18:59
  • @TylerH: I get the impression that they'll skip the queue entirely to be honest with you
    – Makoto
    Commented Mar 28, 2022 at 19:00
  • 4
    I disagree that this is another review queue. We are promoting a different workflow here, and as will be detailed in the next post, offer significantly expanded options on how Reviewers can select questions to interact with.
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 19:09
  • 8
    The plan for the initial test, as described above, is that after a period of inactivity (TBD) any questions that do not have a "strike" against them (close vote, flag, reviewer action requiring a change) will automatically graduate to published state (and will enter the First questions queue). The inclination right now is to not automatically label unreviewed items as being "unaskable".
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 19:11
  • 1
    @Makoto FWIW this issue is also discussed in Dharman's answer meta.stackoverflow.com/a/416860 and Yaakov responded stating that it's a "wait and see" situation but initially posts that don't have any negative strikes/actions against them that 'age out' would get posted (which is the usual behavior today, anyway).
    – TylerH
    Commented Mar 28, 2022 at 19:17
  • 1
    So maybe "queue" is too literal here, @YaakovEllis. The way I'm framing queues is that this is still something voluntary that the community at large has to be incentivized into doing. This is a lot more work than just some review, I'll give you that - so the incentive to do it also has to be pretty compelling. Also too, we'd have to address the risk that someone decides not to engage on the site even though a volunteer spends X amount of minutes/hours on trying to help this person get their question together.
    – Makoto
    Commented Mar 28, 2022 at 21:37
  • Right now I don't see that this proposal does any of that - it just feels like the community at large is being asked to take on on a whole lot more without any real guarantee that we'd see an ROI.
    – Makoto
    Commented Mar 28, 2022 at 21:38
  • "Would those questions still just go on the main site, or would they be unaskable?" Depends on if they have been acted on already. "Questions that are not approved by Reviewers will be auto-published after a defined period of inactivity, except for questions that have received a close vote or flag, or where a Reviewer has asked for major changes that were not made (this will be covered more in the second post).." I understand that as if someone criticized them they are unaskable. Commented Mar 29, 2022 at 8:54
  • 3
    @Trilarion correct - once someone has asked for major edits or put in a close vote, they will not go to the main site unless someone goes in and approves them. No auto-graduate. Flags will also hold it up (so if a mod deletes the flag, then it can auto-graduate).
    – Yaakov Ellis StaffMod
    Commented Mar 29, 2022 at 9:38
  • 6
    @Makoto "it just feels like the community at large is being asked to take on on a whole lot more without any real guarantee that we'd see an ROI" - pragmatically speaking, yes. This is a bigger ask. We will discuss in a different post potential incentives that are more than badges, and more tangible than "feel good that you helped folks", and are trying to create tools that will streamline the experience as much as we can for Reviewers. But it is a worry (and part of the initial test of this whole section is to see how it plays out in the real world with a smaller set of participants).
    – Yaakov Ellis StaffMod
    Commented Mar 29, 2022 at 9:40
  • 2
    @YaakovEllis: Maybe to give some advance indication to what kind of incentives would entice me to participate: my current rate starts at somewhere around $50/hr. I don't really see a scenario in which it'd make sense for my otherwise volunteer time to be spent walking someone through making a question on Stack Overflow with the very high likelihood that they won't want to go through this process again without compensation. But I'll look forward to the follow-up post.
    – Makoto
    Commented Mar 29, 2022 at 17:02
  • 4
    @YaakovEllis - As a counter to what Makoto said, I spend time trying to walk newbies through fixing their posts already on the site today and I'm swimming against the tide of people who just want to downvote and closevote and make the question go away, so I would welcome a better way to help newbies than we have today and, if it's efficient to use, I would likely volunteer some of my time. Just saying that there are some experienced users out there who would appreciate what you're trying to do.
    – jfriend00
    Commented Mar 29, 2022 at 20:59
  • 3
    @jfriend00: It's refreshing to see a counter argument. I'll be clear - I would welcome this too if it were efficient, but I only have so many hours in the day and there are other things I wish to do with my time. Stack Overflow Inc has made it clear that they are monetizing the Stack Overflow experience, and capitalizing on the work that volunteers like yourself are doing. And that's fine. But this volunteer wants more than a pat on the back for doing what they what and a swift kick to the food dish when trying to give feedback that they do not like.
    – Makoto
    Commented Mar 29, 2022 at 21:07
  • 2
    Yeah, those of us who provide lots of answers on SO volunteer here for our own reasons. I like the gamesmanship of trying to "win" the best answer contest and the psychological bump from helping folks and also from seeing my rep go up and the challenge to look up and learn new things to be able to answer something. In that context, the implementation of this staging ground is critical. If it feels like a chore, I won't be there for long. If it feels like a better way to help newbies than we have today and I get to answer some of those questions, I might enjoy it.
    – jfriend00
    Commented Mar 29, 2022 at 21:16
  • 2
    @jfriend00 I sometimes help new question askers if I have free time, and I appreciate others who basically make it their hobby to help others. I even think the company is definitely trying to help both, inexperienced users and reviewers here. But I still think the major problem is that most new questions should not be asked at all, because they have been asked before. And that is not addressed. A basic mandatory "how to search on the internet" course before being able to ask a question here might help more in the end. We will see how it develops. Surely there will be statistics about it. Commented Mar 30, 2022 at 18:22
3

I see Adrian's request to make these posts visible to the Charcoal project, and while I think Charcoal a great project, I'd encourage the company to run spam detection themselves, to whatever degree would be sustainable by the company.

After seeing Charcoal's activity up-close for a couple years now, it's doing an amazing job of keeping the sites clean of spam. This intermediate staging ground is an excellent opportunity to prevent spam from ever reaching the live site.

This doesn't mean that Charcoal would or should stop; it's just that I think prevention is better than cure.

Ditto for a similar project Natty that I've seen running in SOBotics. Perhaps a new post that scores badly enough with that code could be marked appropriately in the staging ground to provide the most appropriate response?

1

Is it a coincidence or a conscious decision to set the current proposed date of the trial to the 3 months when most students don't have classes?

The reason for asking is it doesn't seem like a realistic test of its merits if you avoid the eternal September and the influx of new users it brings.

4
  • 6
    It was a coincidence
    – Yaakov Ellis StaffMod
    Commented Mar 28, 2022 at 19:43
  • 3
    @YaakovEllis - I was hoping for that to be the case, please consider making sure theres some cross over with a student body
    – Sayse
    Commented Mar 28, 2022 at 19:44
  • 4
    eh, i don't see a major issue with the initial test being outside of that influx, assuming that if the initial test is successful a future more expanded/full scale test would be expected next.
    – Kevin B
    Commented Mar 28, 2022 at 19:46
  • 1
    @KevinB - my worry is it might not give a true reflection of how successful it is when the new user base doesn't reflect the majority of the year. I do see the benefits of having it at this time too though so if i were to suggest I'd aim for 6 weeks either side of september 5th
    – Sayse
    Commented Mar 28, 2022 at 20:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .