-63

CLARIFYING UPDATE

Thanks everyone for the thoughtful feedback. I’ve read through much of it and there are a few things that stand out to me.

First, I want to clarify since this didn't quite come across, none of this, except for what we felt were least risky which is outlined as phase 1, has been built. These bigger concepts are ideas we want to test to address issues with the question close-to-open workflow. Our goal with sharing our thoughts on meta at an early stage is to get feedback about what we might be missing as we consider these ideas. For example, defining a “substantial” edit is something our data team is helping us explore and not yet finalized.

Second, it sounds like I missed the mark in articulating why we think these ideas are worth pursuing. We want to help askers increase the likelihood of getting their questions answered. We think that providing a clear path to reopen is like saying “you get a second chance to try again.” And our hope is a bit more privacy and better guidance will be motivating and helpful for askers to craft a better question. It's clear that many of you are concerned about automatic reopens and the potential for abuse with these ideas. This is absolutely top of mind for us, and that's why we are planning to run experiments to better understand and eventually mitigate such behavior before any of this is released broadly. Which close reasons are eligible for automatic reopens? Should it be easier to reclose a question that was reopened? When should we disqualify a question from reopening at all? These are questions we're still working through, and your input here as well as our experiments will get us closer to these answers.

Third, there is always the possibility of a failed test during discovery and experimentation. And that’s okay! Learning from failed experiments is just as important as nailing it. In fact, we will fail more times than we succeed. I think getting comfortable with experimentation and potentially failing is a better mindset to be in than trying to nail the perfect solution or worse, not trying anything at all because we’re too scared to make a mistake. My hope is that we can engage with you openly to test new ideas, question our assumptions, and flag things we’re not thinking of.

Lastly, there are really useful suggestions here as well as callouts to other areas we are actively thinking about. For example the state of review queues is a sore point mentioned several times, or improving onboarding to help new users understand the rules of engagement and expectations of the community. These examples are two sides of the scale we want to balance – how can we both educate and help new users navigate their way and encourage them to continue contributing to Stack Overflow, while improving the tools curators, mods, and our CMs use to maintain the site? This is a recurring theme you’ll see from our product and community management teams this year as it’s top of mind for all of us.

As far as next steps– feedback from meta is in fact research that we formally use to inform product direction. I’ve learned how meta can be a constructive source of feedback for our product discovery - thank you for that! We’ll use the feedback here through this discovery phase and share updates as we hit milestones along the way. In case you missed it, here is a process we're testing that you can use to make sure posts relevant to this project get staff attention.

I hope this helps ease some of your concerns and I will provide additional comments below the best I can, if Catija hasn’t already done so.


Hey! I’m Des. I’ve been here at Stack Overflow for a little while now (4 years and some change) and just recently started working much more closely with the Public Q&A product team as Director of Product, Public Platform. Working within Teresa’s organization, I'm responsible for setting the overall strategy for the Public Q&A Platform. Over the past couple of months, I’ve been working closely with our development and research teams to ramp up on our current initiatives and gain a better understanding of user needs and pain points. I still have a lot to learn! But I’m thrilled to be working with this team (and you!) and look forward to sharing more with you here :)

As mentioned in Teresa’s Q1 roadmap post, we’re working on making changes to how the question closing system works on Stack Overflow so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators.

Why are we doing this?

Our research has shown that often users hesitate to ask questions on Stack Overflow due to the fear of public, and sometimes caustic, feedback. From curators, we often hear about the time-consuming burden of manually guiding users whose questions get closed, and sometimes receiving pushback when they are just trying to help. And from moderators, we’ve heard the ask to reduce toxicity for all users through improved moderation and communication tools.

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

Our research and design team took a closer look at the problem, considered previously proposed solutions and feedback, and came up with what a long-term solution might look like. Then we broke it down into phases so that we can solicit feedback, test, and iterate along the way rather than take months on end to get something out there.

Goals

The goal of this project is to update existing close workflows with a streamlined system which is easier to understand and follow for all users and results in fewer negative interactions between them.

Askers are often unsure how to get their question reopened and onsite guidance can be confusing. They often experience a pile-on effect of downvotes and comments while trying to edit their question in parallel. This can feel overwhelming and intimidating for question askers and drive-by visitors.

Curators are burdened with an enormous backlog of tasks. We want to reduce the burden by helping question askers improve the quality of their question through product guidance, relieving curators of this tedious and often thankless task. We will also be testing automatic reopening, putting the onus on the question asker to improve and reopen their question and explore the notion of “hidden” as opposed to “closed” questions, giving askers the space to (somewhat more) privately improve their questions. More details on these concepts are below.

Things we’re not doing

As a quick call out, we’re not planning the following:

  • Network wide roll out - This project addresses pain points experienced on Stack Overflow. We’ll need to better understand the needs of the other network sites as this solution may not be needed, nor appropriate for other sites with less traffic and peer-to-peer friction. As of now, we don’t have any set plans in the nearterm to roll this out to the network. Having said that, if there are parts of this project that can relatively easily be rolled out to the network, we’ll try to prioritize them.

  • Automatic closures - We won’t be automatically closing questions. Questions will still need to be voted on for closure.

  • Getting rid of question closing - We have no intention of removing the question closing feature, and continue to embrace our commitment to maintaining high quality content on the site.

Feature Overview

Outlined below are the key components to this project. And a bit further down, I provide more detail on how we’ve phased this project out. While the team has validated the general direction through user, historical, and competitor research, we want to test these ideas and get feedback where we can along the way. Here are the key components of this project.

1. Close UX improvements

We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed. It’s a gentler way to remove a low quality question from general visibility while giving the author a chance to edit it. Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance. We are still toying with ideas for what this path looks like, and we welcome your suggestions!

Inbox notifications and emails will alert post authors of newly hidden posts and provide guidance for editing. To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted. And to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

Email notification enter image description here

2. Edit UX Improvements

This new editing UX will provide guidance tailored to the specific reason the question was closed. We believe that this will help users make substantial edits that increase the likelihood of getting an answer. For askers, they get clear guidance about how to edit and reopen their question and ideally have a better experience on the site. For other editors, based on explanations in the "closed" post notice, they’ll know how to rehabilitate the question and help the question author. Changes to post notices have been rolling out over the last several months. We'll be continuously refining the content we show here based on user feedback.

You’ll notice the updated editor borrows UI patterns from the new Ask a Question form.

Edit form, draft step enter image description here

Edit form, review step enter image description here

3. Automatic Reopening

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once. Any subsequent reopens would require review through the reopen queue.

Here is a clickable prototype that illustrates these three core pieces. Note that final designs and copy may change.

Iterative and Incremental Development

The close-to-reopen workflow is a multi-touch process. In order to understand the impact of these various changes, to minimize disruption for users, and to avoid being in development for a lengthy period of time, we’re taking both an incremental (break the project down in smaller pieces) and iterative (continue to refine as progress is made) approach. Here’s how we’ve planned things so far.

The first phase, scheduled to release over the next couple of weeks, is much more solidified. Whereas each subsequent phase is less so, giving us room to adjust based one the data and feedback we see.

Phase 1

This first phase is made up of foundational changes and does not yet introduce the new concepts of automatic reopen or hidden questions. These are things we’d likely do anyway and provide the groundwork for the other phases.

  • Updates to close/off topic dialogue reasons

  • Updated editor for closed questions - pulling from the new Ask a Question design pattern, this provides better guidance through the editing flow.

  • Email and inbox notifications about closed questions - users will first receive an inbox notification when their question is closed. If the inbox notification isn’t seen, an email is sent to encourage them to make edits.

Phase 2

This phase introduces automatic reopening of questions when they’ve been substantially edited and passes questions with more than one closure to the reopen queue. At this stage, we plan to do some quantitative and qualitative testing to understand the impact automatic reopening has on askers (are questions more likely to be answered? closed again?) and curators (do we see a decline in low quality flags? are higher quality questions making it to the queue?). Deletion and dynamic thresholds for closing will be explored during this phase.

Phase 3

Reconceiving “closed” as “hidden.” When a question receives sufficient votes to become hidden:

  • The post is hidden for primary views. Post authors and privileged users would have a direct path to view and edit. Anyone with a direct link can still access it as well.
  • Score and voting capabilities are removed.
  • Post notices, inbox notifications, and emails are all updated to reflect the new language.
  • If the question isn’t reopened, it’s deleted after a certain period of time.

This step will also be an important piece to test and iterate on.

Questions & feedback

As mentioned, these phases are a rough outline of how we plan to tackle this project over the coming months. We plan to share our findings with you here at each milestone and get feedback along the way. For now, we’re happy to hear your thoughts or any concerns you have, as an answer below.

It should be noted that we still have some open questions that need discovery and you might think of something we’ve missed so feedback on things not mentioned here would be great. For example, what should we do with existing closed questions? There are lots of close questions today on Stack Overflow and we hear that users still find them helpful.

Thanks all for taking the time to read this and provide feedback at this stage.

72
  • 45
    What about duplicates? Does the same automatic reopening logic applies to them too?
    – Dharman Mod
    Commented Mar 27, 2020 at 20:31
  • 208
    It is a little bit insulting when you realize you want to let 1 person who has probably no experience with this site overrule a decision made by 3 people with ceratin amount of experience just because they have edited the question. If 3 people closed the question then 3 people need to review it after edit is made and decide if it can be reopened.
    – Dharman Mod
    Commented Mar 27, 2020 at 21:00
  • 51
    Something that is good to note: this is a test/plan, nothing is set in stone. So, don't panic. :) Commented Mar 27, 2020 at 21:51
  • 19
    @Dharman we plan to treat closed as duplicate slightly differently. Current thinking is to allow the user to confirm if it’s a duplicate question by selecting from suggested duplicates. If it is, the question becomes public and references the duplicate. If it isn’t, the question will need to be edited before automatic reopening (if it’s the first closure).
    – Des StaffMod
    Commented Mar 27, 2020 at 22:13
  • 152
    Current thinking is to allow the user to confirm if it’s a duplicate question by selecting from suggested duplicates. --> most of users don't agree with duplicate simply because they don't like their question being closed as duplicate not because the duplicate isn't suitable ... the question will need to be edited before automatic reopening --> most of the edits are my question isn't a duplicate! so I don't agree at all with an automatic reopening of duplicate closure ... This will give us more job to reclose them again by calling another gold badge. Commented Mar 27, 2020 at 22:32
  • 40
    I think these steps are really good experiments to figure out what askers and curators need. But I don't expect these experiments to be particularly successful, for the reasons outlined in all the answers. Please don't activate automatic reopening unless the other components indicate an increase in edit quality. But nevertheless, thank you for engaging the Meta community early on.
    – amon
    Commented Mar 28, 2020 at 7:03
  • 27
    even if it's not a dumb check, a system cannot evaluate (based on OP's edit) that a question is no more a dupe and should be reopened (even the most advanced AI cannot). I usually close question with canonical target having 20 answers and OP always comment the same thing that answer is using X but I am asking for something different because he simply look at the accepted answer and never take the needed time to check ALL the answers, test them, understand them. I am already facing issues with blind vote (meta.stackoverflow.com/q/393400) and this automatic reopening will make it worse Commented Mar 28, 2020 at 8:11
  • 29
    Am I really seeing another "this is what we're going to do, whether you like it or not" announcement? Another "our staff thought about it, but we didn't discuss any details with the community until it was a fait accompli"? If so, then this is an unsatisfying flavor of transparency. Commented Mar 28, 2020 at 17:59
  • 30
    @JohnBollinger Hardly. There’s three phases and they talk a lot about testing and trying things out. We’re pretty early in this, which is why we’re posting now. We want to know what y’all think.
    – Catija Staff
    Commented Mar 28, 2020 at 18:31
  • 19
    Never mind the downvotes; thank you for putting some thought into this and sharing your ideas thus far. The regular visitors of meta are not your target audience – "bad" questions are our arch-nemeses, and out of sight is out of mind – but I can see how this may be able to turn a Bad Question into a (Reasonably) Good one. Is there objective evidence that this would be a good investment of your time? Can a significant number of quite obviously "bad" questions be salvaged this way?
    – Jongware
    Commented Mar 28, 2020 at 21:04
  • 15
    Like others, I have issues with what is "substantial editing". Since this needs to be clearly defined before implementing it, could you, um, edit your question to at least give us an idea of when Phase 2 is likely being rolled out? Finally, why not automatically route "substantially" edited dups to the review queue instead? I've found that many dups have comments arguing that it isn't one from the OP.
    – user7014451
    Commented Mar 28, 2020 at 21:14
  • 24
    I'm pleased to hear that, @Catija, but I hope you appreciate that the announcement reads as if all the major decisions have already been made, and the multi-phase incremental development process is primarily about delivery. A schedule is presented for developing everything described in the announcement. If that is not actually the way the announcement should be read then it would be really helpful if the announcement were rewritten to more accurately convey what is happening here. Commented Mar 29, 2020 at 12:39
  • 18
    @RandRandom Users closing questions need to be able to turn off those pings per question. I close 40-50 questions a day... many of them cannot be edited into shape because they're inherently off-topic. I don't want to get 40+ pings a day about users complaining in an edit that their question was closed or that SO sucks.
    – TylerH
    Commented Mar 30, 2020 at 13:11
  • 45
    I would argue that "just 3% of closed questions are ever reopened" is a sign that the people who close questions do so with 97% accuracy. Commented Mar 30, 2020 at 14:31
  • 29
    Please don't cast close votes on this, folks. It's absolutely great that the company want to share their research and ideas with the community.
    – halfer
    Commented Apr 1, 2020 at 10:28

31 Answers 31

222

You had me up to this:

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once. Any subsequent reopens would require review through the reopen queue.

How do you define substantial? Or, more specifically, given a specific close reason, do you factor that in when looking at the change? Something closed as belonging on Server Fault for an instance could be edited with server configurations. It's still off-topic, but would that be considered a "substantial" enough change to reopen it, in spite of it being completely off-topic?

What about people who edit their question to ask for a reason for the closing? What if OP editing in a question about the closure ends up being classified as a substantial edit, and reopens it for no good reason?

And, because it has to be said, what about the abuse potential? People deciding to copy-paste the same code over and over? Adding gibberish? What about the more extreme with people adding blatant CoC violations targeting the closers, and then subsequentially having it reopened? "Welcome to Stack, where you can get questions reopened by saying mean things to close voters"?

You had me up until that point, but there's so many ways it can be abused. Moreover, what happens if the edits are reverted in the grace period? As in any of the above just to get the question reopened, and then instantly removing the edit? If abuse patterns spread wide, it'll effectively take 6 close votes to close a question, which is more than we originally had before dropping it to three.

The reopen queue is more than manageable atm. A low reopen rate doesn't necessarily signify that we aren't reopening enough. It could indicate that we're closing trash that the posters then don't edit into a reopenable shape. Auto-reopening just sounds like a great way to leave stuff that should be closed in an open state, and risks doing the opposite of what this change is supposed to do

11
  • 90
    "What about people who edit their question to ask for a reason for the closing?" Indeed; I've seen plenty of edits to closed questions where the edit is a huge paragraph (or multiple paragraphs) complaining about incorrect closure or going into ridiculous detail about why the Q should not be closed...
    – TylerH
    Commented Mar 27, 2020 at 20:53
  • 40
    there would be no need to add gibberish to break the system relying on automatic detection of substantial edits. I bet that top hit on google when someone enters "my homework was deleted at Stack Overflow" will become a recipe of an easiest way out, "copy some code from somewhere, indent it with 4 spaces and add it to your question and it will be immediately salvaged". And no automated system will be able to tell whether this is a random code dump or a relevant mcve. So this will essentially turn into increasing close vote threshold from 3 to 6 for vast majority of homework dumps
    – gnat
    Commented Mar 27, 2020 at 21:31
  • 14
    How we define it is exactly what we're going to figure out. There's a lot of work to do and learning to gain before we will know what the best solution here is. Heck, there's always the chance that we'll try it, iterate on it a few times to see if we can make improvements, and it'll be a bust... but we want to work with y'all to find solutions that don't make the situation worse while also helping users not feel like a question being closed is permanent and unfixable. Yes, SF and SU questions are never on topic, regardless of editing... maybe we base reopen process on the close reason...
    – Catija Staff
    Commented Mar 28, 2020 at 3:13
  • 11
    There's a lot of thinking yet to do and a lot of work going into this. The post here is not a final plan. It says that we're going to iterate and that's the plan. We're not assuming that we know the one best solution and we're going to do it and walk away. I've been working pretty closely with Des the last few weeks and it's been great to talk about my concerns and what we're working on that may have better/different solutions to meet the same need. Closing is something that needs help. 5k questions in the queue here isn't sustainable... there's a lot of interesting thinking we're doing. :)
    – Catija Staff
    Commented Mar 28, 2020 at 3:16
  • 99
    There is never a good reason to allow a single edit by the original author to automatically re-open the question without any community review. There is a major conflict of interest here, not to mention a near guarantee of failure. No amount of added "smarts" are going to make it a good idea. It'll always be a "dumb check" because it's being done by a machine.
    – Cody Gray Mod
    Commented Mar 28, 2020 at 8:34
  • 31
    “Closing is something that needs help. 5k questions in the queue here isn't sustainable” Yes, exactly. And you are drawing exactly the wrong conclusion from that fact. Closing needs to be easier to do, and it needs to cause the question-asking bar to get much higher for that questioner, immediately. We don’t need you to “iterate”. We need you to respond to our needs using fact based statistical analysis and rational decision.
    – matt
    Commented Mar 28, 2020 at 16:10
  • 17
    Not to forget about triage and the horrible H&I queue that comes from mishandling in triage. Reopening is practically of no concern relative to some of the other issues. Commented Mar 28, 2020 at 16:22
  • 7
    Since there is now some notification/follow mechanism for questions: Wouldn’t it be enough to notify the users that originally closed the question and give them the ability to instantly reopen the question (alone) if they see that an edit salvaged the question? That would totally avoid having to rely on automatism that won’t work anyway.
    – poke
    Commented Mar 29, 2020 at 17:26
  • 6
    @poke Somewhat related, I'd like to see this suggestion implemented (TL;DR: give silver badges a reopen hammer on closed questions when edited). Some of the UX changes presented (up to the automatic reopening) seem like a great start though, but combining that with the question following, and letting silver badge holders hammer reopen when edited seems like a great idea. Commented Mar 29, 2020 at 18:04
  • I'm not worried about it getting gamed. If it gets gamed badly it will be changed again.
    – Joshua
    Commented Mar 30, 2020 at 3:00
  • 12
    Having edits automatically make the question visible again seems perfectly sensible. Having edits automatically make the question open again seems likely to be pretty self-defeating. Commented Mar 31, 2020 at 9:42
211

I'm a low reputation user with no history on this site.

I think it's a misunderstanding that high-reputation means more willingness to curate or that low reputation users don't care. I could have spent more time getting reputation, instead of going through meta in an endless search to understand the rules. It would certainly have given me more moderation tools, but I would have been even more clueless as to how to use them, than I am now. The more I learn, the less I want those moderation tools. Just don't make the decision for me, but leave it up to me to opt-in or opt-put, see bahrep's Option to hide deleted posts.

Not answering

I initially stopped providing Answers when I started to learn about curating the site. For 2 (3) main reasons:

  1. Lots of rules and explanations are hidden deep in meta, and it's extremely time consuming to get through it. So, I don't have time to answer questions.
  2. I'm confused about the rules. I don't really want to answer a post, that gets migrated, closed or deleted later on.
  3. A combination of 1 and 2. Even if I find a post about a rule, it isn't very clear whether or not there's a conclusion or even any consensus. And it's almost never one post. It's an endless myriad of links.

Not asking

I have lots of Questions, but I don't ask them. Because:

  1. All of the above. I don't know if my Question will be valid here or not.
  2. If the Question isn't OK here, it'll get a lot of downvotes. It's a lot harder to bring a Question back from -6 than to start at 0. Creating a Question is high risk when I don't know all the rules.
  3. Looking at currently open Questions is no measure for what's OK here. Nobody will understand that unless they spend a lot of time here.

The last point is significant: Why would "my" Question be closed when its asking style is similar to all the other Questions that are still open? Look in the tag. Lots of those questions are answered, a few of them are closed before the first answer. Lots of them are not closed, even if they should be. I had no idea that those Questions were off-topic until I found Why should I provide a Minimal Reproducible Example for a very simple SQL query?. It was a link in a comment on a closed post! The type that's soon to be hidden away from me. Currently I go through closed post to gain experince about what should be closed, because it's the only way I can gain experience without getting review-banned (Deliberately not using the term review-suspended, because while I'm learning, being barred from the review queues is not perceived as a time-out).

The abyss of "meta"

There's information to be found in the help center, sure. But it's tricky to apply (for me anyway). There's also the FAQ, but they usually lead to a specific meta Answer, like the post about SQL queries, that is indeed linked from the FAQ. But just reading that one meta answer doesn't provide context, and reading the meta thread also lacks some context. Reading the links in the thread leads to more answers, and the spiral sets off. I've been at it for almost 2 months now, and there's still no end in sight. I even had this particular problem when I read this thread. Just count the amount of links on the page (or even just my post here), and you'll get the picture.

What would help?

  1. Find consensus on what's allowed and not allowed. Then publish that consensus/conclusion in clear text.
  2. Elaborate on the help pages. Do not provide links to endless spirals on meta, but to a document that's completely self contained. With examples and tricky corner cases.
  3. Lock closed posts for voting with an artificial -1 vote making it Roomba enabled. It doesn't make sense to ask users to improve on their Question asking skills by requiring them to be better than most other users, when it's clear that they're already worse. If the question is closed, that should be enough information about the post. It doesn't need to be closed and have a score of -3. I know this is unpopular, but see the next point:
  4. Don't allow new post from inexperienced users to be open by default. These Questions are already in a review queue. Let them pass the bar on the queue first. This would also ensure that they're not initially put on the main page. Once a Question clears the queue, remove the artificial -1 vote, and publish the post. This way the Question ban will still take effect, but it will be easier to bounce back when already posted Questions are improved.
  5. Make more information about posts available. I already can't see if a Question is protected when I'm logged in, so now I actually have to open Questions up in a separate browser to see that information. If you want to hide closed questions, by all means, do that. Just don't do it to users that are logged in, unless they choose to opt-out.

Shattered dreams

I had hoped that an initiative would be to make is easier to find information and learn both what's acceptable and how to curate the site. For the latter, Machavity made a suggestion: Create a community curated review training simulator. But the truth is that those two things (what's acceptable and how to curate) are in fact the same information that users need to create better Questions. I keep a file with notes. It's currently counting 4350 lines of links, comments, hints, todo's & reminders. It just shouldn't have to be this hard nor complicated.

The problem in Triage from my perspective is not how to convey the information about what one button really means. That can be conveyed by changing the wording of the button ("Unsalvageble" → "Close / OP needs to edit this" and "Requires Editing" → "We can edit this"), or for the "Requires editing" an extra layer of "Are you sure?" with an added explanation. It's probably the one button on the entire site that should be used only very rarely, so the extra pop-up shouldn't be that much of a nuisance. The difficulty in any review queue is knowing the rules. Also, many users think that "being welcoming" to new users means "don't close their Questions".

I have lots of conversations with experienced users about the rules. Sometimes there's a consensus. Kindly, read this again: Sometimes there's a consensus among experienced users. For me, all the alarm bells are going off.

This way of gathering information is common for big organisations. Everyone that's been around from the start don't see this problem very clearly. They, often subconsciously, know how to avoid the tricky stuff. Everyone that's just arrived feel completely left out of the loop. Pun intended.

Welcoming a new user does not mean:

  1. Making it hard to gain information, by:
  2. Hiding it away deliberately or by "meta".
  3. Ensuring that if they make mistakes on their posts, it puts them into a deeper hole. Allowing more downvotes to an already closed and downvoted post, makes is increasingly harder to bounce a Question back on track. This is exactly what is perceived as "permanent and unfixable", referring to a comment made by Catija. There's no point in iterating over it.
  4. Helping them to get Question banned faster, by suggesting creating a new Question. I completely agree with Machavity's answer

I didn't meet the quality standard, but they are so unfair

Users do not like being told off. They post their Question and hope/want an answer. Lots of them do not read any help pages. They do not even read the help they get during the process of creating their Question. I really don't think there's anything that can be done about that. Hiding their post will even remove the ability for most other user to comment and help. Just to be clear: There is NO WAY to make people happy while expecting quality, unless they want to provide the quality. So make it easy for those that want to do that.


I didn't address the proposed auto-reopen "feature". I think that's already plenty covered by all other answers here.


You read the post, but I wasn't clear about the point

I don't think this suggestion is addressing any of the real problems, except for "Score and voting capabilities are removed". I think it'll create more.

25
  • 21
    If I told you that the real problem was that people just ask questions here without having done their own homework (e.g. research), or give us a requirements dump, or give us incomplete code or problems to work with, would you then think that the problems are going to be at least somewhat redressed in this proposal? People may not like getting off on the wrong foot with asking questions, but the reality is any other approach that deals with one-on-one interaction with an OP will not scale. This solution at least scales to deal with the volume of questions we get daily.
    – Makoto
    Commented Mar 28, 2020 at 17:12
  • 30
    @Makoto I'm suggesting to make it much easier to find relevant information for everyone's benefit. I don't see where I propose something that doesn't scale. The easier it is to find information and get training, the more reviewers will join in and curate.
    – Scratte
    Commented Mar 28, 2020 at 17:40
  • 68
    This post captures the true issues behind well-meaning newcomers feeling unwelcomed by well-meaning oldtimers.
    – philipxy
    Commented Mar 29, 2020 at 6:04
  • 9
    @philipxy In that context I feel the term irony fits to describe the downvotes :)
    – Scratte
    Commented Mar 29, 2020 at 22:04
  • 22
    With respect - I'm absolutely certain you (and your experience) are an exception to the rule: Most newbies asking questions which get closed don't even know what meta is and haven't gone through the tour properly if at all; and while the problem you bring up does exist, I (and probably others here) believe it is orthogonal to the rest of this discussion. Not downvoting you though. Also, I very much agree that the phenomenon of many downvotes accruing on closed questions is in itself a problem worth addressing.
    – einpoklum
    Commented Mar 30, 2020 at 19:28
  • 8
    @einpoklum I think you're right. Nobody in their right mind would ever put this much effort into a voluntary thing with little reward. But the first month I spent here I had no idea what meta was. I just answered Questions. Some of my answers were deleted with the Question and I lost my reputation on them. I didn't understand why they were removed. Some times I spent an hour getting an answer ready and just as I was about to post it, the Question got closed. I didn't understand why. I didn't ask Questions because I didn't understand why. I think that first month may be very representative..
    – Scratte
    Commented Mar 30, 2020 at 19:33
  • 5
    @Scratte: It took me quite a while before I started answering questions. Also, SO is very harsh relative to other SE communities when it comes to voting. Some other communities will more delicately nudge you to edit-or-delete less-useful answers, and are more likely to reward a newbie with an upvote for a half-decent question.
    – einpoklum
    Commented Mar 30, 2020 at 19:49
  • 4
    @einpoklum I know about the harsh voting. I just deleted my only Question today because it's too easy a target when someone is upset with me. So I've removed it only to avoid a Question-ban. But new users are not aware of possible Question-bans, nor the requirement for quality. I think that's where the focus should go. And hiding away options, pretending that it's OK to ask a new Question (instead of fixing the old one), or presenting canned short suggestions, is the wrong way to "fix things" in my opinion. I was actually just lucky in that I already liked to present things nice and complete.
    – Scratte
    Commented Mar 30, 2020 at 19:59
  • 13
    You are right. "Sometimes there's a consensus among experienced users" is a primary failure of the closure system as it exists today. Too many close reasons are combined, open to interpretation, or are too vague. The result is an interpretation of rules that not only creates friction in the community but also effects the quality we all clamor for. We desperately need to refactor the close reasons to be clear, concise, and not open to interpretation.
    – Travis J
    Commented Mar 30, 2020 at 23:06
  • 43
    @Scratte You're exactly the kind of user we want here on Stack Overflow.
    – Ian Kemp
    Commented Apr 1, 2020 at 22:32
  • 8
    There's a lot here that's worth listening to. If SO was presenting something like this as an issue that needs to be resolved, I think we as a community would feel a whole lot more respected and would have a lot more to contribute to fixing it. That said, I do think you're somewhat... off-track. Rather than hunt down every last rule and keep hundreds of pages of notes, you need to focus on instead understanding the core principles and developing your intuition about how to apply them. Quality is inherently subjective, so it's okay if we disagree on particular examples. ... [cont]
    – jpmc26
    Commented Apr 2, 2020 at 19:18
  • 5
    [cont] ... What's important is that we have shared basic principles, and there is a large degree of consensus on those. They are also largely encoded in the Help (or at least used to be) and maybe a scant few important blog or Meta posts, but I would agree they could be made plainer. What's really confusing now is that SO itself seems to have largely rejected those core principles over the past year or so, so it's not surprising you're confused. Anyway, work on boiling down all the things you've read to a set of core principles that leads you to the conclusions you're seeing on Meta. [cont]
    – jpmc26
    Commented Apr 2, 2020 at 19:24
  • 3
    @Pureferret That is a separate thing. It's not helping new users to just tell them that their contributions are bad. They need to know why and what to do, and it would help them to learn before they make the contributions through better information. Keeping their posts locked and pending until they are fit may also discourage "I'm feeling lucky"-type Questions and reduce the huge amount of posts that should have been closed, but never were. A lot of those posts have answers that often starts with "I think this is what you want..."
    – Scratte
    Commented Jun 8, 2020 at 10:17
  • 6
    It's funny: I cannot relate to your woes. The general idea of what constitutes a valuable question is not hard to grasp and is not surprising. It certainly does not require hours of Meta crawling. For coding questions, the vast majority, the rules are very general (not at all SO specific): Don't expect us to do your work; do not ask what's been asked before (lmgtfy); and then the usually necessary core information: Input, expected output, actual result, any diagnosis, and an example if possible. And, yes, you need to have tried something. What about that is hard? Commented Jun 9, 2020 at 12:04
  • 3
    Nothing @Peter-ReinstateMonica. You're talking about simple debugging Questions. It's all the other types of Questions that are tricky: Like how-to Questions. Sometimes they get a score of -12 and are deleted outright. Other times users are fighting over whether or not to close them or open them. Other times they are highly upvoted. This is very confusing. And what exactly do I need to have tried if I want to ask a how-to Question? Can I ask a Question about what is the difference between A and B? Or how do I improve on this code? Or can I get this error in other circumstances than X?
    – Scratte
    Commented Jun 9, 2020 at 12:11
149

while making it easier to edit and reopen closed questions

Please keep in mind the need in this case to make it easier to re-close questions, too. More on that below.

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

Has there been a study done to see what of the 97% of un-reopened questions (or 80% of un-edited questions) you think deserve to be reopened? E.g. is there an actual problem with only 3% of questions being re-opened? It could well be that that's actually a good number of questions to be reopened from a quality perspective.

Askers are often unsure how to get their question reopened and onsite guidance can be confusing. They often experience a pile-on effect of downvotes and comments while trying to edit their question in parallel. This can feel overwhelming and intimidating for question askers and drive-by visitors.

Some better guidance and/or handholding for askers whose questions get closed (by others; self-accepting a recommended duplicate should be left out of this basket, as one example) would go a long way. The other hurdle is getting askers to read the guidance already given to them...

Curators are burdened with an enormous backlog of tasks. We want to reduce the burden by helping question askers improve the quality of their question through product guidance, relieving curators of this tedious and often thankless task.

Part of the reason we have an enormous backlog of tasks is that we are artificially rate-limited at many curation tasks. Specifically, the number of close votes we can cast per day and the number of review items we can cover per day. If we were given 100 close votes per day instead of 50, or if the Close Vote Queue's pile of 40 close votes were separate from our pile of 50 close votes per day, I imagine the backlog would drastically decrease.

Shog long held that the reason for never* upping these limits was to prevent burnout, but I have never bought that. Additionally, these burnout concerns somehow don't matter when it comes to moderators, who are allowed to do as many of each task as they want.

I hear some people also routinely run out of flags. While I do use flags almost every day, I personally have never gotten close to running out once I hit 100 flags per day. I'm guessing those that do are probably members of Charcoal or power users of SmokeDetector, at least.

* There was one test of raising the limits a year or two ago. As far as I remember, the test was a pretty clear success. Unfortunately, an increased cap never got implemented in a more permanent manner.

We will also be testing automatic reopening

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once.

Oh god no, please don't do this. There's no good reason to open a question automatically. If you do this, you have to let those close voters who closed the question vote a second time, and you should notify them about the question being reopened, too.

putting the onus on the question asker to improve and reopen their question and explore the notion of “hidden” as opposed to “closed” questions, giving askers the space to (somewhat more) privately improve their questions. More details on these concepts are below.

We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed. It’s a gentler way to remove a low quality question from general visibility while giving the author a chance to edit it. Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance.

I think this is a pointless change made under the (failed) guise of "being nice", personally. Telling me my question is hidden isn't any nicer than telling me my question is closed. However, I think it is a good idea to hide closed questions from question pages/lists for users under at least 3,000 reputation.

To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted. And to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

Can you detail these criteria, specifically how they compare and contrast to the existing conditions for automatic deletion (e.g. the Roomba)?

Phase 3 - Score and voting capabilities are removed.

Please don't remove the ability to vote on a question while it is closed. If the reason you want to do this is to prevent an overwhelming number of downvotes, simply implement a floor for the displayed score... e.g. -1 or -2 or something like what was tested last year by Shog (successfully, I might add). Downvoting is one of the most important curation tools that we have. It also affects the ability to delete vote a question or whether the question will Roomba.

Some things that weren't covered by this question:

  1. Please consider adding more regex checks (yes, yes, I know regex is dangerous and bad all you programmers reading this) for questions at the asking phase for off-topic reasons. Some examples of phrases in a question body or title that should at least show a warning banner (saying something like "opinion-based questions are off-topic. please do not ask them") before a user is able to ask the question:

    • best practice(s)
    • idiomatic
    • how should I
    • what do you recommend
    • what's the best

    etc.

    Probably somewhere in the region of at least 40% of the 40,000 close votes I've cast on Stack Overflow were on questions that contained one of these phrases or similar.

  2. Is there any effort still on expanding the close reasons for which gold tag badge holders are able to unilaterally close (Mjolnir) a question? It's been wanted for years, and we were just starting to get a hint that it might be coming last year as Shog started testing changes to the closure process.

34
  • 11
    increasing limit of close votes and reviews is unlikely to help: we tested limit of 60 before and results weren't much encouraging (my own experience was that 50-60 votes could be too much to me - even though I picked easiest, low hanging fruit during this experiment). Other point in this answer look good to me
    – gnat
    Commented Mar 27, 2020 at 21:24
  • 6
    @gnat agreed on the close vote limit being too high. Well, again - for me. Although, I do agree with TylerH that something should be done. Splitting the close votes into "for the queue" and "outside" seems quite good, actually - I avoid the close vote queue because I ran out of votes fast. Then it's really frustrating to later see a question that is off-topic and I cannot VTC. I also don't want to do the close queue last thing in the day because I don't want to plan my day around this. Leaving some flags is sometimes not enough. It's just overall too annoying to use and manage.
    – VLAZ
    Commented Mar 27, 2020 at 22:23
  • 7
    The floor to the displayed score was already tested and considered ... not helpful. It wasn't in any way "successful". It was confusing and a bad UX - meta.stackoverflow.com/questions/393907/…. If a closed question is automatically deleted after n days, why does the score matter? Why do you need to add more downvotes if the closure is enough to get it deleted?
    – Catija Staff
    Commented Mar 28, 2020 at 3:27
  • 7
    @Catija to trigger rate limiting, IP level blocks and impact triage heuristics maybe?
    – gnat
    Commented Mar 28, 2020 at 4:00
  • 3
    @Makyen Well, I agree that something has to be done. I want to stress this: for myself, it would be useful to decouple the votes. Because for me personally, I don't want to juggle how exactly to divide my close votes. Upping the overall limit also works to an extent but it still requires each user to "divide" the close votes as they see fit. Which is a valid solution, but it's not something I myself want to bother with. Does that clarify the paradox? There is no paradox - it's just my own preference to have a split vs having a single pool of votes.
    – VLAZ
    Commented Mar 28, 2020 at 9:10
  • 1
    @Catija If your plan is really just "Roomba will have relevant deletion conditions consolidated into one: delete hidden questions after 7 days" or something, then the need is lesser. Though I would still say that downvoting should not be disabled, because people downvote for all sorts of reasons. This announcement also doesn't touch on the fact that posts are already hidden from the question list at -2 score (or is it -3? I forget).
    – TylerH
    Commented Mar 29, 2020 at 16:09
  • 8
    My strong suspicion is that a lot of automatic reopen votes (i.e. due to the first edit after closure) are actually cosmetic or otherwise not resulting in reopen worthiness. This might well account for the fact that 85% of edited questions remain closed. A later substantial edit would then end up in ... nowhere. Unless people actively see the edit and vote to reopen, the post is dead. Currently there's one chance for reopening, and most people blow it, unknowingly. Would be helpful to see the statistics on this.
    – Adriaan
    Commented Mar 30, 2020 at 7:34
  • 3
    I think you're missing the point about being "nice." Nobody likes to have their work criticized, but what people really don't like is being publicly shamed, which is what the current system effectively does. You seem to recognize that when you agree that hiding closed questions from the view of users with less than 3k reputation is a good idea. The change from "closed" to "hidden" directly reflects this move away from a public shaming approach. It's not just nicer wording; it reflects a different practice that is fundamentally more humane.
    – senderle
    Commented Mar 30, 2020 at 11:25
  • 1
    @senderle I think hiding closed posts from the views of users with less than 3k reputation is a good idea not to shield the asker from shame but because users with less than 3k reputation can't do anything productive WRT the question. They can't answer it because it's closed, and they can't vote to reopen it, either. Maybe drop the threshold to 2k so that users with edit privs can at least make an edit.
    – TylerH
    Commented Mar 30, 2020 at 13:05
  • 2
    @senderle So what's holding "us" back from informing the users about the option to delete their post while they find out what they should do with it? Apart from that, I just deleted a Question of mine and I get this when trying to edit it "You cannot edit your own deleted question. Undelete before editing.". But if I wanted to edit this post, I could edit with my own favorite editor, then undelete the post and put in the edit. This way it's not exposed in it's horrible state for more than a few minutes. At least this gives me the control of reducing downvotes.
    – Scratte
    Commented Mar 30, 2020 at 15:06
  • 3
    @senderle Closed questions are not broadcast across the site as you seem to be claiming. Users have to look for them and then willingly navigate to them from question lists/searches to find them (same as pretty much any question). On top of that, the majority of users do not assign a real or identifiable name to their account. The concern of "public shame" doesn't have much ground to stand on here.
    – TylerH
    Commented Mar 30, 2020 at 16:49
  • 1
    I take issue with flagging "idiomatic." While many operations do not have idiomatic approaches, it's extremely important to follow the standard norms and idioms of a language when writing code. These norms are typically based as much on the practical implications of a design choice as they are stylistic ones, and they generally have such a wide consensus that calling them "opinion based" is an exaggeration.
    – jpmc26
    Commented Apr 6, 2020 at 19:17
  • 1
    @jpmc26 If you can ask about how you should write code in a way that can be answered with data, examples, and objective content, then sure. But if you're just asking "what's the most Pythonic way to write this (typically code that works just fine)", then sorry; that's Primarily Opinion-Based all day long.
    – TylerH
    Commented Apr 7, 2020 at 0:18
  • 1
    @TylerH Writing sum(my_list) is objectively more "Pythonic" than sum = 0; i = 0; while True: sum = sum + my_list[i]; i = i + 1. "Most Pythonic" questions are problematic not because they ask about standard idioms, but because "most" is unquantifiable. They can easily be reformulated into, "What is a Pythonic/idiomatic way to [perform small task X]?" Those questions are answerable, and they can lead to good presentations of the pitfalls of the tools involved and how to avoid them or account for them. In other words, discussions of why and not just how, and that's the kind of post we want.
    – jpmc26
    Commented Apr 7, 2020 at 1:31
  • 1
    @TylerH I'm saying most, or at least very many, questions about idiomatic code fall in that category or can be easily edited to fit into it, thereby making auto-flagging them a bad idea.
    – jpmc26
    Commented Apr 8, 2020 at 2:45
115

Please, do not implement automatic reopening!

As a curator I do not want to see more posts being reopened, I want to see more questions getting closed and deleted. I also want to see better quality questions being asked, so that I do not want to close them.

You could solve this by giving us more close votes, more delete votes, greedier Roomba and one-vote close privileges to gold badge holders.
Currently a lot of posts of low quality are hammered by gold-badgers to prevent them receiving any answers. In combination with downvotes this will trigger Roomba. I imagine if gold badge owners had the same privilege for other close types, the process would improve a lot.

I could easily go and spend all my 50 close votes and all my delete votes. I could easily spend 3 times as much as I have. The problem is I do not want to waste my time casting my votes there, because it is unlikely I will get 2 more people to see that question and cast their precious votes too.
What works better are down votes, because they are binding and carry more power. Ideally I should probably both close and down vote questions, but if I see a question in low-traffic tag I only downvote and follow the question to see if gets deleted by Roomba before someone else finds it and answers it. The end goal is the same, the question is deleted.

There should also be better guidance for new users what this site is about. We are not some free user support. I do not want to answer the same question again and again a million times, because plenty of users have the same problem.

How about every question starts in closed state and gets opened once it receives positives score? I actually think 1-5% of all new questions are good questions.

Newcomers are often frustrated not because their question gets closed, but because they receive no help they wanted. What follows is they ask the same question again, edit some beef into the body or simply rage in the comment section about us providing them with too much guidance. Once they ask them same question 3 times they receive question ban and start complaining how unwelcoming Stack Overflow is.

The site is not the problem, the new users are the problem. Why do I get revenge downvotes after commenting on NAAs? Why do I get mean comments when I provide guidance in the comment section under a question? Why do I get threats when I edit noise out of the posts?

The curators acquire patience and better people skills with experience and we are more welcoming the more experience we have working with this site. What tests our patience are all these questions, which keep coming and there's either no way to answer them or they should never ever be answered here.

4
  • 3
    "How about every question starts in closed state and gets reopened once it receives positives score?" Not a practical approach for low-traffic tags. And there are questions I'd answer as I understand them, without feeling enough effort has been put in (whether research, formatting or language) that I feel they need upvoting. Commented Mar 28, 2020 at 11:05
  • 20
    @cindy That is the problem. We answer low quality questions without improving them and then there's no way for me to delete those. If you think the question with your answer is going to be useful to the community edit the question and upvote.
    – Dharman Mod
    Commented Mar 28, 2020 at 11:11
  • 5
    I do not think you will find much room for agreement with the line "new users are the problem"; Stack Overflow is a business, and the current zeitgeist of the business world is 'constant growth', even if that is really a toxic viewpoint. I think you'll have a better chance if you focus on why or in what way new users are problematic.
    – TylerH
    Commented Mar 29, 2020 at 16:13
  • 2
    @Dharman There's a lot of middle-ground questions though. They're complete and answerable and a valid question. Often I kind of think more research should have been put in (but maybe that's because I know the topic well). It isn't useful to me so I don't think it justifies me upvoting it, but equally if it's the answer is obvious and quick then why not give it?
    – DavidW
    Commented Mar 29, 2020 at 19:05
103

I predict that this part of the feedback will cause more harm than good:

enter image description here

And that's because it...

  1. encourages users to take the easier path, that is, drop the current question (which is closed and most likely downvoted) and ask a new question about the same thing (which is something users already do).

  2. is a bit misleading in the part where it says "There's no penalty for asking more questions". Well, although that's technically true (there is indeed no penalty for asking new questions), there is, however, a penalty for the habit of leaving downvoted/closed questions to be deleted by the system and moving on to creating another post (possibly about the same thing) which is something this message fails to address.

The same wording is also used in post notices as indicated by @rene in the comments.

Can we either remove or reword of this part?


Hmm...

Example

5
  • 7
    That wording is already present in some post close notices today but I managed to get a small change in: meta.stackoverflow.com/questions/394552/…
    – rene
    Commented Mar 28, 2020 at 8:30
  • 37
    "There's no penalty for asking more questions" - question bans tho. Asking a sufficient amount of bad questions actually does come with a penalty. Unless that's being removed, in which case we have even more problems Commented Mar 28, 2020 at 12:18
  • 29
    @Zoe not only that, there's a good chance the new question will end up being closed as a self-dupe - and downvoted just for the exact behavior that's being encouraged here. Commented Mar 28, 2020 at 20:39
  • 6
    I'm sure many of us have seen people who re-ask questions because they received comments they didn't want to awknowledge or for whatever other reason were disappointed by the attention they received. It's difficult to realize these are duplicates unless you happened to see the original one and noticed they were similar. So, I think imploring people to try and re-open their questions is better than asking them to make new questions. Commented Mar 31, 2020 at 21:34
  • 1
    Perhaps "ask a different question about a different topic" could remove that misleading hint that some people will read as "repost this". But yes, probably better to just remove any discussion of new questions because that's not going to help them get a solution to the thing they were asking about, which is presumably their current goal for interaction with SO. Asking about it some other way is likely going to be a duplicate. Commented Apr 7, 2020 at 11:27
86

I'm going to take a different tack here

We need to communicate better with users about closure

About 6 months ago Stack Overflow made the blue closure blocks. No matter how you slice it these blocks were arguably worse in many cases

We discussed this in detail a few months back. While the new closure reason is nice, it really needs to have a link to the help center page in the close reason.

Which was responded to with

The post notice currently links to the /help/closed-questions. We could replace this or add another link but we think that two links to documentaiton in a post notice is too much. So we're thinking we should just include a link to the the MRE page within the new close UX for this close reason.

The blocks are often unclear. Take this off-topic question about setting up wifi. The closure block says this

We don’t allow questions about general computing hardware and software on Stack Overflow. You can edit the question so it’s on-topic for Stack Overflow.

There's two massive problems here

  1. There's no way to "edit the question so it’s on-topic for Stack Overflow". It's about hardware
  2. It no longer tells the user to ask the question on Super User or Server Fault, where it would likely be on topic.

Or take the new MCVE/MRE/minreprex message

Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.

At which point, the link dumps them onto the main help page and expect them to figure out what to do from there. I mention all this because your #1 commits the same mistake. You have this massive EDIT button but you offer the user no guidance what to do with it. I don't know why there is an aversion to directly telling people what we expect. The page offers guidance that, if coupled with the new email notice, could guide people to edit a MRE in.

Worst of all is that you close the email with something really REALLY untrue

Deleted questions can't be answered. You can always ask a new question, though! There's no penalty for asking more questions as long as they are on-topic and answerable

There really is a question ban and it is triggered by asking too many downvoted and deleted questions. So the email is basically advising them how to get a question ban faster. Worse, if they find that FAQ entry, the first thing it tells them to do is undelete the questions (which they may not be able to do if Roomba has done it) and clean them up. They're not going to be happy, or likely to ask more questions.

Give tag badge holders a one-shot reopen for edited questions

If we're going to have faster reopens, make it possible for people with a silver or gold badge to review the edited question for a reopen. Make it easy for them to do this (the ability already exists in the Close Vote and Reopen Queues, it's just not obvious). Toss out some badges for people who pitch in on their silver/gold reopens here. Do not let a single edit reopen. I think this point has been covered in detail.

7
  • 9
    "At which point, the link dumps them onto the main help page and expect them to figure out what to do from there. I mention all this because your #1 commits the same mistake. You have this massive EDIT button but you offer the user no guidance what to do with it." I agree with the entire post here, but I want to piggyback on this - I'm also very irritated at this. Hence my suggestion to allow people to set out actionable things for the OP. Instead of a blanket "something is wrong with the post", we should be able to specify what is wrong.
    – VLAZ
    Commented Mar 28, 2020 at 9:26
  • 1
    @VLAZ You can always add a comment, or use a custom close reason, that describes what you think is needed in the post, so what you're asking for already exists. Is is just that you're wanting a way to do that anonymously?
    – Makyen Mod
    Commented Mar 28, 2020 at 10:39
  • 6
    Not necessarily anonymously. Whether it's identified who asked for the improvement or not is irrelevant - it's a way to add more guidance for the question asker when they are editing the question. Because you've explained it wekk - the current guidance is not specific enough. And yes, we do have comments but they are not always followed. And sometimes that's the fault of the question asker, sometimes it's the fault of the comment not being clear enough. Linking to "here is everything that MIGHT be wrong with your post" help page is not helpful guidance, "you're missing relevant code" is.
    – VLAZ
    Commented Mar 28, 2020 at 11:16
  • 5
    Give silver badge holders a one-shot reopen for edited questions is a great idea, and it should be done as soon as possible.
    – Travis J
    Commented Mar 31, 2020 at 16:29
  • @VLAZ I played with the thought of also telling people where something is wrong, in addition to what specifically is wrong. Maybe something like an annotation system could be valuable: in the question, select some paragraph, sentence, word, variable name, or whatever, then pick one canned comment like “This part is unclear, because [… type reason here]” or “It’s unclear what the value of this variable is; please edit and include its definition” or something like that. Has something like this been suggested on Meta? Commented Apr 2, 2020 at 4:05
  • @user4642212 All I can say is that I've not seen such a suggestion. That doesn't mean it hasn't been asked for. I get what you mean, though - like the notes feature in Google Docs where users can annotate parts of the document with comments. You can certainly suggest it, if you wish - either here or maybe as a new question. I'm sure you'll get linked to a dupe if one exists.
    – VLAZ
    Commented Apr 2, 2020 at 5:41
  • @user4642212 - In other words, code review for questions.
    – bta
    Commented Jun 10, 2020 at 1:59
69

The proposal is based on multiple doubtful premises

Our research has shown that often users hesitate to ask questions on Stack Overflow due to the fear of public, and sometimes caustic, feedback.

And that's a bad thing?

Certainly, users should not have to fear caustic feedback, but I think that such a concern is more a PR problem than anything else, especially these days. However, I challenge the apparent premise that users shouldn't hesitate to ask questions on SO at all. I can certainly see how SO corporate might take such a view, but even from that direction I think it's at least short-sighted. I hesitate to ask questions on SO,* albeit for altogether different reasons, and that's as it should be.

The thing that most distinguishes SO from the many other get-your-tech-questions-answered sites is the cadre of experts that has coalesced here to respond to and curate questions. It is because of these volunteers and their work that SO has a reputation for good answers. A long-time major concern of many of these people is how many poor questions are asked, so now Corporate wants to make users less concerned about posting questions that fail to meet SO standards?

Furthermore, to the extent that the reasons for the specific exercise include reducing review-queue pressure, the general attitude that we want to reduce users self-filtering seems counter-productive. Perhaps implementing the proposal would reduce the volume of the reopen queue, but that queue is more or less under control already, and it is one of the easier ones to review, anyway. On the other hand, focusing on reducing barriers to question-asking seems certain to increase the load on the triage, first-post, and edit queues, at least, which are more of a problem to begin with.

We want users to hesitate to post questions, at least long enough to think about the question itself, the way they are presenting it, and what their other options are for getting an answer may be. All segments of the SO community should want this as a means to manage the limited resources of the volunteers that make the site work. SO's efforts in this direction, such as the question wizard, have born at least a little fruit, but that does not mean we have any slack to play with.


From curators, we often hear about the time-consuming burden of manually guiding users whose questions get closed, and sometimes receiving pushback when they are just trying to help.

Then the proposal is for a how-to-get-my-question-reopened wizard? Alas, no. Elements that might fit in such a wizard do appear in the proposal, but in most respects it's still the same process, except

[Only] Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance.

In other words, the available pool of people who can provide assistance in this area will be reduced, they have to jump through extra hoops to do it, and there is no reason to expect that their attempts to help will be any better received. I don't see a single reason to think that these characteristics would make anything better for anybody.


And from moderators, we’ve heard the ask to reduce toxicity for all users through improved moderation and communication tools.

I keep hearing these words "toxic", "caustic", and similar. Frankly, I rarely see any such thing other than what users -- typically little-engaged ones -- bring with them. The story usually plays out with a user receiving negative, but acceptably polite, feedback about a question, answer, or comment, and they then responding in a hostile manner. Sometimes it builds over an extended interaction, and sometimes users who should know better are drawn in.

Askers don't like being told, however nicely, that their question does not meet our standards. No amount of guidance about how to bring it up to snuff -- when that's even possible -- mitigates that very much. They don't like being told that it has already been answered, especially when the dupe question is not a near-verbatim copy of theirs. Many seem to dislike anything other than "here's your answer".

SO has both a cultural practice and explicit mechanisms for delivering such messages, as it should. These address SO's quality objectives. That means that yes, sometimes users' interactions with SO are not to their liking. This seems most often to be the case for users with low engagement in the community, and it is common enough to have become part of SO's reputation. But it does not mean that SO itself is "toxic" or "caustic", and there's certainly no reason to think that lowering the barrier to entry, or at least appearing to do so, will improve the situation in any way.

With respect to moderator perceptions specifically, I observe that mods have much greater exposure than most users to those excessively negative interactions that do occur, because that's part of moderation. I'm certainly in favor of tools that make their lives easier, happier, and less stressful. However, I don't see anything in the proposal that I would expect to have that effect. Have mods actually asked for any of the proposed specifics?


"Lies, damned lies, and statistics"

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened.

I'm sure these statistics are accurate, but they don't tell an actionable story. In particular, the position that

We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

seems premised on the suppositions that

  • a significant proportion of questions that go without being reopened could have been rescued, if only the OP had been willing / able / motivated to fix them;

  • post-closure edits generally represent good-faith attempts to rescue the question; and

  • those post-closure edits that do represent good-faith attempts to rescue a salvageable question have a poor success rate for reopening.

Some or even all of those premises might in fact hold, but I'm inclined to suspect not. It would be prudent to test them before devoting substantial time and resources to a project that looks to them for motivation and direction, even if the proposed features constituted a clear win.

Some proposed features are ill-conceived

We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed.

I'd be more open to creating a new "hidden" status (but see also below), but I am completely against changing the existing "closed" status to have different significance and different effect on questions. For example, the longtime accepted wisdom about questions closed as dupes is that it is useful to keep them (visible). There are also closed questions that we want to keep visible because of import and / or historical significance, even though they are no longer considered on-topic. And generally speaking, closed questions that are not anyway eligible to be roomba'd often have some valuable attribute or other.

Consider also that we already have a "hidden" status. We call it "deleted". We don't want existing deleted questions to become eligible for resurrection, but functionally, "hidden" is a better description of this status than "deleted". Simply enabling those users who can see such posts in the first place (owners, high-rep users, and mods) to comment on them would produce something very like the proposed hidden status. If an existing status is to be converted to "hidden", then "deleted" is a better match. Perhaps all that is needed in addition to the commenting is to add a flag to indicate whether the question is eligible for reopening.


When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state.

Absolutely not. Perhaps there's room for some other variation on expedited reopening, but there are too many problems with automatic reopening to count. Among them,

  • it is easily exploitable
  • it will make more work for the community in many cases
  • it can be triggered by edits not intended to fix the question in the first place
  • it will often expose users to more of the kind of negative interactions that the overall proposal seeks to avoid, as their self-reopened question is rejected for a second time

Additionally, the whole idea seems to be based on the dubious proposition that a desire to avoid public scrutiny during question revision is a significant barrier to people fixing their questions. Data please?


*presently 17 questions vs. >4200 answers on SO

12
  • 15
    Your point about hesitating to post questions is so relatable to me! Every time I run into a programming problem I can't solve with Google, I ask myself over and over "is this suitable for SO?" and I will only start writing the question if I'm 100% sure. And then I spend about an hour writing the question, picking all the right words, showing everything I tried, my thought process...
    – Sweeper
    Commented Mar 29, 2020 at 16:03
  • 9
    The problem with that view point on hesitation re: asking is that it requires high standards, and most people who are new do not have the same high standards. Because the company wants to maximize new users, that means high standards (read: "barriers to entry") are bad. We saw the ugliest form of this business viewpoint when the Meta FAQ on 'when should I post a question on Stack Overflow' was unilaterally (and without warning) hijacked/defaced by a staff member to give the complete opposite guidance from what Meta had decided upon.
    – TylerH
    Commented Mar 29, 2020 at 16:25
  • 2
    @Sweeper I used to do that too. But then I spent much more time finding an answer without posting either. Now I've stopped writing my Questions up, because I don't want to risk a Question ban.
    – Scratte
    Commented Mar 29, 2020 at 16:40
  • 1
    @TylerH I can't seem to find the FAQ post that you are talking about. Link?
    – Sweeper
    Commented Mar 29, 2020 at 16:48
  • 4
    @Sweeper see meta.stackoverflow.com/questions/261592/…
    – TylerH
    Commented Mar 29, 2020 at 19:22
  • 5
    As I said, @TylerH, I can see how Corporate might suppose that they should minimize the barrier to entry, but that's short-sighted. It produces a worse experience for everyone, including, I'm inclined to think, Corporate. Commented Mar 29, 2020 at 19:47
  • 3
    @JohnBollinger Of course, I agree. Endless growth of userbase is a toxic viewpoint. Companies should find their niche and stick to it.
    – TylerH
    Commented Mar 29, 2020 at 21:51
  • 3
    "Additionally, the whole idea seems to be based on the dubious proposition that a desire to avoid public scrutiny during question revision is a significant barrier to people fixing their questions. Data please?" I don't see how even proving this to be true would matter. SO is a public resource readily accessible to everyone on the internet. It was founded explicitly for the purpose of publicizing information. If you're not willing to deal with public scrutiny, your desires are fundamentally incompatible with the site's mission. Why are you even here in that case?
    – jpmc26
    Commented Mar 30, 2020 at 15:27
  • Agree with this, except that I think it's perfectly ok to have an artificial separation of a "hidden" and "deleted" status if it has some positive psychological effect.
    – einpoklum
    Commented Mar 30, 2020 at 19:04
  • 2
    "Simply enabling those users who can see [deleted] posts in the first place (owners, high-rep users, and mods) to comment on them would produce something very like the proposed hidden status."—yes, please! I've been prevented from helping users simply because their question was deleted dozens of times. A comment can make all the difference.
    – Chris
    Commented Mar 31, 2020 at 13:59
  • 1
    Absolutely it's a bad thing. StackOverflow has become a virtual "meme" between myself and my colleagues, because whenever we post a legitimate, well-thought-out, eloquently-written question it's very often closed by some over-zealous editor who thinks it's a duplicate or an "opinion-based" question. I absolutely agree with relaxing the ability to re-open questions.
    – WackGet
    Commented Apr 2, 2020 at 16:09
  • 2
    @WackGet, if questions are routinely getting closed that should never have been closed in the first place then that is indeed a bad thing. In fact, I would rate it a considerably worse thing than what we're actually talking about, and one that would be a more worthy subject of Corporate's attention. Commented Apr 5, 2020 at 12:19
63

After reading this, my first question is the same as what Kevin B commented - if questions are reopened automatically, does that mean that people have to vote to close (or hide) it again?

Sometimes it's hard to get traction to close a post even after the close vote count was lowered to 3. On popular tags, "old" questions (say, posted more than 10 minutes ago depending on the current activity) may have a hard time even getting to 10 views. So, what happens if:

  1. If the question gets barely closed because very few people see it in the first place.
  2. A non-significant edit is done which automatically reopens the question but doesn't fix the problems with the post. (there are MANY such edits in the reopen queue).
  3. The question appears in its pre-closed state...but not part of the newest questions.

The question might have a hard time getting another 3 close votes and views. And this is going to be frustrating for anybody who is involved with the question - the asker and the people who want the question to be improved.

8
  • 54
    Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened This indicates that around 86% of edits post closure are insufficient to make a post re-openable. Automatic reopening may well result in a huge proportion of false positives that then need more curation effort from the community to revert. Commented Mar 27, 2020 at 20:38
  • 6
    @CertainPerformance I completely agree. It would make sense to make the question public again, but still closed.
    – Dharman Mod
    Commented Mar 27, 2020 at 20:42
  • 20
    @Dharman to be honest, I'm really not sure what's best here. The reopen queue is full of questions that were edited after being closed but the edits don't actually address anything (e.g., fixing few spelling mistakes but not the lack of code). On one hand, taking those off the queue would be welcome. On the other, if that means that reopened questions would still be bad, then...we just merely turned needing 3 VTCs into needing 6.
    – VLAZ
    Commented Mar 27, 2020 at 20:49
  • 1
    @Dharman Then there is the "hidden"/"closed" thing - as far as I understand they are the same now closed is hidden, so it can be edited in peace. If a post is then subsequently shown but still closed...then what can we do about it? The post author can't do anything, and people can only vote to re-open. Which is basically what we have now but with a bit more enforced "quite time" for editing, while the post is hidden.
    – VLAZ
    Commented Mar 27, 2020 at 20:49
  • 2
    Interesting metric for "old" questions there... I consider questions old once they are over about 6 months to a year old :-P
    – TylerH
    Commented Mar 27, 2020 at 20:50
  • 2
    @Dharman That's a great idea (make the question public again but still closed) that meets the proposal halfway from the current implementation.
    – TylerH
    Commented Mar 27, 2020 at 20:51
  • @VLAZ What would happen is that the post which do not get edited in the quiet time are never shown to me again and they will be removed properly from the site. If someone made proper edit I don't mind taking a second look and voting to reopen.
    – Dharman Mod
    Commented Mar 27, 2020 at 20:52
  • 4
    @TylerH as I said, depends on the traffic. I see questions that get to just abut 10 views in 5 minutes then people just stop opening them. Even days later they might get only a couple of more views. I've also seen clearly off-topic questions hang at 1-2 close votes for a week without getting more VTCs just because nobody looks at them. So...yeah, some questions get old fast.
    – VLAZ
    Commented Mar 27, 2020 at 20:57
56

Most people don't know that the first, and only the first1, edit after closing (if not by one of the close voters) puts a question into the reopen queue. OPs and want-to-be-helpful third party-editors already make small and/or cosmetic edits2, getting a question into the reopen queue, where it subsequently gets voted to remain closed, and rightly so. After that, no edit, however substantial, will ever automatically push the question into the reopen queue. I therefore read the automated reopen proposal as "Let's just skip the reopen queue altogether, shall we?".

Why not fix the ways posts get into the reopen queue, by e.g. not allowing edits by others than the OP into pushing it there? If only cosmetic edits could've made a post on-topic, it shouldn't have been closed to begin with. It's not like the reopen queue is ever overflowing.

Another thought on this would be to let every N-th edit by OP, or every K-days when edited, a maximum of M-times etc, push the question into the queue. Care would need to be taken of course to prevent abuse.


1 Reopen votes will also put a question into the reopen queue, but I'm considering the automatic process here. The premiss of the automatic reopening under consideration in this post is that apparently doing it via vote isn't going fast enough, hence I'm ignoring that for the time being.

2 For example inlining images, code formatting, removing thanks etc. Anything not adding the information required to reopen the post (or removing the multitude of questions to reduce the post to just a single question, etc etc).

A good example is this edit, where the editor just removed tags (not even the garbage in the post itself), but didn't vote to close. This post now gets pushed into the reopen queue, where it will probably be voted to remain closed. No amount of editing by OP will put it back into the queue now.

29
  • 15
    I strongly agree with this. It's very rare for an edit by a user other than the OP to be able to change a question such that it's on-topic. It's even more rare for such an edit to come from a user with <3k rep. If a user with >3k rep edits such that the question is now on-topic, they should know that it's now on-topic and vote to reopen. So, the only edits by people other than the OP which make the question on-topic where the editor shouldn't be voting to reopen is when the user has <3k rep or >3k rep and has previously voted to reopen. My guess is that's a very low percentage of edits.
    – Makyen Mod
    Commented Mar 29, 2020 at 16:16
  • 6
    @Makyen if only someone with database access and affinity with the community could whip some statistics...
    – Adriaan
    Commented Mar 29, 2020 at 18:52
  • @Adriaan - How about a user, with enough reputation to edit the question without being put into a review, has the ability to put the question back into the reopen queue? This could easily be done with a checkbox. This allows for editors, to throw questions back into the queue, if they believe the question has a chance. This should of course come without any penalty if the community believes otherwise. I think the biggest failure is the fact, a reopen vote, does not push a question back into the reopen queue. A question can sit without another vote even if it's a perfect question. Commented Mar 30, 2020 at 20:51
  • 1
    @SecurityHound would you mind clarifying what you mean? There currently already is the ability to cast a reopen vote on a question once you have >3k rep. It appears as a button beside the flag, share, edit etc buttons. That will, as far as my knowledge serves, push the question into the queue. What can happen, of course, is that the consensus after the queue is "Leave closed", but that's another story. Can you give a reference that a reopen vote does not push a question into the queue? I find that hard to believe, as that'd make the feature almost pointless, both vote and queue.
    – Adriaan
    Commented Mar 31, 2020 at 7:19
  • 2
    @SecurityHound A reopen-vote will push the question into the reopen queue, if the question isn't already in the queue.There's a delay until the task which moves things in/out of queues runs. How often it runs depends on the site: 5–30 minutes; 5 on SO, IIRC. There may be conditions under which a reopen-vote won't put the question in the reopen queue, but I'm not sure what those are. I do know there are rare times when a close-vote doesn't put the question in the CV queue, so I assume there are some corner cases for reopen-votes too. Why do you believe it doesn't get put in the RO queue?
    – Makyen Mod
    Commented Mar 31, 2020 at 16:45
  • @Makyen - Once a question has been in the queue, after the first edit, after the question has been closed it doesn't get thrown back into the queue. We need to make it easier to reopen the question, through a review process, not make it automatic without any sort of review. Commented Apr 1, 2020 at 8:59
  • 2
    @Secur As an example of it working properly, this post entered the reopen queue a second time 1 hour after it exited the queue. In the case of that question, it had entered the reopen queue the first time due to an edit by the OP and the second time in response to a reopen vote. It looks like the task to move things into the reopen queue runs at approximately XX:45 and XX:15, and may take several minutes to complete (at least up to 5 minutes), so there may be a considerable delay between the reopen vote and the question entering the reopen queue.
    – Makyen Mod
    Commented Apr 1, 2020 at 10:49
  • 3
    @SecurityHound Your account on Stack Overflow has <3k rep (and you haven't lost enough rep from bounties to have once been above 3k). You can't have reviewed any posts in the reopen queue, let alone the "thousands" you claim. You have a total of 146 reviews on SO for the review queues you have access to. Your SO account has no other accounts on the Stack Exchange Network associated with it. Based on that, I'm having a hard time believing any of your claims.
    – Makyen Mod
    Commented Apr 1, 2020 at 10:59
  • 2
    @SecurityHound Primarily, I've asked you to provide an example of what you claim to be happening. Please do so. I've provided an example of it working as expected. Without an example of it working the way that you've stated you've observed, I have to assume that your observation was inaccurate (which is quite easy to have happen, given how reviews and the timeline work).
    – Makyen Mod
    Commented Apr 1, 2020 at 11:41
  • 1
    @SecurityHound Yes, that question won't get back into the reopen queue, unless someone casts a reopen vote. [Your claim was that it would never enter the reopen queue again, specifically even if it gets a reopen vote, but I've seen no evidence of that.] I, and I believe a lot of other people, agree that the current system has some serious drawbacks and is significantly broken in a lot of cases. There's quite a bit that can be done to improve that. E.g. giving question authors accurate information about what their question needs in order to be reopened is important, but is currently lacking.
    – Makyen Mod
    Commented Apr 1, 2020 at 11:57
  • 1
    @Secur and that last comment of yours is something we haven't experienced ourselves on SO, hence we would like to see an example of that, such that we can update our proposals on how to fix the review queues. We're not trying to attack or incriminate you, we even agree on that the current system is broken and needs to be fixed. We just want to be precise in what should be fixed, hence the question for an example or meta page with someone explaining and substantiating that claim of yours.
    – Adriaan
    Commented Apr 1, 2020 at 12:04
  • 1
    @SecurityHound Earlier, I linked to an example question of exactly that case, where the post re-entered the reopen queue, as the result of a reopen vote I cast. It reentered the reopen queue 1 hour after having exited the reopen queue, which it had entered as a result of being edited by the OP.
    – Makyen Mod
    Commented Apr 1, 2020 at 12:04
  • 1
    @SecurityHound Yes, you can't see it, I'm aware of that. There's not much I can do about that, other than provide an image, and claim I cast a reopen vote at 2018-01-08 20:26:22Z. I note it spent 7 days in the reopen queue and received no reviews, which may indicate there's an issue with showing the review to users. I haven't checked the reopen queue history for activity during that time to see if there was something else going on. In its previous time in the reopen queue it received 3 "leave closed" reviews within 2 hours, so no reveiws in 7 days is strange.
    – Makyen Mod
    Commented Apr 1, 2020 at 12:31
  • 1
    Sorry if someone else suggested this, but I don't have time to read through all the comments right now. Why not give the OP (or maybe everyone) a prompt after an edit asking if they think they have edited the question into an on topic state? The result of answering "yes" would be equivalent to the "reopen" button at the bottom of the question.
    – jpmc26
    Commented Apr 1, 2020 at 12:54
  • 2
    @SecurityHound The process is the same for all sites, including SU (see last paragraph). This question on Politics demonstrates that questions on other sites can be in the reopen queue multiple times (3 in that case). The first was due to an edit. The later two were almost certainly the result of reopen votes. [Note: I didn't go hunting for an example, that question just happened to be the subject of a hot meta post for that site, which I was visiting as a result of a SmokeDetector report.]
    – Makyen Mod
    Commented Apr 1, 2020 at 13:48
38

Suggestion

Allow curators to propose actionable steps as guidance for editing. This is currently done as comments but then question authors don't always follow the suggestions. I don't even want to put the blame on the asker necessarily, sometimes the comments are not clear enough, other times, there might be lots of comments and the asker might struggle following all of them. And sometimes the asker just doesn't do what people told them.

At any rate, it would be good to standardise the at least some of the requirements. These are things that are already written in the description of the close reason but...that's not always perfectly clear, either.

For example, we often ask for a minimal reproducible example. There can be multiple things that might be wrong:

  • no example at all - they need to add one
  • no data - they have provided their code we do not have the input for it
  • not minimal - the question asker has provided too much irrelevant code
  • not reproducible - the code might not work for some reason or another

And more but lets focus on these for the sake of the example. If the question gets closed with "It's seeking debugging help but needs more information. The question should be updated to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem." then any of the above might be an issue but for the asker it might not be clear which one. Often times I see that people supply not enough data or code and when asked for more, they just give all of their code which is certainly not minimal and makes the question a lot harder to understand.

So, working with the example above, it would be good if we could also more formally ask for specific things. Say, if we are voting for the "seeks debugging" reason, we could also specify "Needs sample data" which would then be added in the edit wizard. It will then show up on the side with information or a link containing specific guidance, as opposed to the blanket one.

Multiple problems might be flagged by curators as things that need to be edited and it would be helpful for the question asker to be able to review what those are.

Comments can still be used for more information but I often see then as not efficient enough.

3
  • 3
    Also roughly known as the mentorship program Commented Mar 27, 2020 at 20:47
  • 6
    @Zoe I was just thinking of doing something like VTC -> seeks debugging. Then doing (some form of) "Add improvement step" -> needs code. Or "Add improvement step" -> explain the problem, what is happening now, what's supposed to be happening. Then those get shown in the edit wizard as a checklist the user can go through while editing. So, the curators are more hands-off - they can just specify more specifically why a given close reason applies, so the asker can then be better informed what to improve.
    – VLAZ
    Commented Mar 27, 2020 at 20:59
  • 4
    @Zoe The mentorship program protocol was essentially exactly the same as the normal protocol--although comments & responses were likely to happen at a faster rate--except question (down)voting was delayed. (Which is the actual change that is needed to implement "niceness".) This proposal is just the same as things are now except with easier access to more standardized & authoritative- & impersonal-seeming canned comments. Which is the next most important change needed.
    – philipxy
    Commented Mar 29, 2020 at 5:28
37

Update to address your update:

Thanks everyone for the thoughtful feedback. I’ve read through much of it and there are a few things that stand out to me.

No problem. Thanks again for reading our feedback and taking it into consideration. Just imagine if you had built everything ahead of time only to announce it and find out everyone hates it. That would be a disaster.

I should warn you that quantifying a "substantial edit" correctly is probably the hardest thing you will do trying to pull this plan off. I certainly don't expect you to get it right the first time. When you do, please give us as much information as you can and be transparent as you can about it. This isn't as much about "transparency" as it is about helping us to be better curators. Knowing what could potentially get a question reopened automatically can better help us decide what action is most appropriate for a question. And, at the very least, it will help minimize frustration if we know what the system is and isn't capable of doing.

Of course, as always, please warn us when you're about to change something. I'm sure I don't have to say that, but just to be clear: this too will help mitigate frustration if we know in advance.

Which close reasons are eligible for automatic reopens?

I have significantly less objection to the idea that some close reasons will be eligible for automatic reasons and some will not than a blanket automatic reopen which (as you seem to now know) is what it sounded like you were saying. This is simply because a hardware question is less likely to become on-topic with an edit than say a programming question that doesn't give enough data to re-produce the result mentioned. The later could be on-topic given enough to work with, the former is just beyond the scope of our site. The effects of opening the later automatically is probably less harmful if accidentally done incorrectly than the former.

All that said, I actually have less objections to this, but you will have to get it right if you do it. But I'm sure you will before you push it live.

These are questions we're still working through, and your input here as well as our experiments will get us closer to these answers.

So I think what you're trying to get at but aren't asking all the way is why questions get closed. If you aren't asking this, then maybe you should as you continue this process. That certainly seems to be what you're wondering anyway.

Something you should ask yourself is why do we even close questions at all? Like why don't we let the question remain on the site, possibly unanswered, but sill open? It's different for every reason, but most close reasons have a reasoning behind them. It isn't because we want to be mean.

A great example of this is closing because the question doesn't have a minimal, reproducible example (I don't remember exactly what is is called now, as all the reasons changed their names recently, and I haven't exactly committed them all to memory yet). Most questions I find myself closing for this reason is not because I don't want to help the user. Rather, it's often very much because I can't help the user. I don't have access to their code base, much less their environment. If I can't make the problem appear myself, I have no way of knowing what is really wrong with their code. I truly can't help them.

Why do I bring this up? Because on the flip side of this, questions shouldn't be reopened if they still have the problems they started with (or even other problems that somehow get introduced along the way). An edit should fix those problems before opening the question.

Try to internalize the rational of every close reason, and that will help you figure out how to guide the asker to fix the actual problem correctly. It will help guide you what reasons should get a automatic reopening (if any) and which ones should not.

Third, there is always the possibility of a failed test during discovery and experimentation. And that’s okay!

100% agree. In fact, it's likely that you will fail at least once.

It's completely okay to fail as long as you a) recognize that you failed b) learn from your mistakes and c) don't accidentally burn the whole house down by mistake because your experiment failed. Please do a and b. Preferably not c. I'm sure c won't happen, but I just figured I'd throw it on there and hopefully at least make you laugh at me being extreme :D

how can we both educate and help new users navigate their way and encourage them to continue contributing to Stack Overflow, while improving the tools curators, mods, and our CMs use to maintain the site?

I don't envy your job. Thanks for doing it anyway :D

That was long enough to probably be its own answer, but I'll keep it as a part of this one anyway. Okay, on to the original post.

Original Post


First of all, thank you for telling us this in advance. That is an improvement from you (the company) compared to 2019, and I just want to take some time and recognize it and thank you. Keep up the good work.

Okay, with that out of the way...


I want to talk about this part right here:

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened.

I get why you would look at this and say, "We really need to open questions up more." This looks like questions are being unrighteously closed. It's really not the problem at all.

The truth is, the problem is not about questions remaining closed, but rather how many deserve to remain closed, even after editing. For the same reason the "Requires Editing" Triage button is confusing, there is only so much other people can do with a question. At the end of the day, a question being opened requires some participation from the OP. If they don't make the question answerable, no one else can.

To give you an idea of what I'm talking about here, the Reopen Queue currently (as of the time of this writing) 424 items needing review. That seems like a lot, that is, until you compare it to the Close-Vote Queue which currently (again, at the time of this writing) has 5.4k items needing review! That's over 12 times as big.

My point is this: As seen by the relative sizes of the queues, the issue here not that the Reopen Queue is under-staffed. Rather, there is something else amiss. It is really likely there are other reasons that these questions aren't getting re-opened. It is not because we're unable to open them.

There's a bigger issue here that I want you to realize. If you reopen a bunch of questions that got closed, how do you think that will affect the Close-Vote Queue? It will almost certainly overload it even more than it is now. This undermines Meg's birthday present. Overall, this will just create even more work, as we'll end up closing things that we already closed once.

TLDR: Please don't re-open questions automatically. That is very counter-productive.

4
  • BTW, if anyone feels like commenting and telling me: should the o in the "Re-open Queue" be capitalized, or does it just look funny to me?
    – user10957435
    Commented Mar 29, 2020 at 19:48
  • Writing Re and open together, like in "Reopen Queue" is the way to go
    – Lino
    Commented Mar 31, 2020 at 8:04
  • 2
    Telling us a couple weeks in advance that it's coming -- as seems to be what is happening here -- is not significantly better than waiting to tell us until it's on the doorstep. It would be completely different if this were a proposal raised to solicit feedback from the community, but it sure does not read that way. Commented Mar 31, 2020 at 12:41
  • @Lino Reöpen queue? Commented Apr 2, 2020 at 0:16
28

First of all: thank you for posting this to Meta before the change is done so that the community has the chance to provide some feedback. I really do hope that the intent with this post was to get some feedback, and that said feedback will not go unnoticed (or worse ignored).

Disclaimer: I've seen some SO people complaining about feeling attacked by answers to their meta posts: this is merely constructive criticism. Please, do not take it personally in any way. I am only highlighting points I think should be re-reviewed and explaining the reason why I think so.

Now let me address some points of this that I do not agree with, and let me explain why.


Hesitation and fear of feedback

Our research has shown that often users hesitate to ask questions on Stack Overflow due to the fear of public, and sometimes caustic, feedback.

I don't understand how this is necessarily considered a bad thing. If anything, in my experience, this spurs me into doing the needed research and diving deep into the problem to make sure I did not leave anything to the doubt before even thinking about asking a question here. Hesitation to ask for help whenever a problem arises is exactly what makes me a decent programmer: I am able to thoroughly research and solve problems by myself.

User John Bollinger also explained this very well in their answer.

I don't think this is a problem, but most importantly: even if it was a problem, it's still not something that can be changed by modifying the system. If you post mindless generic questions, you are going to get critiques. There is no system in which this would not happen. Criticism is something that is solely controlled by the users. Continuously changing the voting or closing system will not reduce critiques to bad questions in any way. It will only cause more confusion.

All this shuffling of rules only confuses users. This has come to the point that I often feel like it's better to comment on a question explaining why it's bad and linking to help pages instead (or in addition of) casting a close vote, because I currently most of the times have no idea about which new reason should be chosen in which exact context.

Low percentage of re-opened answers

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

This really looks like the classical "let's throw some random statistics in there without understanding what they mean".

Again, this is not something that you can control changing some rules here and there. If someone posts a bad question, chances are that the question itself is bad, and no formatting or addition of information will help salvage it. If the user edits the question into a different one, then they should have asked a new question to begin with.

In addition to this, I feel like this also happens mainly because users just "give up" and don't even bother to go back to edit their question after it gets closed. This, again, is not something you can control changing the system. It's up to the single user to put effort into their question.

Changing "closed" to "hidden"

We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed. It’s a gentler way to remove a low quality question from general visibility while giving the author a chance to edit it.

Changing the word "closed" to "hidden" doesn't do any good on its own, and things could get more confusing for new users. This is a really hard thing to do right, and I hope you have a good plan on how to do it.

If my question is bad and needs to be taken care of, I want to be told so clearly, and I want to get slapped in the face by the truth, so that I can quickly address the problem. I don't want nor need to be cradled like a baby with lighter terms. Changing such an historical term would also create a lot of trouble on meta or anywhere else people talk about closed posts. The changes you describe could be very well applied without changing the name.

Regardless of the name though, hiding closed questions does not seem like a good thing to do, for one main reason: closed questions are often still helpful. It happened multiple times to me: I Google something and I find a closed questions which still includes an answer with a solution or even just a comment that is helpful to solve the issue. Hiding posts would only reduce traffic and make the lives of low-rep users (and non-users) more miserable.

Dynamic close vote threshold

to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

What? This feels off. If anything, newer questions are seen by more people than older questions. Are you suggesting to reduce the 3 votes threshold? Doesn't make much sense. The current static configuration feels OK IMHO.

Automated question re-opening

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once. Any subsequent reopens would require review through the reopen queue.

BAD. This is bad. I don't see how this would help the already overwhelmed review queue for close votes. It will merely create new open posts that need to be re-closed. To my eyes, this only exposes the system to abuse. Again, the fact that, as you say, "only 3% of closed questions are ever reopened" doesn't necessarily mean that they should have been reopened in the first place.

Additionally, I don't know what kind of next-generation machine-learning empowered AI you guys managed to come up with, but I don't really think it's possible to discern what a "substantial" edit means, and this will therefore unequivocally cause false positives (and false negatives).

What I would suggest, is to somehow notify people who have originally voted to close the question, saying something like: "Hey, this question that you previously closed was edited and it wasn't just a 10 characters edit, could you please go take a look if it's worth reopening?". It could be an opt-out feature, a non-intrusive different kind of notification, or something else. It could have a timeout threshold. It could really be done in 100 different ways, so I'm just trying to plant the seed here, not saying that this is the solution.

7
  • 2
    On the notification idea on the last point, we now have the ability to follow questions. May be after closing a dialog pops up saying "you voted to close this, do you want to follow the question?" or add a checkbox to follow the question on the close dialog. Commented Mar 31, 2020 at 10:33
  • I hold out hope of being shown wrong, but by my reading of the announcement, it has nothing to do with affording the community a chance to provide feedback. That's an incidental effect of announcing it just a little in advance ("the first phase is scheduled to release over the next couple of weeks"), but the announcement does not convey to me any sense that the main ideas presented are open for discussion or review. Commented Mar 31, 2020 at 12:52
  • @TheLethalCoder that's a pretty good idea indeed. Commented Mar 31, 2020 at 14:28
  • @JohnBollinger you're most probably right, sadly. I don't know what's worse at this point: reaching out with a Q&A meta post ignoring any kind of feedback or just keeping users in the dark and roll out the changes without any notice. I mean, if they did not want to hear anything from us they would have made another blog post, wouldn't they? "Announcing" something in the form of a question on Meta seems like a request for feedback on its own. Commented Mar 31, 2020 at 15:58
  • 4
    My anecdotal experience is about 40% of the questions I land on when Googling something turn out to be closed. A substantial percentage of those end up helping me figure out my answer, despite some people claiming they're "too broad" or "unclear" or whatever they're called now. And even dead questions can sometimes contain hints for things I haven't tried to help me along. So I hope hiding closed questions doesn't actually happen.
    – Troyen
    Commented Apr 2, 2020 at 12:04
  • @Troyen yes, that's also my experience. Hiding closed questions also goes against their own logic of having more traffic. Commented Apr 2, 2020 at 13:54
  • Re. dynamic close threshold and Troyen's anecdote: I've seen moderators go on rampages of closing highly upvoted, old, but still valuable Q&As that don't match their current interpretation of what is on-topic for an SE. It needs to be nearly impossible to close an upvoted question. Commented Jun 9, 2020 at 18:00
22

At the risk of sounding cynical, I am currently seeing two big problems with this. I'll cover them below

This opens the possibility for an even worse asking experience

Suppose you're a user. You write your question, you submit it, and... it gets closed due to being a question that currently is too vague. To take a good example of something I've seen before, you've put in your nodeJS code and went "what do?".

You see that it got closed somehow. So you add your package.json "scripts" key to indicate how you launch the thing. Sounds sensible, yeah?

It gets auto-reopened because it is, after all, a significant edit from the previous. For all intents and purposes you've doubled the length of the question, even though there's a 99% likelihood it was just noise. Your question is live again!

10 minutes later it gets closed and you're wondering:

  1. Why people dislike your question so much
  2. Why it was brought back up only to be re-closed
  3. Why you bothered

And you're off to reddit to ask your question because, over there, they won't close your question to start with.

The "real" solution to this is a human check at the end step, instead of the auto-reopen, because there will likely be a lot of questions where you will need three human checks instead - to re-close it. The feedback provided to users, even in the new mocks, only focuses on the absolute basic, lowest-quality issues, and misses one of the real pain points to start with.

How does a user even know their question got closed?

I don't ask questions very often (I'm the answering type) and I so far have never gotten one of my questions closed, but I've also never heard of a prompt to get my question fixed. Not in notifications, not in emails, nowhere.

This would likely be a much better way to get people back into the loop, on top of this; provide them with a real heads-up that their question got closed. Right now, I'd assume it likely shows up via downvotes, but since the questions will be hidden, that's also likely to not even tick the user into realizing that their content got closed.

Zoe was mentioning the other day that some people post and then do not come back. Re-engaging with them would do way more than an auto-reopen.

21

We don't have too few closed questions being edited and reopened. The gap between the % edited and the % reopened indicates that we have too many closed questions turning into edit-discussions to no eventual benefit. The overwhelming majority of closed questions (indeed, the majority of all questions submitted in 2020) are not useful or salvageable, and the correct response to that is to set them aside, stop them consuming our attention, and stop them competing for resources with worthwhile questions (which are increasingly lost in the noise).

What about the users who submit these questions? We need to admit to the conflict we've created by making ourselves useful. Most people are not here to become part of a community. They're here to extract value from us. Of course we, as a community, and SE, as a company, shouldn't turn up our noses at that — what are we, if we aren't useful to someone? But we need to see to our own health as well.

For a given user who shows up here out of the blue, there are basically four trajectories they can take:

  • Type I: They come here, have a good interaction, become interested, return for more, and contribute to the community. Community benefit: large (we gain a member). SE benefit: large (ongoing interaction).

  • Type II: They come here, have a good interaction, leave happy, but don't get hooked, and don't come back again until the next time. Community benefit: small (we helped someone). SE benefit: moderate (page views, possibility of good word-of-mouth).

  • Type III: They come here, don't get what they want, and go away unhappy. Community benefit: negative (we didn't help anyone, and now they're likely to badmouth us). SE benefit: negative (limited interaction + bad PR).

  • Type IV: They come here, post some crap, come back, post more crap, ad infinitum. They are happy that someone is doing their job for free. Community benefit: negative (we're wading through crap). SE benefit: still positive (ongoing pageviews)... at least until it burns out the community.

SE's focus the past several years has been on changes which are supposed to convert Type III into Types I and II, which is a lovely, hopeful idea that's hard to argue against. But a realist has to admit that they're much more successful in converting Type III into Type IV — and that they're probably not nearly as unhappy about that result as we are. It makes the headline KPIs move in the right direction. But a healthy community needs to know the importance of not making everyone happy. While SE Inc.'s goal is to avoid Type III at all costs, for the community to stay alive we need to fight continuously against Type IV, even at the cost of creating Type III, while still not forgetting that our real goal is creating as many Type I and II experiences as possible.

2
  • While I also worry of conversion into type IV, do you have a basis for claiming this has been happening recently?
    – einpoklum
    Commented Mar 30, 2020 at 19:22
  • 1
    very insightful
    – scravy
    Commented Apr 6, 2020 at 9:17
20

When noted:

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

In the ideal case we all want 100% of the questions to be reopened but obviously that they pass the community quality test, and unfortunately we are not in the world of unicorns, has there been a serious study of why those questions are not reopened? IMHO seems not, and the company only wants more questions without being interested in the quality.

I have a BIG question:

If I vote to close a question as a duplicate using the power of my gold medal then is it enough for the user to "substantially" edit the question for it to reopen? If so, I will be able to vote again to close that question. I point this out because I dedicate myself to a very specific tag where the community is small and, consequently, the number of curators is very few (counted on the fingers of one hand), so I sometimes ask SOCVR to help me close questions. If I can not close the question again, then at least 3 votes from users without a gold medal or another with a gold medal (which are scarce in my tag) will be necessary, which brings me frustration since I will see more garbage in my area.

7
  • 2
    No, we really don't want 100% of the questions to be reopened. Many - probably most - of the closed questions can never pass the quality/relevance test, at all, or without being duplicates of existing questions.
    – einpoklum
    Commented Mar 30, 2020 at 19:19
  • @einpoklum Maybe you have partially read my sentence, the complete sentence is: We all want 100% of the questions to be reopened but obviously that they pass the community quality test
    – eyllanesc
    Commented Mar 30, 2020 at 19:25
  • 5
    No, I actually did read that. You're making the implicit assumption that all closed questions can be made to pass this test, which is not the case. So your caveat cannot justify the beginning of your sentence. It is probably closer to the truth to say "We all want the salvagable 10% or 20% or whatever to be reopened, but obviously that they pass etc."
    – einpoklum
    Commented Mar 30, 2020 at 19:32
  • @einpoklum I think I should say: In the ideal case we want 100% of closed questions to be reopened obviously by passing the quality test, but unfortunately we are far from the ideal case. That is, that phrase does not refer to reality but to ideality. So I talked about unicorns.
    – eyllanesc
    Commented Mar 30, 2020 at 19:36
  • 1
    Consider, for example, one of the several questions posted daily that are near-verbatim dupes of simple questions with highly upvoted answers. There is no good path to reopening such a question. Adding enough to it to make it a non-dupe would produce a completely different question, which should be posed as such. Or consider a question that just asks for a third-party library that does job X. Again, there is no good path to reopening such a question. Many closed questions are like this in not being suitable for reopening, ever. Commented Mar 31, 2020 at 12:58
  • 1
    @eyllanesc While I might admit that eyllanesc's choice of wording is a bit awkward, all they are saying is that SO is making a massive assumption that the low percentage is even reflective of a problem in the first place. They're just stating your own point in slightly different wording: that poor questions should stay closed (real world), even though we wish that all question were great (rainbow and unicorn world).
    – jpmc26
    Commented Apr 2, 2020 at 20:28
  • Was looking for an answer that made this point. Yeah, if questions are going to reopen on us, we need to be able to re-cast our close votes, especially dup-hammer. Like you, I also follow less popular tags (x86 / assembly) where there are only a few of us with gold badges. Commented Apr 7, 2020 at 11:46
19

Useful closed questions

There are plenty of old, closed questions that provide valuable information even though they are off-topic.

I don't want to get any emails nor any notifications about questions that I've asked in the past that are closed, either.

I would consider hiding them if they have low views and a non-positive score, or if there's some (exponential?) proportion of the views to the score that they exceed.

Search

You should stop crawlers from getting hidden questions with some sort of robots.txt. I often find not-useful-at-all closed questions from a Google search and I'd rather these not be seen.

Potential for abuse of automatic reopening?

I see a potential for abuse on the automatic reopening: Someone posts a crappy question, it's closed, and wants an answer to the question.

The OP can make an edit so the question is seen again, and gets their answer, before the question is closed again. I could also see this for spam seeds (that are not obvious spam) or tool requests.

I'm not convinced that dynamic vote thresholds will solve this problem.

Vote refunds

There should be some way where the users who closed the question to at least get their votes back if it's auto-reopened, so they can close it again if the edit didn't make any edits that allow reopening.

Disallowing voting?

I don't think it's a good idea to stop people from seeing the score or voting on closed questions, especially given some points above. People need to know if a closed question is useful or not, and this won't do it any good. Iterating on that, the vote count is a measure of how useful a question is, not how bad or good it is. It won't do any good hiding that from people.

My takeaway

Just fix the reopen queue.

6
  • 2
    If spam is getting closed rather than deleted, I think we have a bigger problem? :D Fix the reopen queue? Can you say more about that?
    – Catija Staff
    Commented Mar 28, 2020 at 3:31
  • 3
    @Catija If the reopen queue allows some way for the OP to enable pushing to the reopen queue once (instead of auto-reopen or first edit), questions might have a better chance of being reopened. However, a large part of this is probably just post owners not wanting to put effort into their questions and then going on rants when their questions are closed.
    – S.S. Anne
    Commented Mar 28, 2020 at 12:31
  • "I would consider hiding them if they have low views and a non-positive score, or if there's some (exponential?) proportion of the views to the score that they exceed." This is already implemented (almost exactly as you describe) as a metric for Roomba deletion.
    – TylerH
    Commented Mar 29, 2020 at 16:27
  • @TylerH That sounds good, especially considering that hidden questions will eventually be deleted.
    – S.S. Anne
    Commented Mar 29, 2020 at 17:10
  • +1 vote refunds. A couple other answers and comments have also made the point that for niche tags it's often the same people looking at a question after an edit that closed it the first time. I know that will be true for me as I keep a close eye on the x86 / assembly tags. If I've already looked at a question in its initial state, it'll take me less effort to re-evaluate it than for someone else who hasn't seen it yet. Commented Apr 7, 2020 at 11:55
  • @Peter As I have some experience with the software aspect of thay area, I might be able to help you.
    – S.S. Anne
    Commented Apr 7, 2020 at 12:00
16

Thanks for the update. However, it seems you still want to go ahead with automatic reopening of questions and just plan to make it a bit smarter e.g. with regard to what close reasons are eligible. Let me just state my opinion on this:

Questions should not be automatically reopened under any circumstances

I can guarantee you that whatever heuristics you use to determine if an edit was sufficient to re-open a question will simply not be good enough, at least not for my standards. Imagine the typical "plz halp debug" code dump question that gets closed because it needs an MCVE - probably one of the most common cases of closed questions that's not a dupe or straight up off topic. So what if the user edits the code, or the description? How would any automated system even begin to measure if the code is enough (and not way too much) to reproduce the described problem, or if the description is clear enough to warrant re-opening? To do so, for starters you would need to implement very sophisticated code analysis for all programming languages supported on SO, nevermind the NLP requirements to understand the problem description.

What you should do instead, is simply throw the questions which your automated system flagged as potentially reopenable into the reopen queue and let the users handle the task of actually judging if the question is good enough.

3
  • 1
    Perhaps automatic reopen could be limited to questions with score >= -2, if someone insists on having it at all? Casting a close vote without a down vote is usually a sign that a question might be salvageable. But any auto-reopen is probably a bad idea. Commented Apr 7, 2020 at 11:58
  • 1
    @PeterCordes given how many trash questions I've seen getting upvoted over the years (e.g. unreadable rants, uncommented code dumps, or even plain old spam), I'd say the threshold would need to be somewhere around +5 to even be remotely useful. But as I think the potential negative effects of automatic reopening are far worse than its benefits, I simply don't think any question should be automatically reopened at all.
    – l4mpi
    Commented Apr 8, 2020 at 10:58
  • I don't either, but as a compromise if someone at SO insists on doing this, to make it less bad during a hypothetical trial period. After the trial period it will hopefully be obvious to everyone that it didn't work (or else we're both wrong and it could work given sufficiently-good natural language heuristics that magically can detect good edits). Commented Apr 8, 2020 at 11:00
15

    I have been discussing this topic for nearly 7 years now, so this post is long because of that and because it includes a lot of reference that people seem to always ask for in recent discussions I have taken place in on this subject. Closure is quintessential to the exchange, so I will not be cutting any detail corners here.

RE: Updates to close/off topic dialogue reasons

The closure system here is outdated.

    The first design was implemented and stood for half a decade before only being altered with a change to the reason verbiage. Only recently has there been any sort of progress on updating this system, which I applaud. The change to 3 close votes seems to be going over well.

The current set of close reasons is problematic

    It is clear that Stack Overflow is famous for its collection of high quality content. It is also clear that adding to that content has become problematic. A facet of that is that we have become so hyper focused on only allowing perfection that the availability of new content is vastly diminished. We need to unleash the creative effort of answerers.

In curtailing questions with a set of close reasons that focuses on as narrow a question as possible with only a single distinct answer that is easily and demonstrably correct we have closed the door to everything else that should be on topic. There is absolutely no reason for questions to be pigeonholed into those which only have 1 answer, and the drive towards ensuring that "on topic" means "questions with 1 answer" has led to a steep decline in not only question to answer ratios but also to questions without answers or without accepted answers.

For a while now there has been more focus on curating a library more than building the library. Instead of building a tall, all encompassing sky scraper of content, we are building a pyramid. Each piece added makes the remaining pieces available smaller until there are no longer any pieces that can be added, and only a thin golden lining is allowed to cover the topmost portion. Meanwhile, there is a still the production of perfect bricks forming an unending multi-million brick line approaching the pyramid.

This is what we have all been observing, in slow motion. Piece by piece the pyramid builds, and the remaining potential dwindles. This is something that should change. We change it by removing the barriers that limit the potential of content. These barriers are the currently narrow topic defined by the set of close reasons.

These barriers were put in place with good intentions, however, they have not had their intended effect. The reason we need a sky scraper instead of a pyramid is because of external search engines. The pyramid makes sense when you have to take a long stairway, but we don't have to traverse each brick to get to where we need to be - there is an elevator. The library we are building has the advantage of external search providing access to the most relevant content. This means that posts with low views are not in the way, as they are dismissed by the search engine themselves; as evidenced by the low views themselves. We need to stop preventing content solely in order to prevent low view content.


Improving the closure experience, a timeline

    The last time we changed close reasons was 2013, during one of the original welcoming initiatives. The mindset was that explaining in more detail to users why their question was closed would make the process less jarring. So, for the sake of clarity, the reasons were changed...

There was an internal dialog that was discussing how to improve the closure experience for users.

duplicates

"we're taking a look at closing behavior and trying to figure out how to make it less jarring for new users. Our goals are (1) to make it clear to people why their question got closed and (2) to make it obvious what they can do to fix it or get better (3) in order to reduce the likelihood of getting into an argument about whether this should have been closed. We started with dupes." states David Fullerton♦

This was in early (February) 2013. They started with duplicate closures, and made some small progress.

"off topic"

    By March (still '13), the team had moved on to addressing "off topic" concerns. Their main goals for improving "off topic" were

  • clarity - we want to make it as clear as possible to the OP exactly what makes their post off-topic to minimize frustration and reduce the misperception that they’re just being “picked on for being a noob”
  • improvement - we want the language and workflow to encourage editing wherever possible (and in particular, to make improving a post seem more logical than arguing that it shouldn’t have been closed.)
  • accessibility - we need to convey that information with as little dependence on other sources (FAQ, etc.) as possible

Help us make “Off-Topic” close reasons clearer to the OP

    It was decided that "Each site needs their own reasons for closing questions as "off topic". Once a number of predefined "off topic" close reasons has been chosen, anyone voting to close as off topic may now use these reasons for clarifying their vote."

Unfortunately this effort turned out to be a large contributor to the problem over time that we now face. While the "improvement" section was accomplished by making an easier workflow for use, accessibility became worse and clarity suffered in certain situations.

By June, during the implementation of some of these updates, the goal for accessibility was dropped. "The idea is that the new help center (formerly the FAQ) will cover all these "sound like our topic, but not allowed for this site" things, and then the top ones will also be reflected in the About page ("do" and "don't" section) and in the custom OT reasons."

"Not a Real Question" and "Not Constructive"

    As a result of the discussion for "off topic", the team also decided to address NARQ and NC in March ('13). At each step of this process, it was clearly acknowledged that the existing set of close reasons worked, so the new ones would need to accomplish the same coverage.

Well, we want to continue to close the questions these currently are used for, but we need to make the feedback clearer, and incentivize improvement (or at least learning). Here's what we came up with:

Eliminate "Not Constructive" and "Not a Real Question", and replace them with more specific reasons:

unclear what you’re asking — Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking.

too broad — There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.

primarily opinion-based — Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.

While the reasons themselves were clear, the coverage expanded to more posts further limiting "on topic", especially with the inclusion of Too Broad and Opinion-Based. The strict interpretation of too broad and opinion-based flies directly in the face of the intent of the site. Subjective questions are perhaps some of the most important material on the site, and they are being actively prevented - we literally have subject matter experts twiddling their thumbs now, just look at the decrease in answering rates from the top users.

Too Localized

    If you read this far, you probably remember too localized. If not, I wont make you read these 1000+ posts on the subject, and will just give a brief overview because this was also an important tool that the community used.

Too localized had problems, and it was improperly used all the time. That aside, the times it was used properly was important to the health of the site and removing that part of the tool is still causing problems today.

too localized — This question is unlikely to ever help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet.

The community needed, and actively needs now, part of this close reason. The part which applies to closing a question which is simply requesting how to make part of the code work. In other words, a fix or implementation which is extremely specific, only applies to the asker's circumstance, and is highly unlikely it could be reused elsewhere.

Right now, users have to simply stare at these posts, perhaps downvote, and wait for roomba to get them. Roomba does get these. All the time.


Improving the closure system

    There needs to be a change in direction here. The barriers put in place to stem low quality questions has not been effective, and worse, has prevented or limited the creation of higher quality content.

In addition, we need to use distinct close reasons, as opposed to wide sweeping ones. If the close reason cannot be summed up in a single sentence that a 5 year old understands, then it needs to be refactored. Each close reason also needs to come with a single sentence that a 5 year old understand with how to fix it if it can be fixed.

Subject matter experts need to be able to partake in questions in which the answer is based on facts, references, or specific expertise. This was the top request for an appropriate question Stack Overflow should support when it started in 2008.

Questions intended to resolve a specific programming problem that have multiple possible answers. As with this answer, but the “correct” response is subjective.

So to, the original guidance was to abide by 3 main checkpoints

  1. detailed and specific
  2. written clearly and simply
  3. of interest to at least one other programmer somewhere

We need to go in the direction of allowing the types of questions that will garner the attention of experts, while still of course stemming the tide of posts simply requesting the community to do all the work or those which clearly misunderstand the platform.

While education can help with the understanding issue, only close reasons can help both stem the tide and encourage subject experts.

Aligning our current set of close reasons with the original guidance will allow us to accomplish these goals.

The goal of this project is to update existing close workflows with a streamlined system which is easier to understand and follow for all users and results in fewer negative interactions between them.

I believe that the stated goal here aligns with the outlook of encouraging subject experts and discouraging misguided or advantageous users.

Loosen the restrictions on what is defined as broad and opinion based

  • Remove Needs More Focus: While Too Broad was redefined as "Needs More Focus", its text still reflects the issue of requiring a one answer question. "Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer". We need to be encouraging answers, and while Focus is important, all that this really needed was detail. This close reason should be removed.

While the details reason can cover the issue of lacking detail here, the secondary part of the close reason "Avoid asking multiple distinct questions at once" is now lacking. This is easy to fix though, because it can just be its own reason. We need more close reasons, not less.

  • Add "Asks too many disparate questions": Avoid asking multiple distinct questions at once. Edit this to remove questions which can be asked in their own post.

  • Remove "Needs details or clarity": The issue with the details close reason is that it still doesn't provide a path for fixing the issue aside from only showing the negative issues present, and it covers two situations. This close reason should be removed in order for two distinct close reasons to take its place.

  • Add "Needs details": There are not enough details provided to answer this question. Please edit this question with additional details to highlight exactly what you need.

  • Add "Needs clarity": As it's currently written, it's hard to tell exactly what you're asking. Edit the question to clarify your specific problem.

There is a clear trend that is forming here, which is the intent. The user must edit their post as the result of the closure. To note, regardless of what the interim is called, these are still "close" votes.

Opinionated needs to be fully remodeled. We want material which is based on specific expertise, however, often posts are closed which include any level of opinion. This unfortunate situation has grown to the point where good subjective questions no longer even show up due to the knee jerk reaction of the "jackbooted heel of justice".

  • Edit "Opinion-based": This question is asking for general opinions. Edit the question to address facts, references, or specific expertise.

"Off topic"

    Off topic is a misnomer in its current use. Debugging issues, non reproducible issues, example issues... these are all on topic, but there is some sort of problem with the question in that it needs additional refinement in order to be answerable. Debugging and typos need to be removed from this category altogether.

  • Move "It's seeking debugging help but needs more information. The question should be updated to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem." from off topic and place it in the global category.

Alternatively, introduce a new category called "Site Specific Concerns" or similarly named category, which can be used by individual exchanges to specifically outline cases such as these. There should also be a way for the community moderators to manage these.

The typo / non reproducible reason needs to be split as well. It also needs to go in either the broad category, or the Site Specific Concerns category.

  • Remove It's caused by a typo or problem that can no longer be reproduced.

  • Add "Caused by a typo."

  • Add "This problem can no longer be reproduced. Edit the question to show to reproduce the issue."

This new category area should also be open to new reasons.

For example, when someone just shows a list of 1.2.3.4 of an assignment or a todo note that they are supposed to accomplish.

  • Add "Only a list of requirements. Edit the question to show what issue was encountered with the specific requirements shown."

RE: 1. Close UX improvements

"We'd like to reconceive "closed" as "hidden"" is a fine agenda as far as wanting it to be gentler on people to have it happen to them. However, the label isn't he issue. The process is. Many people feel pushed out the door when they have a large amount of downvotes on a post and it then gets "closed". Or even when it is "on hold".

The verbiage is a problem here in the meantime prior to the question closure. These are close votes, so calling it closure is appropriate, and so is [closed]. That is a finality that is only reached when the OP stops responding.

Prior to that though, and as noted with the theme of requiring edits above, the process before [closed] should really be called [requires editing], because that is the honest truth. Those questions, for the most part, require editing in order to reopen. Pushing these users towards editing, and attentiveness, will only help them not feel like they are being pushed out the door.

I like the email idea, and the focus on editing here in general, so just go all the way and label it what it really is: requiring edits from the OP.

RE: 2. Edit UX Improvements

Yes.

RE: 3. Automatic Reopening

So long as the automatic reopening is only in response to the asker editing. Sometimes there are significant efforts by community members to clean up questions, and those should not trigger questions to be automatically reopened.

In addition, there are a lot of well articulated concerns regarding abuse of the automatic reopening feature here in other answers. I would expect those to be addressed as well prior to this releasing.

Perhaps instead of fully re-opening, the post could simply enter the reopen queue only needing one vote to be reopened.

9
  • 2
    The one thing not addressed here are duplicates, because that system is functioning well in my opinion.
    – Travis J
    Commented Mar 30, 2020 at 23:08
  • 3
    Thank you for bringing up "Off-topic". Overloading this general concept is so confusing. I just can't bring myself to give a user the message that their programming Questions isn't about programming, and I feel it violates "It's critical that you offer accurate and helpful guidance.".
    – Scratte
    Commented Mar 31, 2020 at 8:10
  • 2
    "Broadening allowed topics". I'm not too sure about that one. We have the concept of multiple Stack Exchange sites for that, but the closure of Area 51, the firing of the responsible employee, and the focus on just the Big Three sites indicates to me that SE,Inc wants the opposite. It's easier to sell a less-diverse audience.
    – MSalters
    Commented Mar 31, 2020 at 13:51
  • @MSalters - I would appreciate it, if you infer that by using quotes in response to me, I actually wrote the text you are citing. I did not state "broadening allowed topics". I believe we need to allow subject experts to weigh in more, we need to allow curators to have a cleaner line of response to questions which need editing or which misunderstand the platform, and we need to allow questions to be posed in such a manner that there are multiple solutions offered or suggested. Please re-read the points made here, I am not suggesting that we include topics covered by other exchanges.
    – Travis J
    Commented Mar 31, 2020 at 16:23
  • 5
    I agree in principle with what you said. "Off topic" not being off topic - that one bugs me all the time. The "seeks debugging help" is on topic, so the close reason is actually "what you're asking is on topic but not complete". Also, the overly broad close reasons are bad for guiding users how to improve their posts. Having more close reasons that are more granular would be an improvement.
    – VLAZ
    Commented Apr 1, 2020 at 13:54
  • Most of all, though, I agree that the current model of SO is self-defeating. Many people who come in and have questions are actually asking a duplicate. To be clear that's not bad - an issue might manifest in multiple ways, so we want different pathways to the same solution. Yet, duplicate closures are shunned by many. Some askers actively dislike it and feel it like punishment. I can't really blame them because even well asked questions aren't rewarded if closed as dupes. We want dupes but we don't actually like dupes. This is further reinforced from the opposite direction by the
    – VLAZ
    Commented Apr 1, 2020 at 14:18
  • 4
    gamification where closing as a duplicate doesn't reward answerers. However, (re-)posting an answer is very likely to net you some rep. So, (some) askers don't want their questions closed as dupes and (some) answerers don't want to close as dupes. There is nothing in the system that says a duplicate is desirable. Askers are supposed to be content with a solution, and curators with a warm fuzzy feeling. Yet, that's not even the main problem here - the more questions there are, the less possible questions you can ask in the future. Moreover, there are even less possible answers. So a
    – VLAZ
    Commented Apr 1, 2020 at 14:18
  • 1
    new user has a hard time earning rep. Sure, rep isn't all that important but it's still put into focus by the gamification. So, people feeling dissatisfied because they cannot get rep are totally justified. Something needs to be done to alleviate this, because otherwise people do come in here and then leave frustrated. At the same time, this has to be balanced to still be (mostly) in line with SO so far. It's a hard task and I can't really give good suggestion but it is something that should be considered.
    – VLAZ
    Commented Apr 1, 2020 at 14:18
  • 1
    @VLAZ - You make some really good points. With regards to the duplicate system, one way to improve the duplicate system would be to allow the closure to actually target an answer on the duplicate target, whereby the targetted answer would then be pinned to the top of the answer list as if it were placed there, with votes, comments, etc. This could be done for each target in the duplicate list. However, that topic is for another day I believe, as it would take significant technical work, and is not as related to the refinement of the closure system itself which seems to be the current task.
    – Travis J
    Commented Apr 1, 2020 at 16:24
15

Which close reasons are eligible for automatic reopens?

I got that you got that "automatic" is seen as a big problem.

Nonetheless, I have a simple answer to that rhetorical question:

... exactly the same that were automatically closed.

I think that is the main point here: if you think you can get some really clever AI to determine what should be automatically re-opened, then you better use those AI capacities first and foremost to prevent users from hitting "submit" on low quality input.

Oh gosh, you can't do that?

But that is the very same problem: you have content there, and there needs to be a decision whether that content is good to go. Where is the difference between deciding "initial question is good" and, "edits made question good"?!

I suggest that you absolutely avoid "the system" making such decisions at this point. Instead: enhance "the system" so it gives helpful information to the curators.

In other words: a good first step could be that the system somehow tries to qualify the edits made, to make a suggestion to curators what to do here. Including options so that the curators can say "yes, this classification was accurate", or "no, you got it totally wrong".

And then, when that system runs for a few months, and we got to the point where curators are 95% happy with the suggestions coming from such a "support facility", maybe then we can allow that facility to make decisions on its own.

In short: help us making decisions, don't make them for us.

1
  • But that is the very same problem: you have content there, and there needs to be a decision whether that content is good to go. Where is the difference between deciding "initial question is good" and, "edits made question good"?! I like this quite a bit. It's one of those things that seems incredibly obvious and straightforward when you finally see it, yet likely wouldn't have thought of otherwise.
    – AMC
    Commented Jun 9, 2020 at 22:42
14

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

To be honest, this seems like a very strange goal. Many questions does not get reopened because they are bad questions to begin with. They kind of fall into the category "you cannot polish a turd".

The real question is how to make new users post less questions that requires closing and editing.

1
  • 5
    It too easy to ask a Question. The site is almost begging users to ask them with that big blue button. There's no real warning attached to it, just a guide. Nobody reads all the pages in the Help center, nor all the linked posts in the FAQ, because 1. It's too much. 2. What could possibly go wrong? Everyone knows this. A lot of legal systems know this which is why a user can still win a court case even when terms have been specified in the "Terms Of Service".. Basically: Everyone knows that nobody reads the manual until something is broken.
    – Scratte
    Commented Apr 6, 2020 at 9:22
12

I apologise if I've covered the same ground as everyone else - but there's aspects that need to be covered in the totality.

Firstly - I've often found SO only capabilities to be a slippery slope to SO having more and more of a unique codebase, which has occasionally bitten the smaller sites in the rear. You might find that these options might actually scale well, or even work better on smaller sites.

I've tried to address each of those points in turn.

  1. Community editing has always been a critical part of the reopening cycle. Along with critical commentary - some comments and edits can be critical to help reopen a question properly, even after closure.

    Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance.

    Needs to be more explicit in the guidance.

    Inbox notifications and emails will alert post authors of newly hidden posts and provide guidance for editing.

    Is excellent

    To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted.

    I suspect that the most ardent voices for quality would love this

    And to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

    To quote Homer (The other one) : I am intrigued by this idea and wish to subscribe to your newsletter.

  2. Literally no disagreement there - but a clarification, is this for closed posts, closed/hidden posts, or the default edit view?

  3. There's a few issues with "automatic reopening on substancial edits"

As a moderator, I have occationally experienced folks trying to get around a question ban by substancially editing very old posts with downvotes or no attention. With this in place - this could get messy.

I wonder if an incremental solution, with "hiding" a post being one state in the lifecycle of a post - then giving it visibility for review before reopening, maybe with lowered thresholds might be an option.

We also already give a reopening vote for edits. We're essentially saying we would treat a new user who has already not demonstrated the ability to ask a good question the same as an experienced user or one with a diamond badge

A clunky option here would be to rename it "on hold", make it visible, and have it pending review... A question as such goes through "Closed/Hidden" -> is reviewed or edited -> becomes visible, and on hold pending review. This might make more people happy about it though.

6
  • 1
    With the editing of very old posts - are they open? If they're closed, they'd be deleted, no? We're not sure how we'll address currently closed posts yet, so there may be a window where a very old closed post isn't deleted but is still on site but it seems like checking for an edit window would be a way to prevent that sort of abuse. :)
    – Catija Staff
    Commented Mar 28, 2020 at 3:35
  • 1
    Closed - the hope is they can get one more question and maybe get it reopened Commented Mar 28, 2020 at 3:52
  • 1
    I'm not sure I follow "Community editing has always been a critical part of the reopening cycle." Certainly community editing is a big thing generally, but I'm not aware of it routinely being part of the reopening process, much less a critical part. Community commentary, yes, but I haven't seen (or at least recognized) community editing being used in this role. Commented Mar 28, 2020 at 13:35
  • 2
    Well - if its possible to edit a post that has clear intent, but is badly worded, people do. You can neither comment, nor edit if a post is invisible. And I've seen folks (rarely but rather gloriously) work wonders in copy editing on a post Commented Mar 28, 2020 at 13:46
  • 1
    @Catija When an asker is question-banned they're encouraged to fix their existing bad questions. Sometimes, instead of cleaning up the question they replace its contents with a totally different question. (I guess that's ok, unless it invalidates existing answers). I don't know how common that practice is on SO, but I see it fairly regularly on Physics.SE.
    – PM 2Ring
    Commented Mar 29, 2020 at 13:48
  • @PM2Ring I've seen it happen. But it doesn't help them much even if they make a really good edit when the Question is at -6 (example) votes.
    – Scratte
    Commented Mar 29, 2020 at 13:53
11

I sort of want to at least note that a bit of a challenge to some of SO's goals here:

We believe that this will help users make substantial edits that increase the likelihood of getting an answer.

I feel like this thought is stuck in a vacuum, we're treating the concept of "Q&A" like ordering a burger; if you ordered right you'll get a burger! But that goes against the "Repository of curated answers" concept, because in that case we only need each type of order once. Edits might not be the REASON our OP didn't get an answer, they may have a duplicate; are efforts going to be made to ensure they NEED to even ask a question in the first place?

SO is like a library we have a vast amount of searchable categorized knowledge, some people just need to be persuaded to look as opposed to asking others to look for them.

11

.. roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened.

That's completely consistent with my experience. When visiting the review queue I really want to find a question to reopen, but often enough I can't. Asking an ontopic question is a hard task.

Curators are burdened with an enormous backlog of tasks. We want to reduce the burden by helping question askers improve the quality of their question through product guidance, relieving curators of this tedious and often thankless task.

While it becomes more clear later on, I doubt that curating can be actually be replaced or even significantly reduced by some sort of guidance currently. Also, curation is all about quality of the content. Please avoid letting the quality standards slip and please don't make it looking like curators would want that. Curators do their job, because it has to be done, because there is no alternative. Of course, there might be another way, but currently I still doubt it has been found yet.

You can try to convince me, but please understand that I will be very skeptical.

We will also be testing automatic reopening, putting the onus on the question asker to improve and reopen their question

It's not as if the reopen queue would overflow currently, but in my experience most of the edits do not improve closed questions and make them answerable (unfortunately). Automatic reopening, especially by the asker, sounds like it could affect quality negatively. Curators would probably be very concerned about content quality at this point. Their motivation might be greatly reduced if their effort in keeping the quality of the content high is undermined in that regard. Maybe this feature should not be tested, or if it is tested, it should be seen very, very critical, right from the beginning.

We’d like to reconceive “closed” as “hidden” ...

That's just wording. I don't care much about it. If the rest stays as it is, it's not more or less gentle. Your question is closed and does not receive answers or your question is hidden and does not receive answers, where is the difference?

..remove a low quality question from general visibility while giving the author a chance to edit it. Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance.

You say it yourself. Hidden is nothing else but removing low quality questions and so is closing. Because most edits by askers of closed or hidden questions do not make the question answerable right away (or more of these would get reopened), feedback from other users is crucial at this stage. Hiding a question may actually not really helping it.

Why not tackle the reported problems directly?

They often experience a pile-on effect of downvotes and comments while trying to edit their question in parallel.

Just limit the downvotes or the score. And a pile of comments is usually a good thing. Most comments are people being helpful.

To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted.

Does that also apply if they already have a positively scored answer?

..we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

How low do you imagine the vote threshold to be? Currently only three close votes are sufficient. I cannot imagine a much lower reasonable number (technically two and one would be possible though).

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state.

Please don't do this. It will likely decrease the quality of the existing questions on the site and will mean that curation work will be spent in vain (the original close/hide votes). Most edits of askers on closed/hidden questions, even the substantial ones, do not result in questions be ontopic or answerable. Opening them is undoing the curation work that has been invested in this question before. A question that we know is low quality, doesn't become higher quality only because it was edited substantially by the creator.

Don't do this and rather stick with reopen queues (which are not overrun really) and only if the reopen queue decision rate for substantial edits goes above 90% acceptance, start to think about it, but not before. Or the reopen queues cannot handle the load, but that's not really the case, is it?

Comment: ..we plan to treat closed as duplicate slightly differently. Current thinking is to allow the user to confirm if it’s a duplicate question by selecting from suggested duplicates.

Not sure if this is a good idea either. If the asker is very inexperienced, I would trust the community to pick a good duplicate target much more than him or her. The risk that a wrong target is picked would be just too high. But for experienced users that could work. The question is where to draw the line.

Finally, I'm not sure writing something like "There is no penalty for asking new question as long as they are ontopic." is a good idea if a question just got closed. The closure or hiding indicates that the asker may have problems with asking ontopic questions. This should probably be changed to something less inviting to post more questions. More like "Please check your question that failed carefully and keep the experience in mind when asking other questions."

Summary:

I like the more explicit guidance and I hope it will have a somewhat positive effect. If only a few percent more of the closed questions can be improved, it was worth the effort.

I like that closed/hidden questions cannot be voted upon. That's kind of fair in my eyes.

I'm undecided about the hiding. It may be better or worse, depending how valuable the feedback from the general public really was.

I'm strongly opposed an automatic reopening. I'm worried about content quality and in the worst case an undermined motivation of curators to do the curation job, which I think is still very much necessary.

8

Automatic re-opening of closed questions

I, too, am skeptical about the automatically re-open proposal.

On the other hand, having closed questions hidden will prevent frivolous edits from people other than the OP that kick a question into the re-open queue where it will be summarily dismissed.

So implementing the hiding of closed questions should reduce the re-open queue measurably.

That means that questions which are edited "substantially" could be kicked into the re-open queue, as is the case now, and have a better chance of being re-opened quickly if they meet the standard.

Providing support for closed and hidden questions

As others have mentioned, it's difficult for most (new) users to understand what needs to be added/changed in order to bring a question up-to-snuff.

How about a checkbox in the close-vote dialog (or elsewhere) where we can volunteer to provide such assistance for particular questions in which we have an interest or suspect we can answer once full information is provided? Perhaps anonomously or with a different user account specifically for this purpose to prevent "stalking" or "revenge" actions.

The dialog with the OP would take place either via comments or a dedicated chat.

In this scenario, simply editing should not place the question in a re-open state (whether automatic or to the queue). Instead, the OP would need to verify that they consider the question now ready for public viewing.

EDIT after replying to a comment:

And/or make a special view/queue, filterable by tag, where those who are interested in helping users dig themselves out can offer assistance to those with closed questions. Again, possibly anonymously. But there should be a way for "mods" to see who has posted in case it's misused and there are complaints.

3
  • 5
    With respect to the issue of other users editing kicking the question into the reopen queue, wouldn't it be better to just fix that? In other words, only have an edit from the OP kick it into the reopen queue? I do agree that this is currently a significant problem. Such edits by other users almost never make the question reopenable and always usurp the OP's one entry by edit.
    – Makyen Mod
    Commented Mar 29, 2020 at 6:12
  • @Makyen see my answer for a, very short, write-up on that. Feel free to share your thoughts in an edit on it.
    – Adriaan
    Commented Mar 29, 2020 at 15:19
  • 2
    @Makyen Sure, that would be good - also for those edits on closed questions that translate non-English. So, you don't want closed questions "hidden"? I could live with it either way, but the OP is less likely to get helpful comments for improving or finding information with a question hidden completely... Commented Mar 30, 2020 at 7:44
6

I like that questions get hidden now instead of being closed. This is not only "gentler", but it also has a few advantages on my side: I don't have to see all those hidden questions in the question list anymore, somebody is already taking care of them so there is not much need for me to see them. That way I can focus on questions that still need eyes on them.

I like that users get notified that their question got hidden, I also like the guidance given to them when editing. That's much better than closing the question and leaving them alone without ways to improve them. There are however a lot more nuanced reasons for closure than we have in the canned responses. So I think there have to be either a lot more closure reasons (maybe even tag specific ones), or there should be some way to write specific advice (e.g. by showing the best (?) comments on the editing page).

I do however dislike the way we are reopening questions at the moment, and I don't think that reopening questions automatically makes it better. Reopening a question should ideally give a second chance to the question. Yet, the question starts her second round with (a few) downvotes, negative comments and possibly some bad answers. Also it does not receive the same attention as a newly asked one.

I'm also curious how you want to detect questions worth reopening. If there would be a reliable metric to detect good questions / edits, why do we even allow people making bad edits and asking bad questions? Should'nt that be prevented in the first place, in the interest of all?

what should we do with existing closed questions?

We already have closure reasons for historic reasons, meaning don't take this answer as a precedent, don't expect that a similar question gets accepted. I think that would be a way to let closed questions exist alongside hidden questions. Closed questions with negative score would be hidden. Duplicates with positive votes would be closed instead of being hidden.

4
  • 6
    Also it does not receive the same attention as a newly asked one. This is a good point. The reopen queue means at least 3 people will see the question so it has some chance of getting up votes if it’s been fixed. Reopened but at -3 or worse probably doesn’t help the OP much.
    – BSMP
    Commented Mar 28, 2020 at 15:55
  • 6
    To @BSMP's point: this is why it is critical that fast closing go hand-in-hand with the change: ideally no one has time to downvote. If questions are still hanging around for hours or days before being closed, this entire plan risks losing efficacy.
    – Shog9
    Commented Mar 28, 2020 at 20:39
  • 1
    @Shog9 So just close while score is negative--gets to -1. They have to improve to get an answer (get upvoted & reopened), they weren't pounded by downvotes, they don't have a big negative legacy to overcome. Until reopened. (Still then a "niceness" downvote pounding problem.)
    – philipxy
    Commented Mar 29, 2020 at 5:38
  • 1
    Resetting a question score to 0 after being reopened is an interesting idea. Though OP's score as a result of the votes should remain the same (and expanding the vote should still show a full record of the votes)... which would mean up/down voters on the question pre-closure would not be able to vote up or down again post-closure.
    – TylerH
    Commented Mar 29, 2020 at 16:32
6

Regardless of the other points made here, I strongly support at the very least changing the word "closed" to "hidden". In my interactions with people on reddit about SO, many people seem to treat closure as the end of the road, and often don't even know that there's a straightforward process to get their question reopened. To me, the word "closed" has a connotation of finality to it, and I feel that changing it to a gentler word as proposed would better encourage askers to revisit their question.

2
  • 5
    What about "On hold" or "Pending"?
    – Scratte
    Commented Mar 31, 2020 at 8:20
  • 4
    "Waiting for {code/example/expected behavior/mvce}" could be used to make explicit why the question is not open. I don't think the exact name of "not open" really matters, but there should be a Call To Action.
    – MSalters
    Commented Mar 31, 2020 at 13:54
6

I agree with essentially all of the highly-upvoted answers here, specifically regarding the problem with auto-reopening on significant edit. I want, however, to make a

(Compromise) suggestion: constructive steps for auto-reopen eligibility

Don't have automatic-reopen already after "significant" editing, but offer such a reopen only when some additional conditions are met, such as:

  1. User has taken the tour (or retakes the tour?)
  2. User checks a checkbox for one of the acceptable question types from the tour.
  3. User clears a checkbox for all of the unacceptable question types/traits from the tour.
  4. User explains their edit, and doesn't just make it. Perhaps - through a guided explanation form with checkboxes "I have [ ] added a concrete example [ ] Made an example complete and reproducible [ ] Clarified ambiguous terms [ ] Refocused to fit the scope of this site etc." - it could have textboxes too.
  5. Something like this question quality checklist (by none other than Jon Skeet).

Also, have the closure (or "hidden") notification encourage the user to go do the above.

3
  • 1
    While I don't completely agree with the conditions listed here, I do like the suggestion, and I agree with the spirit of this - making sure users understand SO. A small test or something is not a bad idea. Sure, it can be cheated and bypassed but currently people just bypass the whole tour and directly post questions that aren't suitable. So, prodding them to take the tour and pay attention to it is going to be of huge help. I'd love to see some sort of more in-depth look into how can this work without burdening new users too much.
    – VLAZ
    Commented Apr 1, 2020 at 12:28
  • @VLAZ: Indeed, the specific conditions might be somewhat different than the ones I suggested. The point is to place a hurdle which requires some conscious effort; and the assumption is that people don't visit SO to "game the system" and push bad questions.
    – einpoklum
    Commented Apr 1, 2020 at 13:41
  • Indeed and I would support such a solution. I think it would even be totally appropriate to throw some rep at the users who complete this "intro course". For example 5 rep seems good - it can definitely get people started on the site while also avoiding them being able to edit protected posts. Users coming with the wrong expectation is one of the biggest problems, so some sort of education should help. Evidently simply having a help centre is not enough. Some sort of not very intrusive way to explain how the platform works seems in order and can be tied with the gamifying aspect.
    – VLAZ
    Commented Apr 1, 2020 at 13:48
4

Just a quick thought after reading question and some answers:

Re-open action should be requests by OP manually

After post is closed, OP should see a button Re-open: he will have to improve his post and after its done and double-confirmed (feel free to be creative here: re-open wizard, red blinking warnings,etc.) it will go into re-open queue.

This may be requested multiple times. With auto-ban if abused, etc.

Currently the aim of re-open queue is to open closed by mistake/improved questions. If question still bad - it will stay closed. Would be good if OP gets valuable feedback, why his question should stays closed. Ideally all who decide to click Keep closed button should add an info (auto-generated comment?) as to why it can't be opened, taking edit into account.

2
  • 1
    Note that any user with >= 250 rep has the ability to cast a reopen vote on their own questions, which effectively gives them a one-shot "reopen" button, in addition to the automatic entry into the reopen queue upon the question body being edited by someone who hasn't voted to close or flagged the question, or who is the OP. That doesn't solve the issue for users with < 250 rep, but means there's potentially data available to see how effective such a button is, when it's single use.
    – Makyen Mod
    Commented Apr 7, 2020 at 0:51
  • 2
    As long as I can vote to close the same question multiple times for the same reason as the user mashes the Re-open button every time it's closed, fine. The number of users who actually agree with a closure is exceedingly small. If you think we have a lot of questions asking for un-bans about "Requires Editing" now, imagine the number of questions asking for un-bans about "Re-open" once this is implemented... One of the reasons in Keep closed should be "because the user clicked Re-open without doing anything". Commented Apr 9, 2020 at 17:05
4

Auto-reopen seems likely to have a vast number of false positives. We probably shouldn't have it at all, until we have AIs with much closer to human-level language analysis.

But if someone really insists on doing this, perhaps question score can be an input to the reopen heuristic. Like only possible automatically for score >= -2?

Casting a close vote without a down vote is usually a sign that a question might be salvageable. Questions that are also downvoted to hell are more likely to be unfixable. Some might still be after a huge edit, so being visible to people who might want to manually cast reopen votes is appropriate for highly-downvoted edited questions. But not auto-reopen.


Separately, if questions do reopen, in less-popular tags they'll often be seen by many of the same people who originally closed the question. If they've already spent their one close vote or dup-hammer on it, they can't deal with it. In the worst case (false-positive auto-reopen) this creates a whack-a-mole situation where the users can't whack. (So be very careful with auto-reopen heuristics).

Preferably refund close votes when a question is reopened, or at least allow new votes to be cast. Especially for Gold or even Silver badge holders in those tags.

People who read the earlier version can probably evaluate the change in less time than fresh viewers.

1
  • A latte of false positives. Commented Apr 16, 2020 at 15:57
-9

I'm interpreting this:

[W]e’ve heard the ask to reduce toxicity for all users through improved moderation and communication tools...

as:

We're going to reduce the perception of toxicity of the site and moderation efforts through this proposal.

...which in this case, I'm actually okay with all of the stated proposals. I don't disagree that the automatic reopening is going to be a sore point, but I feel like we have to be able to measure how well/poorly that goes, and if we have users who willfully close questions just to close questions, or if we have askers who consistently flaunt our standards and requirements such that this sort of behavior continues and festers.

My belief is that Stack Overflow has to start somewhere to start fixing question closure and improve the workflow for all of this. This feels like a reasonable start.

7
  • 13
    I think it's a reasonable start without the automatic reopening.
    – Cody Gray Mod
    Commented Mar 29, 2020 at 0:25
  • 2
    I agree with Cody; if this didn't have the automatic reopening portion, it'd be a matter of just sussing out the details. But for me the automatic reopening is a non-starter that just tanks the otherwise workable portions of the proposal.
    – TylerH
    Commented Mar 29, 2020 at 16:34
  • 2
    At this point @TylerH I'd rather Stack Overflow Inc. do something rather than nothing, so phrases like "non-starter" really sour my hopes on something happening instead of nothing, y'know. I don't think that automatic reopening is appealing, either. But I believe that the reason it's in consideration is due to scale. The reality is that there are not enough people reopening questions, and it does feel a lot like stepping on someone who's already on the floor if they go through a good faith editing effort just to not be rewarded for it.
    – Makoto
    Commented Mar 29, 2020 at 19:25
  • 4
    I suppose my point is, let's just experiment with something instead of wringing our hands.
    – Makoto
    Commented Mar 29, 2020 at 19:26
  • 11
    @Makoto Sure, I agree we should do something. It's just that we have a whole bunch of things in the "probably good ideas" pile, and we don't want to try something from the "almost definitely a bad idea" pile.
    – TylerH
    Commented Mar 29, 2020 at 21:49
  • 3
    And I disagree with "there are not enough people reopening questions". I think there are way too many questions on SO right now in an open state. An enterprising moderator could have their hands full for weeks if they spent 40 hrs/week just going through and closing/deleting bad questions.
    – TylerH
    Commented Mar 29, 2020 at 21:50
  • 1
    @TylerH: I get that there's definitely resistance to the automatic reopening idea, and I'm maintaining that I would acknowledge this as a sore point. But what I'm not reading in any of this is the same old Stack Overflow Inc. that would make a decision and ignore feedback. This entire process reads to be iterative. They're going to try some things and get some actual numbers about how impactful this is. I would much rather automatic reopening be an actual scourge we need to deal with than some boogeyman that we're just afraid of.
    – Makoto
    Commented Mar 30, 2020 at 16:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .