-63

CLARIFYING UPDATE

Thanks everyone for the thoughtful feedback. I’ve read through much of it and there are a few things that stand out to me.

First, I want to clarify since this didn't quite come across, none of this, except for what we felt were least risky which is outlined as phase 1, has been built. These bigger concepts are ideas we want to test to address issues with the question close-to-open workflow. Our goal with sharing our thoughts on meta at an early stage is to get feedback about what we might be missing as we consider these ideas. For example, defining a “substantial” edit is something our data team is helping us explore and not yet finalized.

Second, it sounds like I missed the mark in articulating why we think these ideas are worth pursuing. We want to help askers increase the likelihood of getting their questions answered. We think that providing a clear path to reopen is like saying “you get a second chance to try again.” And our hope is a bit more privacy and better guidance will be motivating and helpful for askers to craft a better question. It's clear that many of you are concerned about automatic reopens and the potential for abuse with these ideas. This is absolutely top of mind for us, and that's why we are planning to run experiments to better understand and eventually mitigate such behavior before any of this is released broadly. Which close reasons are eligible for automatic reopens? Should it be easier to reclose a question that was reopened? When should we disqualify a question from reopening at all? These are questions we're still working through, and your input here as well as our experiments will get us closer to these answers.

Third, there is always the possibility of a failed test during discovery and experimentation. And that’s okay! Learning from failed experiments is just as important as nailing it. In fact, we will fail more times than we succeed. I think getting comfortable with experimentation and potentially failing is a better mindset to be in than trying to nail the perfect solution or worse, not trying anything at all because we’re too scared to make a mistake. My hope is that we can engage with you openly to test new ideas, question our assumptions, and flag things we’re not thinking of.

Lastly, there are really useful suggestions here as well as callouts to other areas we are actively thinking about. For example the state of review queues is a sore point mentioned several times, or improving onboarding to help new users understand the rules of engagement and expectations of the community. These examples are two sides of the scale we want to balance – how can we both educate and help new users navigate their way and encourage them to continue contributing to Stack Overflow, while improving the tools curators, mods, and our CMs use to maintain the site? This is a recurring theme you’ll see from our product and community management teams this year as it’s top of mind for all of us.

As far as next steps– feedback from meta is in fact research that we formally use to inform product direction. I’ve learned how meta can be a constructive source of feedback for our product discovery - thank you for that! We’ll use the feedback here through this discovery phase and share updates as we hit milestones along the way. In case you missed it, here is a process we're testing that you can use to make sure posts relevant to this project get staff attention.

I hope this helps ease some of your concerns and I will provide additional comments below the best I can, if Catija hasn’t already done so.


Hey! I’m Des. I’ve been here at Stack Overflow for a little while now (4 years and some change) and just recently started working much more closely with the Public Q&A product team as Director of Product, Public Platform. Working within Teresa’s organization, I'm responsible for setting the overall strategy for the Public Q&A Platform. Over the past couple of months, I’ve been working closely with our development and research teams to ramp up on our current initiatives and gain a better understanding of user needs and pain points. I still have a lot to learn! But I’m thrilled to be working with this team (and you!) and look forward to sharing more with you here :)

As mentioned in Teresa’s Q1 roadmap post, we’re working on making changes to how the question closing system works on Stack Overflow so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators.

Why are we doing this?

Our research has shown that often users hesitate to ask questions on Stack Overflow due to the fear of public, and sometimes caustic, feedback. From curators, we often hear about the time-consuming burden of manually guiding users whose questions get closed, and sometimes receiving pushback when they are just trying to help. And from moderators, we’ve heard the ask to reduce toxicity for all users through improved moderation and communication tools.

Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.

Our research and design team took a closer look at the problem, considered previously proposed solutions and feedback, and came up with what a long-term solution might look like. Then we broke it down into phases so that we can solicit feedback, test, and iterate along the way rather than take months on end to get something out there.

Goals

The goal of this project is to update existing close workflows with a streamlined system which is easier to understand and follow for all users and results in fewer negative interactions between them.

Askers are often unsure how to get their question reopened and onsite guidance can be confusing. They often experience a pile-on effect of downvotes and comments while trying to edit their question in parallel. This can feel overwhelming and intimidating for question askers and drive-by visitors.

Curators are burdened with an enormous backlog of tasks. We want to reduce the burden by helping question askers improve the quality of their question through product guidance, relieving curators of this tedious and often thankless task. We will also be testing automatic reopening, putting the onus on the question asker to improve and reopen their question and explore the notion of “hidden” as opposed to “closed” questions, giving askers the space to (somewhat more) privately improve their questions. More details on these concepts are below.

Things we’re not doing

As a quick call out, we’re not planning the following:

  • Network wide roll out - This project addresses pain points experienced on Stack Overflow. We’ll need to better understand the needs of the other network sites as this solution may not be needed, nor appropriate for other sites with less traffic and peer-to-peer friction. As of now, we don’t have any set plans in the nearterm to roll this out to the network. Having said that, if there are parts of this project that can relatively easily be rolled out to the network, we’ll try to prioritize them.

  • Automatic closures - We won’t be automatically closing questions. Questions will still need to be voted on for closure.

  • Getting rid of question closing - We have no intention of removing the question closing feature, and continue to embrace our commitment to maintaining high quality content on the site.

Feature Overview

Outlined below are the key components to this project. And a bit further down, I provide more detail on how we’ve phased this project out. While the team has validated the general direction through user, historical, and competitor research, we want to test these ideas and get feedback where we can along the way. Here are the key components of this project.

1. Close UX improvements

We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed. It’s a gentler way to remove a low quality question from general visibility while giving the author a chance to edit it. Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance. We are still toying with ideas for what this path looks like, and we welcome your suggestions!

Inbox notifications and emails will alert post authors of newly hidden posts and provide guidance for editing. To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted. And to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.

Email notification enter image description here

2. Edit UX Improvements

This new editing UX will provide guidance tailored to the specific reason the question was closed. We believe that this will help users make substantial edits that increase the likelihood of getting an answer. For askers, they get clear guidance about how to edit and reopen their question and ideally have a better experience on the site. For other editors, based on explanations in the "closed" post notice, they’ll know how to rehabilitate the question and help the question author. Changes to post notices have been rolling out over the last several months. We'll be continuously refining the content we show here based on user feedback.

You’ll notice the updated editor borrows UI patterns from the new Ask a Question form.

Edit form, draft step enter image description here

Edit form, review step enter image description here

3. Automatic Reopening

When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once. Any subsequent reopens would require review through the reopen queue.

Here is a clickable prototype that illustrates these three core pieces. Note that final designs and copy may change.

Iterative and Incremental Development

The close-to-reopen workflow is a multi-touch process. In order to understand the impact of these various changes, to minimize disruption for users, and to avoid being in development for a lengthy period of time, we’re taking both an incremental (break the project down in smaller pieces) and iterative (continue to refine as progress is made) approach. Here’s how we’ve planned things so far.

The first phase, scheduled to release over the next couple of weeks, is much more solidified. Whereas each subsequent phase is less so, giving us room to adjust based one the data and feedback we see.

Phase 1

This first phase is made up of foundational changes and does not yet introduce the new concepts of automatic reopen or hidden questions. These are things we’d likely do anyway and provide the groundwork for the other phases.

  • Updates to close/off topic dialogue reasons

  • Updated editor for closed questions - pulling from the new Ask a Question design pattern, this provides better guidance through the editing flow.

  • Email and inbox notifications about closed questions - users will first receive an inbox notification when their question is closed. If the inbox notification isn’t seen, an email is sent to encourage them to make edits.

Phase 2

This phase introduces automatic reopening of questions when they’ve been substantially edited and passes questions with more than one closure to the reopen queue. At this stage, we plan to do some quantitative and qualitative testing to understand the impact automatic reopening has on askers (are questions more likely to be answered? closed again?) and curators (do we see a decline in low quality flags? are higher quality questions making it to the queue?). Deletion and dynamic thresholds for closing will be explored during this phase.

Phase 3

Reconceiving “closed” as “hidden.” When a question receives sufficient votes to become hidden:

  • The post is hidden for primary views. Post authors and privileged users would have a direct path to view and edit. Anyone with a direct link can still access it as well.
  • Score and voting capabilities are removed.
  • Post notices, inbox notifications, and emails are all updated to reflect the new language.
  • If the question isn’t reopened, it’s deleted after a certain period of time.

This step will also be an important piece to test and iterate on.

Questions & feedback

As mentioned, these phases are a rough outline of how we plan to tackle this project over the coming months. We plan to share our findings with you here at each milestone and get feedback along the way. For now, we’re happy to hear your thoughts or any concerns you have, as an answer below.

It should be noted that we still have some open questions that need discovery and you might think of something we’ve missed so feedback on things not mentioned here would be great. For example, what should we do with existing closed questions? There are lots of close questions today on Stack Overflow and we hear that users still find them helpful.

Thanks all for taking the time to read this and provide feedback at this stage.

72
  • 45
    What about duplicates? Does the same automatic reopening logic applies to them too?
    – Dharman Mod
    Commented Mar 27, 2020 at 20:31
  • 208
    It is a little bit insulting when you realize you want to let 1 person who has probably no experience with this site overrule a decision made by 3 people with ceratin amount of experience just because they have edited the question. If 3 people closed the question then 3 people need to review it after edit is made and decide if it can be reopened.
    – Dharman Mod
    Commented Mar 27, 2020 at 21:00
  • 51
    Something that is good to note: this is a test/plan, nothing is set in stone. So, don't panic. :) Commented Mar 27, 2020 at 21:51
  • 19
    @Dharman we plan to treat closed as duplicate slightly differently. Current thinking is to allow the user to confirm if it’s a duplicate question by selecting from suggested duplicates. If it is, the question becomes public and references the duplicate. If it isn’t, the question will need to be edited before automatic reopening (if it’s the first closure).
    – Des StaffMod
    Commented Mar 27, 2020 at 22:13
  • 152
    Current thinking is to allow the user to confirm if it’s a duplicate question by selecting from suggested duplicates. --> most of users don't agree with duplicate simply because they don't like their question being closed as duplicate not because the duplicate isn't suitable ... the question will need to be edited before automatic reopening --> most of the edits are my question isn't a duplicate! so I don't agree at all with an automatic reopening of duplicate closure ... This will give us more job to reclose them again by calling another gold badge. Commented Mar 27, 2020 at 22:32
  • 40
    I think these steps are really good experiments to figure out what askers and curators need. But I don't expect these experiments to be particularly successful, for the reasons outlined in all the answers. Please don't activate automatic reopening unless the other components indicate an increase in edit quality. But nevertheless, thank you for engaging the Meta community early on.
    – amon
    Commented Mar 28, 2020 at 7:03
  • 27
    even if it's not a dumb check, a system cannot evaluate (based on OP's edit) that a question is no more a dupe and should be reopened (even the most advanced AI cannot). I usually close question with canonical target having 20 answers and OP always comment the same thing that answer is using X but I am asking for something different because he simply look at the accepted answer and never take the needed time to check ALL the answers, test them, understand them. I am already facing issues with blind vote (meta.stackoverflow.com/q/393400) and this automatic reopening will make it worse Commented Mar 28, 2020 at 8:11
  • 29
    Am I really seeing another "this is what we're going to do, whether you like it or not" announcement? Another "our staff thought about it, but we didn't discuss any details with the community until it was a fait accompli"? If so, then this is an unsatisfying flavor of transparency. Commented Mar 28, 2020 at 17:59
  • 30
    @JohnBollinger Hardly. There’s three phases and they talk a lot about testing and trying things out. We’re pretty early in this, which is why we’re posting now. We want to know what y’all think.
    – Catija
    Commented Mar 28, 2020 at 18:31
  • 19
    Never mind the downvotes; thank you for putting some thought into this and sharing your ideas thus far. The regular visitors of meta are not your target audience – "bad" questions are our arch-nemeses, and out of sight is out of mind – but I can see how this may be able to turn a Bad Question into a (Reasonably) Good one. Is there objective evidence that this would be a good investment of your time? Can a significant number of quite obviously "bad" questions be salvaged this way?
    – Jongware
    Commented Mar 28, 2020 at 21:04
  • 15
    Like others, I have issues with what is "substantial editing". Since this needs to be clearly defined before implementing it, could you, um, edit your question to at least give us an idea of when Phase 2 is likely being rolled out? Finally, why not automatically route "substantially" edited dups to the review queue instead? I've found that many dups have comments arguing that it isn't one from the OP.
    – user7014451
    Commented Mar 28, 2020 at 21:14
  • 24
    I'm pleased to hear that, @Catija, but I hope you appreciate that the announcement reads as if all the major decisions have already been made, and the multi-phase incremental development process is primarily about delivery. A schedule is presented for developing everything described in the announcement. If that is not actually the way the announcement should be read then it would be really helpful if the announcement were rewritten to more accurately convey what is happening here. Commented Mar 29, 2020 at 12:39
  • 18
    @RandRandom Users closing questions need to be able to turn off those pings per question. I close 40-50 questions a day... many of them cannot be edited into shape because they're inherently off-topic. I don't want to get 40+ pings a day about users complaining in an edit that their question was closed or that SO sucks.
    – TylerH
    Commented Mar 30, 2020 at 13:11
  • 45
    I would argue that "just 3% of closed questions are ever reopened" is a sign that the people who close questions do so with 97% accuracy. Commented Mar 30, 2020 at 14:31
  • 29
    Please don't cast close votes on this, folks. It's absolutely great that the company want to share their research and ideas with the community.
    – halfer
    Commented Apr 1, 2020 at 10:28

31 Answers 31

1
2
-20

Personally I think this is a fantastic idea.

In my experience StackOverflow is full of elitists and overly-zealous moderators who close questions far too liberally.

It's so disheartening to spend a long time crafting a detailed and thorough question only to have it closed because it's "a duplicate" (it's not) or "opinion-based" (so is virtually any answer).

As someone who has been a "power user" for several decades I know how to ask questions; and before I do, I RTFM and search for existing solutions. I'm not an idiot, yet my questions have often been closed for reasons I think are invalid.

For me, I am now very reluctant to post on StackOverflow because of the odds my question will be wasted by being closed.

8
  • 14
    I'll just copy/paste something I just wrote in a comment on the question: "The fundamental problem is that most people are not invested in writing a quality post, and this creates friction with a community and system that prioritizes that quality over giving answers." Our emphasis on quality is not elitism. It's a goal that optimizes for questions and answers with long term usefulness. Characterizing it as elitism is rude and itself condescending and insulting.
    – jpmc26
    Commented Apr 2, 2020 at 16:25
  • My questions have always been high quality; I provide context, explain what I'm trying to achieve, and detail what I've already tried. Obviously it's just my opinion but all of the questions I've asked have been totally sensible and justified. Regardless, several of my questions have been closed and it left me feeling extremely frustrated because of the wasted time and - ultimately - the fact that one or two editors took away the opportunity for me to find a solution to my problem from an entire community of knowledgeable people. That's what angers me. If that's not elitism, idk what is.
    – WackGet
    Commented Apr 2, 2020 at 16:31
  • 12
    I couldn't find any closed questions on your profile. You should demonstrate your point with evidence. Even context, explanation, and what you've tried aren't always enough to make a question high quality, though they help a lot.
    – jpmc26
    Commented Apr 2, 2020 at 16:31
  • 1
    I would gently take issue with your last paragraph - if you post to Stack Overflow and format it reasonably and do the necessary research and it is still closed, it surely it is not wasted. It can be copy-pasted to other (less demanding) sites, such as Reddit, where it may be a better fit.
    – halfer
    Commented Apr 2, 2020 at 21:15
  • 10
    There are some good points above about why this answer is wrong. I'd add a reason that stems from my editing experience. All too often I see folks posting a question with a preface This is not a duplicate question. Please do not mark as a duplicate. Read the whole question carefully to see why etc. This is counterproductive as (a) it does not say how it is different, and to which possible duplicates it is different, and (b) it indicates that the writer is not open to the possibility they missed a duplicate. It really does happen.
    – halfer
    Commented Apr 2, 2020 at 21:20
  • 10
    Thus, when I read your answer that insists that your questions were not duplicates, I am hesitant to take your word for it. I sympathise that you wrote a question only to have it closed anyway, but most duplicate closures are good - they point the question author to a treasure-trove of pre-existing answers. If you do not understand how to apply the marked duplicate to your own case, you can ping a closer or commenter in the comments.
    – halfer
    Commented Apr 2, 2020 at 21:23
  • 7
    What I find interesting is that answers favouring less moderation are often written in a passive agressive stance attacking others (read: curators); in this case with words as "over zealous" etc. Those posts express a sentiment the OP has, throws wild accusations around, but rarely ever substantiates them with evidence, let alone write a non-attacking post on why this is a good idea.
    – Adriaan
    Commented Apr 4, 2020 at 9:16
  • 3
    @Adriaan I think what you note actually points to another common trend we see: people take closure and downvoting personally. They feel insulted that their question was rejected in such a manner. Even though curators will (generally, at least) readily emphasize that these tools are not a judgement of your ability or intelligence, we do acknowledge that the experience is unpleasant. And our answer to that is to tell users to step back and try to understand how they can improve, rather than become angry or upset. SO, however, decided to side with those who are angry and upset instead.
    – jpmc26
    Commented Apr 4, 2020 at 20:18
1
2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .