CLARIFYING UPDATE
Thanks everyone for the thoughtful feedback. I’ve read through much of it and there are a few things that stand out to me.
First, I want to clarify since this didn't quite come across, none of this, except for what we felt were least risky which is outlined as phase 1, has been built. These bigger concepts are ideas we want to test to address issues with the question close-to-open workflow. Our goal with sharing our thoughts on meta at an early stage is to get feedback about what we might be missing as we consider these ideas. For example, defining a “substantial” edit is something our data team is helping us explore and not yet finalized.
Second, it sounds like I missed the mark in articulating why we think these ideas are worth pursuing. We want to help askers increase the likelihood of getting their questions answered. We think that providing a clear path to reopen is like saying “you get a second chance to try again.” And our hope is a bit more privacy and better guidance will be motivating and helpful for askers to craft a better question. It's clear that many of you are concerned about automatic reopens and the potential for abuse with these ideas. This is absolutely top of mind for us, and that's why we are planning to run experiments to better understand and eventually mitigate such behavior before any of this is released broadly. Which close reasons are eligible for automatic reopens? Should it be easier to reclose a question that was reopened? When should we disqualify a question from reopening at all? These are questions we're still working through, and your input here as well as our experiments will get us closer to these answers.
Third, there is always the possibility of a failed test during discovery and experimentation. And that’s okay! Learning from failed experiments is just as important as nailing it. In fact, we will fail more times than we succeed. I think getting comfortable with experimentation and potentially failing is a better mindset to be in than trying to nail the perfect solution or worse, not trying anything at all because we’re too scared to make a mistake. My hope is that we can engage with you openly to test new ideas, question our assumptions, and flag things we’re not thinking of.
Lastly, there are really useful suggestions here as well as callouts to other areas we are actively thinking about. For example the state of review queues is a sore point mentioned several times, or improving onboarding to help new users understand the rules of engagement and expectations of the community. These examples are two sides of the scale we want to balance – how can we both educate and help new users navigate their way and encourage them to continue contributing to Stack Overflow, while improving the tools curators, mods, and our CMs use to maintain the site? This is a recurring theme you’ll see from our product and community management teams this year as it’s top of mind for all of us.
As far as next steps– feedback from meta is in fact research that we formally use to inform product direction. I’ve learned how meta can be a constructive source of feedback for our product discovery - thank you for that! We’ll use the feedback here through this discovery phase and share updates as we hit milestones along the way. In case you missed it, here is a process we're testing that you can use to make sure posts relevant to this project get staff attention.
I hope this helps ease some of your concerns and I will provide additional comments below the best I can, if Catija hasn’t already done so.
Hey! I’m Des. I’ve been here at Stack Overflow for a little while now (4 years and some change) and just recently started working much more closely with the Public Q&A product team as Director of Product, Public Platform. Working within Teresa’s organization, I'm responsible for setting the overall strategy for the Public Q&A Platform. Over the past couple of months, I’ve been working closely with our development and research teams to ramp up on our current initiatives and gain a better understanding of user needs and pain points. I still have a lot to learn! But I’m thrilled to be working with this team (and you!) and look forward to sharing more with you here :)
As mentioned in Teresa’s Q1 roadmap post, we’re working on making changes to how the question closing system works on Stack Overflow so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators.
Why are we doing this?
Our research has shown that often users hesitate to ask questions on Stack Overflow due to the fear of public, and sometimes caustic, feedback. From curators, we often hear about the time-consuming burden of manually guiding users whose questions get closed, and sometimes receiving pushback when they are just trying to help. And from moderators, we’ve heard the ask to reduce toxicity for all users through improved moderation and communication tools.
Today on Stack Overflow, roughly 20% of questions are edited after they are closed and just 3% of closed questions are ever reopened. We’d like to see more questions improved upon so that they have a better chance of being reopened and answered.
Our research and design team took a closer look at the problem, considered previously proposed solutions and feedback, and came up with what a long-term solution might look like. Then we broke it down into phases so that we can solicit feedback, test, and iterate along the way rather than take months on end to get something out there.
Goals
The goal of this project is to update existing close workflows with a streamlined system which is easier to understand and follow for all users and results in fewer negative interactions between them.
Askers are often unsure how to get their question reopened and onsite guidance can be confusing. They often experience a pile-on effect of downvotes and comments while trying to edit their question in parallel. This can feel overwhelming and intimidating for question askers and drive-by visitors.
Curators are burdened with an enormous backlog of tasks. We want to reduce the burden by helping question askers improve the quality of their question through product guidance, relieving curators of this tedious and often thankless task. We will also be testing automatic reopening, putting the onus on the question asker to improve and reopen their question and explore the notion of “hidden” as opposed to “closed” questions, giving askers the space to (somewhat more) privately improve their questions. More details on these concepts are below.
Things we’re not doing
As a quick call out, we’re not planning the following:
Network wide roll out - This project addresses pain points experienced on Stack Overflow. We’ll need to better understand the needs of the other network sites as this solution may not be needed, nor appropriate for other sites with less traffic and peer-to-peer friction. As of now, we don’t have any set plans in the nearterm to roll this out to the network. Having said that, if there are parts of this project that can relatively easily be rolled out to the network, we’ll try to prioritize them.
Automatic closures - We won’t be automatically closing questions. Questions will still need to be voted on for closure.
Getting rid of question closing - We have no intention of removing the question closing feature, and continue to embrace our commitment to maintaining high quality content on the site.
Feature Overview
Outlined below are the key components to this project. And a bit further down, I provide more detail on how we’ve phased this project out. While the team has validated the general direction through user, historical, and competitor research, we want to test these ideas and get feedback where we can along the way. Here are the key components of this project.
1. Close UX improvements
We’d like to reconceive “closed” as “hidden” so that users can improve their question without feeling embarrassed or exposed. It’s a gentler way to remove a low quality question from general visibility while giving the author a chance to edit it. Privileged users (mods and high rep users) will have a path to access these questions to provide additional coaching and guidance. We are still toying with ideas for what this path looks like, and we welcome your suggestions!
Inbox notifications and emails will alert post authors of newly hidden posts and provide guidance for editing. To maintain quality, if hidden posts aren’t edited within a certain timeframe, they will be deleted. And to mitigate abuse, we’d like to explore dynamic vote thresholds for closing based on age so that newer questions require fewer votes to close than older questions.
2. Edit UX Improvements
This new editing UX will provide guidance tailored to the specific reason the question was closed. We believe that this will help users make substantial edits that increase the likelihood of getting an answer. For askers, they get clear guidance about how to edit and reopen their question and ideally have a better experience on the site. For other editors, based on explanations in the "closed" post notice, they’ll know how to rehabilitate the question and help the question author. Changes to post notices have been rolling out over the last several months. We'll be continuously refining the content we show here based on user feedback.
You’ll notice the updated editor borrows UI patterns from the new Ask a Question form.
3. Automatic Reopening
When a user edits a hidden question in a substantial way, it will automatically reopen (unhide) the question and return to its pre-close, public state. Additionally, a question can only be automatically reopened once. Any subsequent reopens would require review through the reopen queue.
Here is a clickable prototype that illustrates these three core pieces. Note that final designs and copy may change.
Iterative and Incremental Development
The close-to-reopen workflow is a multi-touch process. In order to understand the impact of these various changes, to minimize disruption for users, and to avoid being in development for a lengthy period of time, we’re taking both an incremental (break the project down in smaller pieces) and iterative (continue to refine as progress is made) approach. Here’s how we’ve planned things so far.
The first phase, scheduled to release over the next couple of weeks, is much more solidified. Whereas each subsequent phase is less so, giving us room to adjust based one the data and feedback we see.
Phase 1
This first phase is made up of foundational changes and does not yet introduce the new concepts of automatic reopen or hidden questions. These are things we’d likely do anyway and provide the groundwork for the other phases.
Updates to close/off topic dialogue reasons
Updated editor for closed questions - pulling from the new Ask a Question design pattern, this provides better guidance through the editing flow.
Email and inbox notifications about closed questions - users will first receive an inbox notification when their question is closed. If the inbox notification isn’t seen, an email is sent to encourage them to make edits.
Phase 2
This phase introduces automatic reopening of questions when they’ve been substantially edited and passes questions with more than one closure to the reopen queue. At this stage, we plan to do some quantitative and qualitative testing to understand the impact automatic reopening has on askers (are questions more likely to be answered? closed again?) and curators (do we see a decline in low quality flags? are higher quality questions making it to the queue?). Deletion and dynamic thresholds for closing will be explored during this phase.
Phase 3
Reconceiving “closed” as “hidden.” When a question receives sufficient votes to become hidden:
- The post is hidden for primary views. Post authors and privileged users would have a direct path to view and edit. Anyone with a direct link can still access it as well.
- Score and voting capabilities are removed.
- Post notices, inbox notifications, and emails are all updated to reflect the new language.
- If the question isn’t reopened, it’s deleted after a certain period of time.
This step will also be an important piece to test and iterate on.
Questions & feedback
As mentioned, these phases are a rough outline of how we plan to tackle this project over the coming months. We plan to share our findings with you here at each milestone and get feedback along the way. For now, we’re happy to hear your thoughts or any concerns you have, as an answer below.
It should be noted that we still have some open questions that need discovery and you might think of something we’ve missed so feedback on things not mentioned here would be great. For example, what should we do with existing closed questions? There are lots of close questions today on Stack Overflow and we hear that users still find them helpful.
Thanks all for taking the time to read this and provide feedback at this stage.