80

In the wake of the famous blog post, I'd like to propose a rework of the reopen system.

The problem:

Poorly asked questions by newbies often get downvoted and closed. Sometimes, helpful users try to improve the question, and leave tips for the user to further improve it, and it gets improved, and put in the reopen queue.

After this, it probably stays in the reopen queue for a couple of hours (my reference is a low-traffic tag, might be shorter for some), and it ends up reopened, but still downvoted, and at the bottom of the new questions page.

Sometimes, these questions then get answers, often by users that were involved in helping, but often, they don't (in my experience, at least, don't have statistics). This leaves a bad experience for those users trying to improve, and might increase the perception that downvotes and close votes are rude and unwelcoming. It also leaves a bad user experience for the people trying to help, since the question they worked for to get into shape still is downvoted and doesn't get much attention.

I'd like to try to improve the user experience for these questions.


My proposal:

When such a question that has been substantially edited (enough to put it in the reopen queue) gets reopened, I'd like to see the same happen as when the question gets re-asked, giving the users a second chance:

  1. Bump the question to the top of the Newest page.
  2. Reset voting on the question if the net votes on the question are negative.

If possible, I'd like the reopen system to prioritize recently-asked questions with substantial edits and lots of activity, to further reward and welcome active OPs, and increase the impact of this change.

Advantages:

  • The OP gets rewarded more for his/her effort improving the question.
  • The subtext of closing changes a bit (in my perception), from you shouldn't have asked this question, and no-one should answer it, to you need to improve this question for it to have a chance of getting a good answer and upvotes. This might reduce the perceived rudeness of close voting.
  • The OP has less incentive to delete and re-ask the question.
  • Question bans can get lifted more easily by improving existing questions.
  • Since the experience for the ones helping improves too, more people might feel inclined to help people improve their questions

Furthermore, this is in line with we vote on content, not on people, in the way that if the content substantially changes enough for people to reopen a closed question, the votes change too.

Disadvantages:

  • Voters might feel they have less influence over moderation.
  • People might feel annoyed when they get to see the same question again, even though it has been substantially edited.
  • The Newest page is no longer fully chronological
  • Askers get an extra incentive to game the reopen system by editing a question in a totally different one (but this already happens to avoid bans, and reopen reviewers should be aware and act accordingly).

Addendum: This effect should only occur once per question, to avoid people being unable to vote in a close-reopen war and a close-reopen war repetitively bumping the question, and it shouldn't occur when a gold-badge owner closes or reopens a duplicate.


I have been thinking a lot about making the system to be more welcoming and promote perceived niceness over rudeness without compromising quality. This is the best I could come up with. I strongly encourage others to come up with feature requests with the same underlying motivation

43
  • 25
    Resetting the votes should be made explicit to reopeners; either it's a separate thing ("Yes, reopen and reset votes"), or it's made clear that the question will be reopened and the votes reset and that the question should be good enough to warrant that.
    – deceze Mod
    Commented May 1, 2018 at 10:33
  • 33
    I do worry about my votes being reset without any action from me. Perhaps some way of being notified that I can vote again due to it being re-opened?
    – DavidG
    Commented May 1, 2018 at 10:54
  • 8
    @DavidG I think notifying people voting to close or voting normally when a question gets re-opened is a good idea. (I thought about including it, but then I thought about notification spam, and then about making these notifications optional, and then I thought I was overthinking and omitted it :).)
    – Erik A
    Commented May 1, 2018 at 10:58
  • 10
    Define "Substantially edited"? I don't want my votes to be reset on questions I downvoted. No matter how much it is edited. If I deem the question good enough to retract my own vote, I will do so.
    – Cerbrus
    Commented May 1, 2018 at 11:06
  • 2
    This sounds like a very good idea to me. Some time ago, I asked how closing works on CS Educators SE as I gained access to the Close Votes queue and was told that it's better to close question early and reopen them later after being improved. I think that this closing strategy matches your proposal very well.
    – TuringTux
    Commented May 1, 2018 at 11:07
  • 4
    @DavidG Maybe that notification, "A question you downvoted has been edited and reopened", would help enough alone. Possibly along with an "A question you voted to close has been edited" notification.
    – Bergi
    Commented May 1, 2018 at 11:09
  • 27
    If it gets reopened then (if the system works) it should be of a proper standard so I'd be in favor of this. If my downvote is "lost" in the process then as far as I'm concerned that downvote has done its job and encouraged someone to improve the question. Doesn't matter to me if it stays around or not, I have close votes age away all the time already.
    – ivarni
    Commented May 1, 2018 at 11:10
  • 1
    @Cerbrus I've currently defined Substantially edited as edited enough to pass the mysterious limit that's required for a question to get in the reopen queue without a vote. A higher limit might be required, but as for most such limits, it probably should remain unknown. I get your sentiment regarding the votes, and it's the main disadvantage of this request imo. If voters were to revisit the question upon reopening and reconsider their vote, that would be best, of course, but I think that's rarely the case.
    – Erik A
    Commented May 1, 2018 at 11:12
  • 3
    @Cerbrus dupes could be excluded, or at least Mjolnir re-opens
    – DavidG
    Commented May 1, 2018 at 11:18
  • 35
    Wouldn't this penalize those users (and their questions) who (which) act according to the rules. If I behave like it's expected, I would ask a question, see it getting downvoted and commented on and then edit it accordingly before it gets closed. Such a question would then keep its downvotes and position down in the newest-question-list. If however, I only follow the rules after it's being closed, I am rewarded by having downvotes cancelled. Commented May 1, 2018 at 11:33
  • 4
    Very much support this. Everyone makes mistakes, especially when they are new at something, and the current zero tolerance policy is extremely hostile.
    – shogged
    Commented May 1, 2018 at 11:48
  • 6
    @Cerbrus Do you, on a regular basis, go back to all the posts you have down-voted, to determine if they are still worthy of a down vote? It seems to me that would become increasingly hard to do. Commented May 1, 2018 at 13:44
  • 9
    @Cerbrus what is the purpose of a vote? In my mind, it is to mark the quality of a question. A question that has been voted into oblivion, and closed has little chance to see the light of day when re-opened, unless it gets a clean slate. That is my point. The votes have done their job, there is no need for them to stick around. Maybe the re-opened question is still bad, but maybe it isn't we don't know until people vote on it. But I wouldn't want a question of mine to have to start out at -10 or -15. The current system encourages deletion and creation of a new question. But... Commented May 1, 2018 at 14:05
  • 8
    @DavidG to be honest, if I was notified "A question you voted [up/down] has been reopened and reset." I would definitely take a second, unbiased look at it! I think that's a great idea Commented May 1, 2018 at 14:48
  • 3
    I don't like the idea of automatically wiping out downvotes. But I'd love it if the system notified me when edits have been made to a question (or answer) that I downvoted so I can consider undoing or reversing my vote. I normally check such things an hour or two later, but I don't always remember to do so. Another nice option is if we could request to be pinged when a post has been edited that we haven't voted on. I like to leave constructive criticism on borderline posts, I subsequently vote depending on how the author has responded to the criticism.
    – PM 2Ring
    Commented May 1, 2018 at 16:49

7 Answers 7

18

I am strongly in agreement that the reopen system has a problem and cautiously optimistic that this proposal can ameliorate the problem. There are potential issues, but I believe the potential reward is worth the risk.

With that said, here are some of the things I think need to be considered.

Voting rings

If a voting ring casts a bunch of upvotes or downvotes, those can be invalidated. If a voting ring casts a bunch of reopen votes, thereby wiping away people's downvotes, it's not clear what would be the best way to undo this effect.

Rep loss

Not all negative-scored questions are rep-negative. A question at +5/-10 has actually gained its asker reputation. Similarly, if a new user gets 5 downvotes and then an upvote on their first question, they are now at 6 rep. If wiping votes unconditionally undoes their rep effect, then the user has now lost rep. There a number of possible ways to handle this situation, with possible effects like phantom rep (receiving rep from votes that no longer exist) or double downvotes (losing rep twice from the same person's downvotes on the same question).

Questioner edits vs community edits

If the questioner edits their question into shape, that demonstrates at least some ability to improve. (How much ability it demonstrates probably depends on how much the community had to walk them through it and whether they've gone through the process before.) If other users edit the question into shape for the questioner, that doesn't demonstrate the same thing. This may be worth accounting for when deciding the proposal's effects on question bans.

1
  • 8
    It seems to me that - if one goes the way of resetting votes - it should be rep-neutral. But personally I'd rather get notified and make my own decision about changing my vote. Commented May 1, 2018 at 22:07
17

I think you're headed the right direction, but it may be worth taking an incremental approach to start with, and re-evaluating based on data.

Specifically, I'd start with this: If the post is:

  • Downvoted
  • Closed
  • Edited by the OP (some minimal "substantial" metric may be needed; e.g., not just a title or tags tweak)
  • Reopened

...then notify downvoters that the question has been improved and reopened. Don't change their vote; they will if they agree. Don't notify downvoters if the post has been edited but not reopened. Reopening is a strong signal that the post has been improved.

No special placement, no automatic un-downvoting. It'll already be an active post (and if downvoters come back to have a second look, that's more activity).

If that's in place for a while, it'll generate data we can draw metrics from to see if it's doing the job or further tweaking (such as automatic vote-reversal) would be useful.


Another thing to consider doing (instead, in addition, before, or after) would be: If the post is:

  • Closed
  • Edited by the OP
  • A previous close-voter votes to reopen

...then notify other close-voters that the question has been improved and may be worthy of reopening.

That addresses the concern ImportanceOfBeingEarnest had that actively-engaged question posters who jump on improving their post wouldn't benefit from the improvement you're trying to make.


In both cases, offering opt-out of the notifications (but defaulting to opt-in) may be necessary, but only if volumes are outrageous.


And finally: Important to remove these features if they don't achieve their purpose.

3
  • 3
    I very much like this idea, especially in giving downvoters the prerogative without the obligation, but I can't help but notice the irony: If someone asks a bad question in earnest and then edits it to be a much better question, they'll benefit heavily if the question is actually closed. If a question only gathers four close votes before the user improves it, their downvotes stay for good. Perhaps this suggests that we shouldn't trigger only on reopening, but instead major OP edits and subsequent positive signals, which would naturally include reopening, upvotes, and positive answers. Commented May 2, 2018 at 17:57
  • @JeffBowman: Agreed, that's very much in the spirit of the above. Just about the most positive signal you can get is someone changing a downvote to an upvote. That may be a reasonable trigger for notifying other downvoters (after some short delay to allow for a mistaken vote to be corrected). Not sure what the threshold should be, but it's in the spirit, definitely. Commented May 3, 2018 at 8:32
  • 1
    +1 voters should be prompted to reconsider their downvote with no obligation or automation (without moderation). Commented Nov 30, 2018 at 9:41
6

Here are some closely related ideas that might work similarly or better:

  • Provide an automatic upvote from the Community user when an edited question is reopened (similar to how a Spam flag automatically adds a downvote from Community).
    • This would be limited to once per question (even if it repeatedly closed and reopened).
    • Possibly limit this to net negative voted questions only (e.g. a question at -1 would go to 0 upon reopening, but one that was at 3 would not go to 4 upon reopening), or net negative or zero voted questions only. This acknowledges that positive-voted questions are already considered well-received and thus don't really need an extra recovery "boost".
  • Add a widget to the Reopen Votes queue so that reopen reviewers can upvote the question in addition to voting to reopen.
  • Add the question to the Newest questions list as per the OP, but mark it in a special way (maybe a colored border, or a sunrise icon) so that it appears less deceptive. Perhaps an opt-out option could be provided to suppress reopened questions so that purists (who truly want to see only new questions) can still do so.
2
  • don't worry, community user can only vote once :) Commented May 2, 2018 at 20:03
  • 2
    "Add a widget to the Reopen Votes queue so that reopen reviewers can upvote the question in addition to voting to reopen" that has been asked before, for other queues as well, on meta SE and SO. Commented May 2, 2018 at 20:03
6

I do not want my votes to be reset. I think a better alternative would be to introduce a system that would allow me to review items that I voted on that have received edits or have been reopened since I initially voted. I don’t think that notifications are the correct way to do this either, maybe make it a special queue.

For some bad content, I will sit on the post after downvoting and periodically check whether the problems have been fixed. I’ll wait maybe 20 minutes and then move on/give up. It might be helpful to incentivize going back and reviewing content that you have already voted on after it has undergone changes.

I don’t like the idea of outright resetting votes because it wipes out me taking the time to vote on something. Effectively, you’re letting the OP post the same thing again and hoping that they got it right. That seems like it could be an abuse vector. It also makes my downvotes seem pointless.

Don’t take away the only mechanism that seems to prod people enough to get them to fix their posts, give voters a way to see that their voting is actually helping.

1

One other advantage of the proposal: reduce the odds that people will downvoted and vote to close the same question at the same time. I looked at the data and that's happened about 800,000 times on Stack Overflow. Close votes should communicate: this question isn't answerable right now so go back to the drawing board and fix it. Downvotes on questions communicate . . . honestly, I have a hard time expressing what they should communicate. The tooltip mentions lack of "research effort", but wouldn't that be a good reason to close instead? If you don't yet have the close vote privilege, downvoting is a reasonable alternative, I suppose.

At any rate, it's likely that downvotes don't much matter to people unwilling to put in the time and effort to ask good questions. They can be crippling to those who try by editing. In addition, we have reason to believe that the more interactions people observe on the site, the more likely they are to ask again. So downvoting and closing might be more encouraging than people might hope.

Meanwhile, we have some precedent for wiping the slate clean:

  • After serving a suspension, accounts are returned to normal.
  • When a question is reopened, we remove the comments autogenerated by close votes. (Also the other trappings of closer such as the title change and post notice.)
  • If a post is deleted (and doesn't meet certain criteria) reputation changes are caused by voting on the post are reverted.

The last is particularly relevant since a good way to avoid the cost of downvotes on a bad question is to self-delete it and re-ask. (Well, good until the system blocks you from asking.) Obviously, we'd rather people edit their question and get it reopened. So I like the idea of treating a reopened question as a new question.

[I'm still thinking about this idea. The goal is solid. Hopefully I'll have more to say in a few days.]

19
  • 2
    "The tooltip mentions lack of 'research effort', but wouldn't that be a good reason to close instead?" As a duplicate maybe (see also). But not other reasons, unless lack of research effort is the reason the question was off-topic to begin with.
    – BoltClock Mod
    Commented May 1, 2018 at 19:06
  • "wouldn't that be a good reason to close instead" ... I haven't seen this position from Community Team yet. Should Stack Overflow (and Stack Exchange in general) be awarding “A”s for Effort?. It's reasonable to downvote a "please send the codes" question and not vote to close: downvoting enables automatic deletion if there is no answer, but also allows for an answer to be given, which after all may prove to be useful code.
    – user6655984
    Commented May 1, 2018 at 19:31
  • 2
    @Jon: "Downvotes on questions communicate . . . honestly, I have a hard time expressing what they should communicate." They communicate that the post is lacking in quality. This is not for the person making the post, but for the following people wondering about its quality. Commented May 1, 2018 at 19:41
  • "After serving a suspension, accounts are returned to normal." That's completely different. You don't remove the votes of a suspended account. You never actually remove the reputation of a suspended account. You simply show their rep as "1". Commented May 1, 2018 at 19:42
  • 1
    @bro: It's helpful to look at old problems with different lenses. I think answer downvotes make a ton of sense: they help sort the page. But on questions, especially on a site that gets thousands a day, downvotes only really serve to cost the asker reputation and feed systems to block them from asking. Those are useful effects, to be sure, but it's hard to argue they target content rather than the user unless people retract votes after edits. Anyway, I'm still thinking about the idea.
    – Jon Ericson Staff
    Commented May 1, 2018 at 20:52
  • @NicolBolas: Good point. If the question is closed, edited and reopened, those signposts are potentially outdated.
    – Jon Ericson Staff
    Commented May 1, 2018 at 20:54
  • 1
    On a site where close votes take forever to take effect (and often expire unreviewed), downvotes remain the only indicator of poor quality that one can use to filter the stream.
    – user6655984
    Commented May 1, 2018 at 21:11
  • I think it often makes a lot of sense to also downvote when voting to close. The vote should just be a quality indicator and if you are voting to close a question, it is lacking quality. The problem lies somewhere else, IMHO: It just doesn't happen often enough, that a vote on a question is reconsidered after it was edited. Maybe there could be a very simple feature to help improve on that: Have some way to see what posts you already voted on were modified after your latest vote.
    – user2371524
    Commented May 2, 2018 at 11:13
  • 3
    @Felix it is entirely possible to have a well written, interesting question that is off-topic for the site for one reason or another. There are many situations where a question should be closed but doesn't warrant a downvote. The two voting systems serve different purposes and should not be conflated.
    – user4639281
    Commented May 2, 2018 at 15:04
  • @TinyGiant note I wrote often. Of course there are such cases, but often, the question should be closed because it's very low quality, and that deserves a downvote.
    – user2371524
    Commented May 2, 2018 at 15:32
  • @FelixPalmen I've never closed a question as being very low quality. As far as I know that is not a close reason. I downvote low quality content, and vote to close off-topic questions. There is no correlation between topicality and quality, even if it may be common for off-topic questions to be low quality. You assert: "The vote should just be a quality indicator and if you are voting to close a question, it is lacking quality.", which is false given that close votes are not inherently quality indicators, but rather topicality indicators.
    – user4639281
    Commented May 2, 2018 at 15:39
  • @TinyGiant I don't know what tags you're looking at, but "unclear what you're asking" is clearly a problem with the question quality, as welll as is "no MCVE". For the other "off-topic" reasons, it is very rare to find a good question that should be closed anyways, and of course, I don't downvote such a question. But it is the standard case that questions to be closed are low quality.
    – user2371524
    Commented May 2, 2018 at 15:41
  • A question can be of reasonable quality while lacking clarity. A debugging question question can be completely well-formed but be missing a vital line of code necessary to diagnose the issue. In these cases closure is beneficial because it prevents "guess" answers that become irrelevant noise once the question is clarified. Such questions are not lacking in quality, but rather clarity. @Felix
    – user4639281
    Commented May 2, 2018 at 15:47
  • @TinyGiant I never said these don't exist. But either you're seeing a different reality than me or it's wishful thinking -- what I see every time I look at SO is that most questions to be closed are, indeed, low quality.
    – user2371524
    Commented May 2, 2018 at 15:49
  • 2
    @Felix i am in no way denying that the most common outcome you see is for a question to be of both low quality and low topicality. That does not change the fact that the two systems are separate, serve different purposes, should be evaluated separately, and should not be conflated. Down votes are used to indicate quality while close votes are used to indicate topicality.
    – user4639281
    Commented May 2, 2018 at 16:16
-2

I agree that improving reopening is one of the most fundamental things that Stack Overflow can change to become more welcoming to novices. It's my (unwritten) answer to this question.

But I think the improvement can be simpler than what you've proposed. My suggestion would be:

When a post is edited, notify users who downvoted the post to take another look.

There might be a threshold to make sure that only non-trivial edits trigger notifications. And we could group the notifications in a way that wouldn't be spammy. They don't necessarily have to be mixed in with regular notifications, but there should be a place for me to see posts I've downvoted that have since been edited.

In my humble opinion, the biggest problem with downvoting, closing, and reopening is that people (myself included) tend to throw out a ton of drive-by downvotes without ever looking at the post again. This means that even if the user goes through a ton of trouble of editing and improving their post, nobody is going to take a second look.

I think a simple solution to this problem is to notify users (or allow users to see) when a post they've downvoted has been significantly edited.

6
  • 6
    This will flood your notifications so fast you'd think someone hacked SO
    – Passer By
    Commented May 1, 2018 at 17:51
  • 6
    @PasserBy We could group the notifications in a way that wouldn't be spammy. They don't necessarily have to be mixed in with regular notifications, but there should be a place for me to see posts I've downvoted that have since been edited. Commented May 1, 2018 at 17:54
  • That I can agree on, if it isn't a notification but a history page I can review.
    – Passer By
    Commented May 1, 2018 at 17:56
  • Opt-in/opt-out could also help. Commented May 1, 2018 at 18:13
  • 3
    In SOCVR, we have a bot that monitors questions which people have been asked to look at wrt. close voting for when they are edited. Even though that bot only notifies when the edit is substantial enough to be above a change threshold, the vast majority of edits are not sufficient to make the question on-topic. My concern with this proposal is that the number of notifications will be so significant and so unlikely to represent a change that actually matters such that people will come to automatically ignore the notifications (due to having such a low usefulness when they do look at them).
    – Makyen Mod
    Commented May 1, 2018 at 18:35
  • 2
    @PasserBy A userscript already exists which will tell you which posts have changed after you voted on them: Get a list of posts you've voted on that have been edited
    – Makyen Mod
    Commented May 1, 2018 at 18:37
-5

Reopen serves two purposes.

The first is what is proposed here, to allow a low quality question to be edited and reintroduced. The second is when a contentious post's closure needs to be overruled.

While I agree the reopen system is lackluster for the first function and it needs change, I disagree on the resolution.

Comparing the proposal and deletion + repost

Bump the question to the top of the Newest page.

Both does this.

Reset voting on the question if the net votes on the question are negative.

Deletion + repost resets rep while the proposal have yet-to-be-decided effects on net positive rep gain. There isn't a significant difference here.

However, the proposal have additional complexity and loopholes associated with it, if there isn't much to gain, there isn't a point to do so.

Another resolution

I suggest ditching the idea that reopening can be used to revitalize a very low quality post at all, and fully embrace deletion + reposting.

Weighing towards a question ban is intended. When a user posts a very low quality question in need of intensive help, it sucks up resource and time regardless if it improved or not later on. The ban is there to put a limit on how much a single user can do so. If the user repeatedly requires such intensive help on their questions, a ban is justified.

If the candidates of being revitalized truly underwent a significant edit, and is pretty much a different question now, then reposting is correct.

9
  • 4
    "fully embrace deletion + reposting" That makes no sense. Deleting downvoted posts negatively impacts ones ability to ask questions, possibly leading to a question ban. And that's not something we want to change. Commented May 1, 2018 at 19:43
  • 1
    If a user shows a pattern of requiring intensive help to get their questions into shape, with no improvement in initial question quality over time, a question ban is warranted. However, users are supposed to be able to escape a question ban by demonstrating that they now understand what a good question looks like and how to write one. Commented May 1, 2018 at 19:52
  • If we embrace delete+repost, users have even less hope. If a user deletes a question and reposts an improved-but-not-enough version, they get slammed again despite demonstrating improvement. Once they are question-banned, they cannot delete and repost, because they are question-banned. Commented May 1, 2018 at 19:57
  • Also, delete+repost obscures a question's history of improvement (or non-improvement). Commented May 1, 2018 at 20:02
  • @NicolBolas I am challenging the idea, the old system didn't work well.
    – Passer By
    Commented May 1, 2018 at 23:41
  • @user2357112 A user demonstrates that they now know how to write a good question by writing one, which won't need that much help and won't be closed in the first place. I did realize this meant not having the oversight of someone in the review queue, I'm not sure how to deal with that, but my initial thoughts was the review queue wasn't made for tutoring as well and it's still a resource and time sink. If we want to make sure the user is given enough help, there should be another system
    – Passer By
    Commented May 1, 2018 at 23:49
  • "A user demonstrates that they now know how to write a good question by writing one" - but if they're question-banned, they can't post a new, good question. Commented May 1, 2018 at 23:54
  • @user2357112 The history and improvement isn't a factor in the question's quality nor its usefulness. The argument being if the question did go through such a dramatic change to warrant being bumped to Newest, the question should be good enough to be answered as is. This then is in effect the same as reposting
    – Passer By
    Commented May 1, 2018 at 23:59
  • @user2357112 If the reason for trying to edit a question that is not likely to receive an answer into shape is to lift a ban, I think something needs fixing
    – Passer By
    Commented May 2, 2018 at 0:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .