505

This experiment has concluded and the affected settings have been restored to their previous values.
Results are available here and here.

-- Shog9, 2019-09-07

For the next 30 days, we'll be running an experiment on Stack Overflow: the threshold for closing or reopening a question will be set at 3 votes instead of 5. The primary goal here is to determine what effects this has on folks' ability to successfully close or reopen questions; we're also hoping to observe the larger effects - if any - such a change might have on the behavior of voters, flaggers and editors.

This is an initial foray into research on how we might make the venerable closing system more effective and less frustrating for all parties involved.

Background

Close voting was introduced as a concept at the very end of 2008, with the original threshold set to 3 votes to close, 3 to reopen. The explicit goal was to make closing more deliberate by restricting the influence that any individual could exert on it. The secondary effects - on reopening, on editing, on duplicate identification, on participation in smaller tags - were not apparent until much later. The closing system worked very differently back then; some notable differences included the ability of any one voter to vote to close the same question multiple times, a lack of any review system, and a much less guided process for closing duplicates. A few weeks after the voting system's introduction, the threshold was raised to 5... And has stayed there ever since. As a result, we have very little idea of what different threshold values might mean on today's Stack Overflow - hence the purpose of this experiment.

Numerous people have suggested such a change over the years, far too many for me to give credit to them all. So here are a few highlights:

  • EJoshuaS wrote a very well-researched proposal for reducing this threshold (including a reference to a prescient observation from tvanfosson in 2009!)
  • Andre Silva wrote another well-researched proposal for conditionally reducing the threshold based on the order of reviews
  • Yvette Colomb has been gently prodding us for months to reduce the threshold in some scenario just to reduce frustration for folks doing close reviews
  • Jon Ericson tested this once before on Software Engineering - from which we learned the importance of nailing down useful, plausible goals ahead of time.

Also related: How high an invalidation rate do reviews currently have?

Tangential effects

Currently, migrating a question to another site requires that 4 of the 5 votes all agree that the question should be migrated to a specific site.

As migration would be blocked otherwise, I'm also temporarily lowering the threshold for migration from 4 votes to 3 - essentially making a migration require unanimous agreement from close voters.

Also... There are, of course, a LOT of questions with three close votes currently pending. These will not be instantly closed - they will instead close upon receiving another vote (so if they are closed then they'll be closed with four votes, or possibly three if one or more pending votes age away before a fourth is received).

Concrete goals

Obviously, more questions will get closed and more closed questions will be reopened when the thresholds are lower; that's not what we're testing. Rather, we're looking to improve the efficacy of individual actions: if you, a member of Stack Overflow, vote or flag a question for closure, vote to reopen a question, or edit a closed question... How likely is that action to have any tangible effect on the outcome?

Next steps

  • If you observe any serious bugs, please report them here.

  • If you observe bad behavior, please flag it or open a discussion here on meta. In both cases, please focus on the behavior, not the people involved; let's keep this positive and constructive!

  • If you observe anything interesting, write it up!

At the end of 30 days, we'll restore the threshold to 5 (and the migration threshold to 4). Then spend some time analyzing the results. Depending on the outcome, we may adjust the threshold further in the future, leave it alone, or... Try something else.


Community Alternative Future Experiments (Suggestions from comments)

  • Lower the threshold to 4 votes but give gold tag badge holders two votes instead of one (provided they also have the standard moderation badges). – Pearly Spencer
17
  • 16
    Perhaps it would be a good idea to have the best of both worlds? Like requiring 5 votes to close, but if the question has pending close votes / certain amounts of downvotes / no edits / no answers (just general ideas) and is x amounts of days old, the threshold lowers to 4 and eventually to 3. This so it's more likely to get picked up eventually, but less likely to be abused in any way.
    – icecub
    Commented Aug 8, 2019 at 23:05
  • 79
    I understand why not being able to vote to close a question multiple times might be a good idea to avoid abuse, but being able to change your close vote reason or retract your vote and resubmit it prior to the question being closed might be a good idea. Sometimes a question is edited from an "Unclear what you're asking" to a clear duplicate of another question, but because you've already close-voted it, you can't point the user at the answer to their problems (other than by leaving a comment). Commented Aug 9, 2019 at 4:47
  • 26
    My thoughts would be to give trusted users more weight when voting instead of lowering the overall requirement. Gold tag badge = 3 votes, silver = 2, perhaps as it would still require at least two people for non-duplicates. This would prevent abuse by users that aren't experienced with the system but would allow users that know what they are doing to quickly close or reopen issues which obviously need it. Commented Aug 9, 2019 at 9:23
  • 8
    3 votes for most questions SO gets looks great, but one potential issue is that it's now a lot easier to close (or reopen) old questions with lots of votes. Perhaps more of a consensus should be required for those big old questions that still get lots of hits (despite possibly being closed), a bit like delete votes scale. Not 100% sure, just a situation to think about Commented Aug 12, 2019 at 21:15
  • 16
    I reviewed now for a couple of days and while I like the feeling of increased weight and achievement that the lower number of votes brings, still my enthusiasm is fading quickly again. It's just work and it never ends, the queue is sometimes larger after I finished my reviews than before and it's unpaid. I don't think I can do it really in the long term. It's a pity, because I feel like I cleaned up quite a lot and did something good and I feel the site will surely be for the worse if nobody would do it. I just admire the ones doing thousands of reviews. These persons are heroes! I'm not. Commented Aug 21, 2019 at 12:22
  • 5
    Mod Note we've done a massive comment clean up. @Shog9 can still see everyone's suggestions when writing an answer. I recommend people write an answer if their ideas are not already included in the answers.
    – user3956566
    Commented Sep 10, 2019 at 11:50
  • 5
    I notice the close vote queue is already 50% longer than it was at the end of the experiment...
    – Nick
    Commented Sep 14, 2019 at 5:16
  • 11
    Just FYI: I've spent the past 3 days chasing down a bug in one of my queries that was heavily skewing the results. Ain't it funny how you can stare at something and just... not see it? Anyway, that set me back a bit, but still working on this.
    – Shog9
    Commented Sep 14, 2019 at 20:24
  • 6
    @Shog9 Update? How are you doing on this?
    – S.S. Anne
    Commented Sep 19, 2019 at 2:08
  • 2
    based on personal observations, I would prefer to re-run the experiment for a longer time span, like 2 or 3 months. Although my general impression was quite positive, I would like to deeper study longer-term impact of such a change. That is, unless @Shog's findings provide sufficient data for clear yay or nay for it already
    – gnat
    Commented Sep 19, 2019 at 15:23
  • 4
    Still plugging away as I have time, @JL2210. Been a hell of a week for... unrelated reasons :(
    – Shog9
    Commented Sep 20, 2019 at 20:33
  • 9
    Just before experiment was over CV queue was under 4K, sometimes even under 3K. Reopen queue was not overwhelmed. Now two week after experiment ended CV queue is back on 7K or more. I don't need any other numbers to say experiment was successful. 3 close votes are better than 5.
    – Dalija Prasnikar Mod
    Commented Sep 22, 2019 at 17:12
  • 4
    Whats the status?
    – Nouman
    Commented Sep 23, 2019 at 11:55
  • 15
    Report is done. Results are fantastic. Got colleagues checking my work. Week was shit otherwise, but happy about this one thing. More next week! @jl2210
    – Shog9
    Commented Sep 28, 2019 at 2:35
  • 1
    @Shog9 awesome, thanks, here's hoping your next will be delightful Commented Sep 28, 2019 at 11:21

30 Answers 30

195

Awesome, thank you! I very much hope we will have more order with a distinct lack of an increase in the number of dumpster fires.

Now is probably the best time to visit the close vote queue. I know I've been neglecting that for a very long time, but this gives me motivation to go back in with a shovel.

32
  • 103
    All ye review strikers, report back!
    – Bhargav Rao Mod
    Commented Aug 8, 2019 at 19:06
  • 53
    The effect this experiment has on the CVQ is actually one of the most interesting things to look at IMO.
    – Baum mit Augen Mod
    Commented Aug 8, 2019 at 19:10
  • 24
    @BhargavRao this is actually motivation to use the review queues again. This should significantly lower the amount of wasted reviews Commented Aug 8, 2019 at 19:28
  • 11
    @BhargavRao I'll actually be heading to the CVQ for the first time in quite a while because of this change. When posts need 5 close votes, even from the queue, it leads to far too much wasted effort.
    – jhpratt
    Commented Aug 8, 2019 at 19:32
  • 7
    @BhargavRao ain't gonna happen - reasons for alienation plenty and already exposed, but ignored (and practically derided); it's a perfectly understood move though - alienating reviewers predictably led to less reviewers, so why not lowering the requirements?
    – desertnaut
    Commented Aug 8, 2019 at 21:32
  • 27
    Eh, there's already lots of negative atmosphere @desertnaut. I think it is time to enjoy some positive changes, while it lasts, without getting too pessimistic.
    – Bhargav Rao Mod
    Commented Aug 8, 2019 at 21:52
  • 5
    Yes, this inspired me to go and do some reviewing. I did 6 and couldn't take it any more :(
    – DavidG
    Commented Aug 9, 2019 at 11:54
  • 13
    @DavidG then it can't be helped. Reviewing is not for everyone. Commented Aug 9, 2019 at 12:01
  • 5
    @AndrasDeak The problem for me is that it's both thankless and rewardless.
    – DavidG
    Commented Aug 9, 2019 at 12:03
  • 8
    @DavidG I agree that it's thankless but it's not without reward. You're doing everyone a favour by separating the chaff from the wheat. But of course I understand where you're coming from, the insurmountability of the CVQ is why I stopped doing reviews. But I think with the experiment we have a chance of effecting change. No pressure though; do what you're comfortable with. Commented Aug 9, 2019 at 12:05
  • 49
    @DavidG many still believe that SO is primarily a knowledge base. Closing close-worthy questions is the primary guarantee of a high signal-to-noise ratio beside downvotes. There's only so many things we can do to keep the site tidy, and this is one. When you go to SO to search a problem you shouldn't have to wade through dozens of unclear questions with a similarly hazy answer that somehow helped that one asker. You want clear questions with clear answers. If achieving this goal is not helpful then I don't know what is :) Commented Aug 9, 2019 at 12:08
  • 9
    @DavidG Apart from Andras's mesasge, revieweing helps me reading thousands of lines of code in a way that helps me understand and grok code and intent faster, much faster than if I were not exposed to reviews. I see the difference from my colleagues. I stopped believing in "nobody wants to read other people's code". It's stopped being an issue for me. Commented Aug 9, 2019 at 12:19
  • 5
    @smci those also get reopened easily if closure was not warranted or if the question was fixed in the meantime. Commented Aug 13, 2019 at 8:38
  • 14
    @smci you seem to have a biased view of how closing should work on SO. Most people (including CMs) will probably agree that the best closure is a fast closure. Ask question -> is it unfit? Close it! -> educate the asker, force them to clarify their question or make it otherwise answerable -> reopen (reopen queue!), everyone's happier. Commented Aug 13, 2019 at 9:19
  • 5
    @smci: Question needs improvement => close it. Period. There's a reason that's called "on hold" for a period of time. I don't think I misrepresented you at all: the position you just described is the position I'm arguing against. With oodles of material in the help centre and tour, plus literally over a million questions to look at for inspiration before posting a new one, there's really no excuse for getting it wrong; I don't care how new someone is. Commented Sep 9, 2019 at 22:58
153

This experiment will turn into a pumpkin end in the very beginning of September. It somehow feels suboptimal given that it is the time when school year starts (in northern hemisphere) followed by multiple low quality homework dumps.

Consider running it for 10-20 days longer. This will let you collect data about how such a change may impact the system working under stress.

Or maybe think of some other ways to tame possible shock when people will find that things got back to even worse state than it was before the experiment.


Speaking of other ways to smoother end of experiment, one that comes to mind is to raise a system flag for mod attention if a question with 3-4 close votes gets stuck in review queue for longer than a day or two, to let them finish it with a binding vote.

Attentive readers may notice that this somehow resembles the way how LQ review works - well yes, I got this idea when pondering about how come that LQ queue works so much better compared to CV review, maybe one of key differences that made it succeed is having a "moderator backup" like that.

Personally, I would prefer to spare mods from handling slippery cases and wouldn't let system raise automatic flags on questions with edits or Leave Open votes. I don't insist on that though, guess it more depends on how much extra work you would want to off-load to moderators.

20
  • 9
    Realistically, that would be another test (and probably closer to the situation on Software Engineering I referenced above).
    – Shog9
    Commented Aug 8, 2019 at 21:33
  • 42
    you would have to wait for a full year if you wanted to have this other test separately, that's not too good. Also, think of how it will feel for people to discover how easy closing of homework dumps will suddenly change back to hard right in the start of a yearly flood. Could be quite frustrating
    – gnat
    Commented Aug 8, 2019 at 21:40
  • 4
    @Shog9 Perhaps another test like the one mentioned in the post, such as 4 CVs with gold badges counting as two?
    – jhpratt
    Commented Aug 8, 2019 at 22:36
  • 7
    There's a similar bump in the spring, @gnat.
    – Shog9
    Commented Aug 8, 2019 at 22:42
  • 15
    well @Shog9 having to wait till next May is not much better than next September I'm afraid. Although my primary concern is about how people will feel when things suddenly change back to even worse than it was prior to experiment. Unless this is what you want to test, you better consider some measures to tame a possible shock
    – gnat
    Commented Aug 9, 2019 at 8:25
  • 2
    Well, that assumes the experiment is a success, @gnat. Which, I hope it is... But if so, that's gonna be a let-down at the end for at least a while anyway unless we set it up to end on... Christmas day. Which would have its own set of problems.
    – Shog9
    Commented Aug 9, 2019 at 18:48
  • 2
    @Sayse worth noting that it will skew results anyway, no matter if experiment lasts longer or as originally planned because currently it is expected to run for first 7-8 days in September. I think that no matter what, it would be better to somehow separately analyze results obtained in August and September - even with current schedule
    – gnat
    Commented Aug 12, 2019 at 11:24
  • 2
    @gnat - Thats true, I also just noticed your question's september link points the the very thing I was on about anyway which made my comment irrelevant
    – Sayse
    Commented Aug 12, 2019 at 11:26
  • 11
    @Clonkex Start of school year (northern hemisphere) Commented Aug 13, 2019 at 4:04
  • 4
    @gnat We're usually experiencing 2 waves, which start in September and February. Commented Aug 13, 2019 at 15:26
  • 2
    @πάνταῥεῖ I'm mostly concerned about how experiment intersects with one of the waves, no matter which. If it run in July or October I probably wouldn't worry. Or if it ended near mid September instead of beginning
    – gnat
    Commented Aug 13, 2019 at 17:22
  • 12
    Having lived through the first transition from 3->5 votes, I can attest: it sucks ass. But, gotta do what we gotta do. FOR SCIENCE!
    – Shog9
    Commented Aug 21, 2019 at 15:16
  • 2
    I like the idea of the moderator flag for stuck reviews. Maybe this should also be a separate feature request to not get lost if it's just here. Commented Aug 30, 2019 at 10:55
  • 3
    @Trilarion yeah my plan is to make such a FR after the dust settles (in fact I was going to make it already but experiment started before I finished drafting it and I decided to postpone, to avoid interfering with stuff related to experiment)
    – gnat
    Commented Aug 30, 2019 at 11:13
  • 5
    Low quality homework dumps by a couple of school kids aren’t even close to being my main beef with this site. If this could help to reduce the endless onslaught of people who put fancy job descriptions onto their profiles, but then ask questions that are nothing more than blatant “outsourcing” attempts, I am all for it.
    – misorude
    Commented Sep 6, 2019 at 9:30
102

A few days in, let me offer some observations on the psychological effects of this change, purely based on introspection (n=1).

  • I have been a frequent contributor to the SOCVR chat room, but some days (especially on weekends, it seems), there was way more closeworthy content than room policy would allow me to submit, and often, it felt like my close votes would expire anyway for anything which wasn't trivially obviously off-topic. Now, there is again some hope that close votes which I don't escalate to the room will not simply age away. Maybe I don't even need to raise the slam-dunk cases in the room because they will get closed organically without coordination "behind the curtains".
  • Revisiting the close vote queue doesn't feel like a supreme waste of my time. I can click Skip without the nagging feeling that nobody else is going to review the post and that it will remain open in eternity.
  • Paradoxically, I now vote to close as unclear on posts which might be salvageable if the OP sees my comment. Previously, I would abstain from voting, but make a mental note to review for edits - but when I did, the usual outcome was "nope, they didn't edit" or "the edit didn't help much at all." Now, it seems feasible that a good edit will be noticed and picked up, and attract enough reopen votes - in the past, this felt like entirely too optimistic. Because reopening is feasible, I can be more efficient in the closing phase, and trust the system a little bit again.

It's obviously too early to say if these reactions are at all grounded in reality. Again, this is just a report from the ground about how I feel at this point about this change - A+++, would upvote again!

Before I discovered SOCVR, many years ago, it felt as if my close votes were purely a distant signal to some digital archaeologist maybe 300 years from now as evidence that close votes were actually not entirely ignored by all users even though casting them was clearly pointless.

4
  • 26
    I sympathise completely with this answer and much of this type of 'moderation' is what led me into becoming a "moderator". I remember clearly the feeling of futility in trying to review in the close vote queue. Honestly - it's the best tool the network can give us- the ability to close low qual posts expediently.
    – user3956566
    Commented Aug 10, 2019 at 11:28
  • 22
    This is helpful. Thanks for taking the time to reflect and share your observations! I'd be interesting to hear how it may change after a few weeks.
    – Meg Risdal StaffMod
    Commented Aug 10, 2019 at 18:30
  • "A few days in"? How many days precisely? 48 hours or less? What about today, one week later?
    – Mari-Lou A
    Commented Aug 16, 2019 at 12:16
  • 6
    @Mari-LouA That was a couple of days after the announcement. My impressions have not changed significantly since then, though.
    – tripleee
    Commented Aug 16, 2019 at 12:39
76

I like that it's easier to put questions on hold, the following concern might have solutions other than to rollback.

The close reason might be misleading more frequently

Since the 3 votes could point to different reasons with no majority (nothing new here, just more frequent now), the close reason could be misleading to OP (or any visitors).

Let me illustrate with a simple example (that I saw happen)

  1. OP post unclear question
  2. Someone votes "close as unclear"
  3. Another user votes "Off-topic, needs a MCVE"
  4. OP edits with code that is clearly a duplicate.
  5. Someone votes for a duplicate candidate.
  6. The question gets closed as unclear.

Now, anyone visiting sees "Put on hold as unclear" while the duplicate is buried in the comments.

As mentioned by Shog9 in the comments

when there's no clear majority, the oldest vote wins

But since it's now more frequent to get no majority, would it make more sense to use the newest vote? Since that vote has more chance to take into account the current state of the question.

Or what gnat suggested in 2016:

Exposing close votes in the timeline

Or my own suggestion, listing the close reasons in the "Put on hold" message at the bottom of the question instead of choosing one. Now, there would be a max of 3 reasons, which is reasonable!

It's maybe a good time to revisit these (if the experiment is a success):

enter image description here (source)


Proof

In the moderation tools:

enter image description here

In the question after my duplicate vote:

enter image description here

30
  • 2
    No, last vote breaks any ties.
    – Braiam
    Commented Aug 12, 2019 at 19:43
  • 4
    @Braiam I just encountered it in the wild... Commented Aug 12, 2019 at 19:45
  • Er. There's no record you can see that tells who voted which way, unless you're watching live and keeping track manually. It's going to be hard to prove this. In the event of an equal number, the majority vote is used. If you use three separate close reasons, I believe the system will display the last one used.
    – fbueckert
    Commented Aug 12, 2019 at 19:49
  • @Braiam or is it just a bug? Like you're right but it doesn't work anymore with 3 votes? Commented Aug 12, 2019 at 19:50
  • 2
    Well, I was working with old information apparently meta.stackexchange.com/a/187301/213575
    – Braiam
    Commented Aug 12, 2019 at 19:58
  • 1
    Well, that's the thing, Emile; you can't. I'm not even sure mods can. I'm just pointing out the potential that perhaps someone flagged it as off-topic, and that's why it shows up like that in the review mod tools. So, not so much counting them, as accounting for them.
    – fbueckert
    Commented Aug 12, 2019 at 20:10
  • 1
    Well, that's not a new concern; it happens whenever we get multiple reasons to close a question. It just means two is the majority now, instead of three. My guess is both Quentin and Pointy voted unclear, and that overrode your dupe close vote reason.
    – fbueckert
    Commented Aug 12, 2019 at 20:21
  • 21
    Actual rules for determining close reason are documented here: meta.stackexchange.com/questions/304663/… - tl;dr: when there's no clear majority, the oldest vote wins.
    – Shog9
    Commented Aug 12, 2019 at 20:29
  • 1
    Similar thing happened to me today also. I was third vote and didn't like other 2 reasons. One of those other reasons became the final one. With 3 vs 5 votes to close may not have any majority
    – charlietfl
    Commented Aug 12, 2019 at 21:25
  • 7
    related: After question is closed expose close votes distinguished by reason in the timeline page "Exposing close votes in the timeline is probably the easiest of all options here: doesn't require backfilling data for close records, doesn't require adding stuff to the data-dump... If folks are happy to see their close votes exposed on an obscure page, that's actually feasible..."
    – gnat
    Commented Aug 13, 2019 at 8:36
  • 3
    I agree with showing all 3 close reasons, but for a different reason: the majority isn't always right! Minority opinions can add value too. One example, a really bad questions asks for a code review (one that would not be welcome on CR). 1st vote - move to CR, 2nd a pile on move to CR vote, 3rd Unclear maybe with a (risky) comment that CR is a bad idea. Result is a message to OP to move it to CR, which is the worst thing we could tell them. Same scenario can play out with Dup votes around a bad Q that might be a dup of something else Commented Aug 13, 2019 at 20:29
  • 5
    @chrisneilsen I agree, there's a ton of reasons why the current behaviour is less than perfect. It also list our names close to the others with the single chosen close reason as if we voted for that reason, it puts words in our mouth... Commented Aug 13, 2019 at 20:32
  • 3
    Just happens to me: 3 voters with different reasons, it seems the first voter reason was chosen as comment for close reason. Feature request: show ALL reasons somehow.
    – Sinatr
    Commented Aug 14, 2019 at 7:02
  • 3
    This answer should be a feature request, regardless of the number of votes. Commented Aug 15, 2019 at 10:18
  • 2
    @Trilarion there are already a bunch of feature requests for that. One at MSE is listed in my comment above, and we have it here at MSO too, see eg I gave a different close reason yet the system lists me and questions linked to it
    – gnat
    Commented Aug 17, 2019 at 7:56
76

Please bring this back.

This change helped questions be closed, fixed, and reopened at a faster pace than usual. For that reason, I'd like this to be brought back (at least for a little while). If it's not asking too much, I'd also like this to be made permanent.

6
  • 21
    Absolutely. It made such a difference in "my" low-traffic tag to be able to get something closed before it accumulated answers... Commented Sep 8, 2019 at 8:07
  • 18
    Amen to that! Already seeing garbage answers to garbage questions that would have been closed 2 days ago
    – charlietfl
    Commented Sep 8, 2019 at 16:44
  • 15
    I agree, I don't think it's hyperbole to say that this is miserable going back to 5 votes. It's such a waste of effort to see five votes needed to close hideous questions. Commented Sep 9, 2019 at 13:17
  • 10
    @LittleBobbyTables extremely demotivating.
    – Script47
    Commented Sep 9, 2019 at 15:59
  • 5
    I agree but find it even more demotivating to wait for extra votes to reopen a question that's clearly been fixed. You put in the effort to help the OP, they put in the effort to bring it up to snuff, doing the right thing and there it sits for want of attention. I'd love to see the reopen process reduced maybe even down to just 2 votes to reopen. Commented Sep 10, 2019 at 14:15
  • 5
    @JaredSmith If a question has been edited into shape, please feel free to post a reopen-pls request in SOCVR. Such requests are welcome from anyone, as long as you are not "involved" in the question (effectively, that you're not the author of the question or the author of an answer on the question). People in SOCVR like to see good quality questions reopened and are quite happy to help reopen such questions.
    – Makyen Mod
    Commented Sep 11, 2019 at 22:21
63

An observation:

This experiment has exposed how negatively the curators have been regarded during this entire process.

If anyone in power wants to talk about "welcoming", let's not forget to shield the curators. Being called outside of our name or having our intelligence challenged because we elect to close questions is exposing the true unwelcoming attitude that's prevalent on the site.

I get that people just want their question answered, but the unmitigated rage when we're either closing it because it's too broad, or it's a dupe...that certainly makes me feel unwelcomed.

14
  • 10
    I've noted several complaints on Meta, but nothing that seems excessive. I think some concrete examples would be helpful to drive the point home.
    – fbueckert
    Commented Aug 26, 2019 at 13:37
  • 7
    I agree w/ fbueckert, I haven't seen any change in reception of close voters since this has started. Do you have some specific examples, Makoto?
    – TylerH
    Commented Aug 26, 2019 at 15:37
  • 2
    @TylerH - all of the issues I've seen that have come up have largely been in this thread.
    – Makoto
    Commented Aug 26, 2019 at 17:28
  • 1
    @fbueckert I was once stalked at Facebook (I have no link to my FB page in my profile) by some random guy who was angry that I closed/edited his question. He was calling me names, that was not pleasant. It was long time ago however, and the experiment is not relevant here. Commented Sep 5, 2019 at 10:13
  • 41
    If you are looking for out of SO stalkers ... here's a shot of my inbox i.sstatic.net/D95Dj.png ;p
    – Bhargav Rao Mod
    Commented Sep 7, 2019 at 18:13
  • 20
    @BhargavRao: *low whistle* That's a lotta angst.
    – Makoto
    Commented Sep 7, 2019 at 18:22
  • 3
    @BhargavRao holy moly how do you even manage an e-mail account like that. That made me completely re-draw the pros and cons of being a mod here. Commented Sep 9, 2019 at 19:12
  • I am honestly confused as to what you're talking about. Are you receiving comments that needed flagging? Are users of closed questions harassing close voters in some other manner? Are you referring to something SO staff said/did?
    – jpmc26
    Commented Sep 9, 2019 at 19:16
  • 7
    What happened with Meg? Commented Sep 10, 2019 at 3:39
  • 2
    @jpmc26: If seeing the comments around Meta about the increased closures didn't really draw the point home, did you look at the picture from Bhargav's email? C'mon. Curators are getting run ragged behind them doing a volunteer ask. If the company really wants to be "welcoming", then they need to do something about those assholes too.
    – Makoto
    Commented Sep 10, 2019 at 3:48
  • 14
    @ReblochonMasque I was demanding to know why he deleted one of my questions. ;)
    – Meg Risdal StaffMod
    Commented Sep 10, 2019 at 7:07
  • @Makoto I'm not here to debate your point. I just wasn't sure exactly what problem you were addressing. Some examples or links to other answers containing them would have made it more clear. I have largely been abstaining from interacting with the site very much for the past few weeks, aside from casting a few close votes and improving some older content of mine.
    – jpmc26
    Commented Sep 10, 2019 at 7:09
  • 7
    haha, well played @MeganRisdal Commented Sep 10, 2019 at 7:23
  • @fbueckert Someone replied @JL2210 go away from the world! and deleted their question once when it was closed.
    – S.S. Anne
    Commented Oct 8, 2019 at 11:06
61

I participated in the experiment, in that I regularly reviewed 40 questions from the close vote queue, whereas in the past I might not spend that amount of time doing it. I found it more rewarding seeing the number of questions in the queue go down, and it made me feel that when I identified questions that needed closing elsewhere, such as the First Posts queue, they would get actioned rather than just timing out.

In the past it was frustrating seeing obviously bad or inappropriate questions not being dealt with when one had taken the time to flag them.

I look forward to reading some feedback, statistics and a review of the experiment.

1
  • 10
    Thanks for sharing your thoughts on how you experienced the experiment. Hope others will do the same while we work on writing up the analysis. :)
    – Meg Risdal StaffMod
    Commented Sep 7, 2019 at 20:16
58

It's great feeling that close-votes are actually useful.

It used to be that casting the second close vote on a bad question was slightly depressing. Even worse if the question was older and had couple of upvotes, despite being blatantly obvious that the question was not on-topic for the site.

The fact that so many close votes where going to age away to nothingness put a huge damper on my participation on the Close Vote queue, and I mostly used my CVs only on very new questions where I could rely on getting some help from SOCVR; and as a result didn't cast that many CVs along the day.

Which sucked, because close votes where more necessary on the queue than now, and yet one felt less inclined to participate on it because of the feeling of futility.

This experimental setting feels very empowering. I'm sure many other users like myself are casting more votes now than before since they feel they are more likely to accomplish something.

And while for users that were feeling like me the novelty effect may wear off eventually, I believe the increased effectiveness of these will make the whole thing more engaging for everybody.

Closing and reopening have much less friction, everybody is served by this.

When the experiment started, not even 4 days ago, the CV queue had almost 8.5k questions. Now, it's getting to 4k! And the reopen queue is not overflowing. It looks like this is having an immediate effect that goes well beyond the meta-hounds that saw the announcement.

Thanks for giving this a try. Hopefully by the end of the experiment the results justify making it permanent.

2
  • 10
    Yivi, great to hear your impressions of the experiment so far! Thanks for sharing.
    – Meg Risdal StaffMod
    Commented Aug 12, 2019 at 18:29
  • 9
    I concur. The 3 vote threshold seems optimal. Still requires some consensus, but actually gets bad questions closed instead of languishing with 3 or 4 close votes. Commented Aug 23, 2019 at 13:37
48

Although I've certainly noticed the effect of this change in the higher-volume tags I frequent, getting non-dupe questions put on hold more quickly, today I saw my first example of its effect in a low-volume tag.

It is my experience that questions in low-volume tags almost never get successfully closed, even when closure is a no-brainer. I assume that arises from there being few users who cast close votes in such tags, for I know a couple where fewer than five users are responsible for almost all the new answers. Today I saw a question in one of those tags that had been (appropriately) closed by three votes, and to that I say well done.

1
  • 29
    Your observation is correct in all points. I'm not looking forward to the day when this experiement ends <sigh> Commented Aug 25, 2019 at 18:45
44

A few days into the experiment and I have already noticed it is much easier to close off non-English posts, which is a good thing.

Stackoverflow requires all posts to be in English but we still get a moderate number of non-English posts every day. These should all get closed, but previously a few would hang on, having accreted < 5 close votes (ex1, ex2) which would eventually age away. With the reduction in the size of the Close Vote queue, plus the reduction in the number of votes required, my not-entirely-scientific sampling queries of words that commonly appear in non-English posts show these are all getting closed promptly. This is good for the site, because it cleans away off-topic posts, and good for the questioner because they aren't left hanging wondering whether they're going to get help.

Thus, even if Stack Overflow reverts back to requiring 5 close votes, I'd recommend that we retain the 3-vote requirement for non-English posts. Doing so would require a new, standardized closure reason, e.g.

Off-Topic / Non-English: Your question is not in English and so is off-topic on this particular stack exchange site. You may be able to get help at one of our other language sites.

Then 3 "Off Topic / Non-English" votes could be deemed sufficient for closure.

An explicit "Non-English" closure reason would also be more helpful to the questioner, because we could point them to a more appropriate site. Currently we're closing these posts as "unclear what you're asking" which may not be true from the point of view of an author who speaks the language fluently.

Finally, there are services that provide APIs (at some cost?) that attempt to determine the language of a given string or page. Once the experiment ends, it would be interesting during the postmorten to attempt to determine what percent of new foreign language posts were closed during the experiment vs before the experiment started, using one of these services.

4
  • 4
    This is a good suggestion as it is based on close reason for the question to be closed. This might save good questions from being closed
    – Nagaraju
    Commented Aug 14, 2019 at 11:55
  • 10
    That's interesting. Has SO ever investigated using heuristics to identify non-English text before the person even posts their question? Instant feedback would be even friendlier to the asker than a close and/or migration.
    – Joe White
    Commented Aug 29, 2019 at 22:33
  • @JoeWhite: My first reaction was, wouldn't that flag code? But then I realized your idea has two advantages.
    – MSalters
    Commented Sep 10, 2019 at 15:09
  • These non-English posts could be automatically detected using language identification software. Commented Oct 26, 2019 at 15:26
43

This change will increase the number of questions closing. Can you guys share the relevant data to meta too? After you have taken your judgment? I would like to know about how this change altered the closing relevancy.

We surely won't let you down. Everyone is armed with their shovels.


Close votes were initially five, that number made many (including me), feel a little lazy towards closing questions. The change brought many lights towards the issues and now

  1. More and more people are investing time in the queue as they know that their close vote can make a huge difference
  2. Bad questions are getting closed more often because of the first point and were getting closed quickly because of the change in the number from 5 to 3.

This change brought with itself, so many qualities that StackOverflow needs.

6
  • 2
    I personally noticed that questions were being closed even more agressively than usually - which is to be expected. And that includes good questions that 3 people just did not understand well enough
    – lucidbrot
    Commented Aug 23, 2019 at 19:36
  • 4
    @lucidbrot how do you know that they didn't understand the question, and not that the question wasn't clear in the first place? And no, just because you understand it, doesn't mean it's clear. A unclear question can be understood different ways.
    – Braiam
    Commented Sep 10, 2019 at 17:09
  • @Braiam If I can think of only one way this question can be answered, I am as sure as I can be. One example is where the close voters asked in the comments for code examples, but imo that would have only cluttered the question and added nothing of value
    – lucidbrot
    Commented Sep 10, 2019 at 17:37
  • 2
    @lucidbrot that's not enough. Questions very rarely have only one possible answer, the fact that only that one occurs to you, doesn't mean others don't exist.
    – Braiam
    Commented Sep 10, 2019 at 17:40
  • 1
    @Braiam Yes. But I don't like your argument because from it follows that nobody can tell if a question is worth closing and we perform statistics (close votes) on unqualified decisions.
    – lucidbrot
    Commented Sep 10, 2019 at 20:02
  • 1
    @lucidbrot there are ways to objectively perform decisions, they are stated in the ask/don't ask pages of the help center. If you combine that with the "how to answer" you will notice that the guidance prefers that the users close questions instead of exposing themselves to have their contributions scrapped, because they were misled by an unclear question.
    – Braiam
    Commented Sep 11, 2019 at 9:06
39

I'd like to share another (very) subjective view as someone who goes through new questions and who doesn't spend that much time doing review.

On a good day, there are enough people under the tag to close-vote questions right after they were asked, and that has a few positive side effects:

  1. There are less bad answers (cause one can't answer closed questions), and thus I have to spend less time pointing answerers to mandatory metas, downvoting, flagging, etc.

  2. There are less comments, usually just one or two by the ones that close-voted pointing to some guides for [mre]s, pointing out the typo etc. There are less guesses into random directions, a lá "have you tried xy?", "Do this xy", "This valid code is wrong".

  3. There are less open questions over all, and one can find good questions to answer more easily.

  4. I usually spend a fixed amount of time here (i.e. when I'm on the bus/train). If I have to spend less time moderating, I can spend more time writing good answers.

I think good days are good for everyone:

  1. Askers get a clear signal what to do: If their question is not answerable, they can do their research, add their own tries, fix typos, and can come back with good questions, resulting in good answers in the end. On a bad day, they receive bad answers and won't benefit from them in the long run.

  2. Answerers aren't wasting their time with not yet clear questions, homework dumps and others. Even opening, reading and downvoting takes time that could be spend researching good answers.

On a bad day, I'm sometimes getting grumpy cause of all the bad questions around that need to be cleaned up.

In the past, there have been a lot of these good days, but there were also quite a few bad days when there were just one or two users around that voted actively. Since the close vote threshold was lowered there were just good days. That might be a coincidence. Or it might not.

4
  • 26
    i confirm that there are more "good days" with this change. I work in low-volume tags. Often, nothing gets put on hold; bad questions get poor answers from low-rep users... With the change, things acuatlly get closed in less than 24 hours! Commented Aug 15, 2019 at 15:46
  • On the other hand, some things get closed too quickly and still don't get reopened after a sufficient edit (example).
    – Bergi
    Commented Aug 22, 2019 at 21:31
  • @bergi You have my vote there :) And yes, the reopen queue has to be fixed. Commented Aug 23, 2019 at 2:41
  • @bergi it went through the reopen review queue ... and wasn't reopened ... But it's open now ... Commented Aug 23, 2019 at 15:05
31

I was having a look at what people were saying at r/programming and I'd summarise it as people are happy with bad questions being closed, but hate it when their questions as closed as duplicates of questions which are similar, but they don't think provide an answer. It's also much harder to get a closed question reopened than they'd like. I can definitely sympathise.

In theory it would be possible to have different numbers of votes required for different kinds of close/reopen votes, but I can see that being incredibly difficult to communicate clearly about.

Just to throw an idea out there, what about if the reopen queue could have reasons, such as I-don't-think-this-is-really-a-dupe, etc? Would take a substantial amount of time for the SO dev team though, for what is arguably a social problem. And people can already comment saying that now.

5
  • 23
    GK Chesterton once described Christianity as people not having tried and found it wanting, but found difficult and left untried. I think that's the problem people colloquially have with closure and reopening. They've learned that SO is a mean, cynical place, so if your question gets closed, you should shake your fist, rant elsewhere on the Internet and give up. Yet, if you engage with people who closed or commented, you can often figure out what you can do to get it reopened. I've closed thousands of questions, but rarely ever been engaged to improve or reopen them. That's the pity here.
    – Machavity Mod
    Commented Aug 21, 2019 at 1:12
  • 10
    Thanks for digging this up & summarizing, curiousdannii. That thread hints at some of what Meg's blog post from earlier today is trying to get at: for way too many years, closing has been the bugaboo for folks when it comes to SO - it doesn't have to be that way.
    – Shog9
    Commented Aug 21, 2019 at 1:43
  • 6
    "It's also much harder to get a closed question reopened than they'd like" Is this about SO in general or specifically about SO currently? Because currently it's 40% easier to get a question reopened than it was before August 8th.
    – TylerH
    Commented Aug 21, 2019 at 18:32
  • 4
    Sometimes close-as-duplicate of a similar question is used as an alternative to "too broad". Reopening usually requires not only that it's not a close enough duplicate, but also that it's an actual good question on its own. (Some people might have their internal standards set too high on that; maybe reopen and downvote if it's on topic but not useful.) Also what's obviously a duplicate to an experienced programmer is not always obvious to a beginner. It's important to explain why it's a duplicate (in a comment) if it's not totally obvious, but some of the complaining might be unwarranted. Commented Aug 26, 2019 at 1:11
  • 2
    @Shog9 "closing has been the bugaboo for folks when it comes to SO - it doesn't have to be that way." Fundamental, the aversion to closing comes from wanting to be handed an answer irrespective of effort spent and narrowness of question and having not gone through all the common issues or similar material. I am certain changes in presentation will not affect those attitudes significantly. Those users will not feel better about getting their questions closed.
    – jpmc26
    Commented Sep 10, 2019 at 10:55
29

This will be super helpful with languages/technologies that are not so main-stream as the number of views they get are very few. In these forums, closing an obvious question would take at-least a day or two.

With this feature closing off-topic questions are pretty straight-forward and there wouldn't be any need to go on a quest for votes in the SOCVR! Few examples would be questions on typescript, angular, node.js etc..

2
  • 4
    Yup, I can confirm we've had significantly more than usual questions closed sooner in assembly and/or x86. Commented Aug 20, 2019 at 7:07
  • 1
    Also, much more old off-topic questions with non main-stream tags were closed Commented Sep 5, 2019 at 10:16
28

Fine, I'll play along.

This should produce some interesting results given the volume of traffic on the site. My hope is that this becomes a more permanent fixture for content curation, and will allow questions which need to be closed to have a half-life on the order of tens of minutes.

24

This experiment has made an old design flaw surface far more frequently.

Whenever a post is closed, the close reason picked is the one with most votes. In case several close reasons get the same amount of votes, which one that is picked seems to be chosen randomly.

This means that posts are far more prone to get closed with the wrong close reason now.

Another problem is that everyone who close-voted get their accounts added to the "signature" of the close vote. So if I close vote as "too broad" and someone else casts a close vote with an incorrect reason, the site makes it look as if I'm behind the incorrect vote.

For example, I just closed a post where the other 2 voters didn't seem to understand the question simply because it was a technical niche question. One voted for off-topic, which was plain wrong. Another voted for unclear. While it should have been closed as too broad, as it was on-topic but lacked details.

Suggested fix:

  • In order for a question to be closed, it should get 3 votes with the same close reason. The site should leave the question open and keep counting votes until there is a consensus about the close reason by 3 voters.
  • When a question is closed, all other close votes except the 3 that closed it should be removed.
  • In case of dupe hammer, the close reason should always be "duplicate". All other close votes should be removed.

Related, it would be very nice if there was a close vote option "the question is fine and should not be closed". Currently we can only cast such votes through the cumbersome close-vote review queue, which is another flaw.

(I'm all for giving more close privileges to gold badgers, but that's a discussion for another post)

11
  • 2
  • 17
    I do not agree with the Related part at all. Commenting against the closure should be enough. Neither do I agree with the close vote reason needing a consensus. 3 close votes is enough. At first it can be that there is no [mcve] and it gets 2 close votes with that reason - and when it is provided, it becomes evident that there is [mcve] but the entire problem arises from a tpyo at which point closing as brainf4rt is the correct close reason. But you suggest we should leave these worst of the lot to wait for 5 close votes?! Commented Aug 21, 2019 at 10:36
  • 1
    The chosen reason displayed during an equal dispute between close reasons is the oldest/first chosen reason; it's not random.
    – TylerH
    Commented Aug 21, 2019 at 18:30
  • 1
    Re: your Related idea, that doesn't make sense to me. You can already pass judgment on a question you think is fine by... not close-voting it.
    – TylerH
    Commented Aug 21, 2019 at 18:31
  • 2
    @TylerH If if 3 people think a question should be closed and 3000 people think that it shouldn't, then it will get closed needlessly and has to go through the re-open procedure. This creates needless frustration for the OP and pointless busy-work for those who have to re-open it.
    – Lundin
    Commented Aug 22, 2019 at 7:04
  • 1
    @Lundin I don't think it's needless frustration; rather it's a system intended to let people voice their opinions. The system allows for differences of opinions and lets the majority win. If 3 people think a question should be closed, they get to say so... once. I'll never be worried about a question staying closed where 3,000 people think it should be open but only 3 think it should be closed. And anyway, if it's pointless busy-work for those who have to reopen it, then it would be pointless busy-work for those who are closing it, too... by which logic we shouldn't be able to close any Qs.
    – TylerH
    Commented Aug 22, 2019 at 13:35
  • 4
    Choosing between unclear and too broad can be difficult: if it were clearer, it would be narrower.
    – Raedwald
    Commented Aug 22, 2019 at 13:36
  • @TylerH The point is that closed questions get brushed aside and eventually disappearing from the domain tag, waiting re-open review by generic users who don't necessarily have domain knowledge. If you have done re-open reviews, then you know they are tedious and time-consuming. As opposed to casting a close vote on a question in the tags you follow and have domain expertise in, which doesn't take much effort.
    – Lundin
    Commented Aug 22, 2019 at 14:12
  • 2
    @TylerH As for the current system, it does not let the majority win in the short term. If we have 3000 users who think the question is fine, 1 user who thinks a question is too broad, 1 user who thinks the question is a dupe and 1 user who think it is off-topic because they like ponies, the question may get closed with the reason "I like ponies", something that 3002 users didn't agree with. After which the OP gets frustrated and 5 (3?) re-open reviewers have to chew through the post.
    – Lundin
    Commented Aug 22, 2019 at 14:14
  • 2
    Design flaw: the set of close reasons are esoteric.
    – Travis J
    Commented Aug 23, 2019 at 20:09
  • 2
    Is your answer mostly a duplicate of this one from the 12th? Commented Aug 29, 2019 at 16:06
21

Excellent idea.

It really helps a lot, especially during the weekend to get the really bad "do my homework" segment put on hold quickly.

The one observation I made: I am now surprised to often find only 1 or 2 downvotes. So the ideal workflow of

  • 3 people doing the votes ... to get the question put on hold
  • 3 more people doing the delete vote, to dispose such rejected questions quickly, too

isn't always possible.

And just for the record: there is no need for more than 3 downvotes. But when close-voting an obvious homework dump, that downvote is essential, too. But to be really clear here: I am not advocating for more than 3 downvotes. There is no point into "shaming" a new user into -5 or -10 on his first question(s).

But in my experience, for 10 bad questions, only 1, 2 OPs will ever come back and try to improve their input, and they almost never delete bad questions themselves. Thus (my) main focus: disposing of content that shouldn't stay around.

And note: 3 seems about right, too. I am fine "dup closing" on my own (and also willing to undo quickly when I got that wrong), but I am also very grateful that "owning" a "too broad" vote lies on more than my shoulders.

5
  • "And just for the record: there is no need for more than 3 downvotes. ... There is no point into 'shaming' a new user into -5 or -10 on his first question(s)." If enough people are still seeing the question to downvote on it, then yes, there is. Downvoting is not a shaming mechanism; it's a quality control mechanism. Having 5 downvotes is no more shameful than having 2.
    – jpmc26
    Commented Sep 13, 2019 at 17:07
  • 4
    @jpmc26 Opinion, not fact. Imho: for a newbie, every downvote feels like a slap in their face. You notice if there are 3 slaps, or 5, or 10. I agree, there are for sure situations where I don't hesitate to add downvote number 6 or 10. But then: far too often, those are really first time question askers. It is perfectly fair to send them a message, but that message shouldnt be so devastating that they never come back. It should make them think, and do better the next time. Not turn around, walk away and complain about our toxic, hostile community.
    – GhostCat
    Commented Sep 13, 2019 at 17:17
  • We shouldn't treat new question askers any differently than experienced ones. A person's experience has no impact on how the site works. As for whether they come back, there are absolutely people we don't want to come back. Presuming that every contributor is contributing positively or will eventually do so given enough attempts is wrong. And I suspect (rather strongly) that those that would eventually contribute positively correlates to those who would not take downvotes or any other kind of criticism personally and that those that blame us for being "toxic" are the least likely to improve.
    – jpmc26
    Commented Sep 13, 2019 at 19:09
  • In other words, you cannot blame that response solely on the quantity of downvotes. It stems largely from the person's own view of themselves and the world around them. Protecting them from negative feelings associated with being told they messed up badly does not solve any particular problems, except maybe increasing revenue through advertising and other things dependent on sheer quantity of people, and interferes with our ability to maintain quality.
    – jpmc26
    Commented Sep 13, 2019 at 19:14
  • 1
    One last thing: If you'll recall, getting a downvote increases the likelihood of the person returning over being ignored: meta.stackoverflow.com/questions/386178/…. So it's not so obvious to me that your theory that a large number of downvotes chases them away is valid from the get go.
    – jpmc26
    Commented Sep 13, 2019 at 19:17
17

Now that the experiment is done, will you also be reviewing close-vote volume post-experiment?

I would be very interested to see the number of close votes per day before, during and after the experiment.

Maybe nothing has actually changed, but I really feel like voting-to-close overall has nosedived since the experiment finished. Questions seem to stall with only one or two close votes.

1
  • 2
    which seems very much like the before situation. Can't quite achieve critical mass despite people downvoting and commenting the vtcs don't get there...
    – QHarr
    Commented Sep 11, 2019 at 3:39
14

I think the experiment is great, and I'm curious about the outcome.

Looking ahead of that, and maybe a goal for another experiment, is to test the effects of a different weight depending on the type of vote. "Unclear" or "Broad" are more subjective, and more often cast in "error" (there is a thin line between just a downvote and closing as unclear).

Reasons like "Duplicate" or the various "Off topics" often are much more clear and objective.

So I can image that you could already close a question as duplicate after 2 or 3 votes, while for 'unclear' you would want at least 5 to prevent okay questions to be closed by rushers by who tend to select 'unclear' instead of only a downvote.

I think reopening should require less votes than closing anyway, because I think it's way harder to draw any attention to a closed question after you've improved it. If you can get 2 others to vote for that, you probably really made big improvements.

2
  • 7
    I support having a different threshold for "duplicate" compared to more subjective reasons. Investing lots of time in writing a good question and having it closed because people just don't like the question (downvotes are more appropriate than close votes, but close votes are common instead) is a quite unwelcoming experience.
    – WBT
    Commented Aug 9, 2019 at 15:25
  • 3
    If people genuinely interpret closure as a dupe as "people just don't like the question" then that's an altogether different problem. If the dupe target(s) in fact doesn't answer the question then that's yet again a different problem, albeit one that I think we handle more or less ok through reopening. But when closing as a dupe works correctly -- as often it does -- the question is thereby answered by reference to existing content. If that's a problem then we've come around to "SO is not a discussion forum", which is key to why it's any good in the first place. Commented Aug 25, 2019 at 18:54
11

Is the amount of "leave open votes" required to kick something out of the review queue also altered?

Or is that still 5?

12
  • 23
    It's three (it was already three though)
    – Shog9
    Commented Aug 9, 2019 at 14:33
  • Thx @Shog9, makes sense ofcourse seeing 3 is the majority out of 5.
    – Luuklag
    Commented Aug 9, 2019 at 14:35
  • 2
    Luuklag 3 is majority of 5 indeed, but 2 is a majority of 3, so it probably would make better sense to decrease Leave Open to 2 while experiment is running. cc @Shog9
    – gnat
    Commented Aug 17, 2019 at 12:46
  • @gnat, Im not sure about that. It takes either 3 close votes or 3 leave open votes to get something out of review, so nothing changed?
    – Luuklag
    Commented Aug 17, 2019 at 18:04
  • 1
    I also thought nothing changed at first but after more closely reading recent comments under the question figured that balance has tilted indeed. Look, in the past some questions were getting to 3rd L/O after 3rd or 4th vote close, were kicked off the queue and left to organic handling. Now these questions miss their chance because 3rd CV closes them. It's maybe not critical but still, we probably are getting more "slippery" closures because of that. I think lowering L/O limit to 2 would help a bit here
    – gnat
    Commented Aug 17, 2019 at 18:23
  • 6
    Not planning to change anything else for the duration of the test, but will consider this if we reduce the threshold after the test. FWIW: tasks that got 2 LO reviews already had a pretty poor success rate before the test, so... Might've been a good idea to reduce the LO threshold anyway.
    – Shog9
    Commented Aug 17, 2019 at 23:09
  • would be funny if decreasing Leave Open limit will turn out the only long-term change after experiment is over (if memory serves we had a plot twist like that in the de-duplication project a while ago)
    – gnat
    Commented Aug 18, 2019 at 6:46
  • @gnat that would simply be reduce max reviews to 3, instead of 5. Which woukd make perfect sense IMO, but perhaps not change too many variables during this experiment at once.
    – Luuklag
    Commented Aug 18, 2019 at 9:21
  • 1
    So, there's a reason I'm not looking at LO effectiveness (or review at all) as a criteria for this test, @gnat: there are a TON of variables, and the easiest ones to modify are often not the most effective. For example: the most effective close votes have long been those that were effective independent of review: questions that were getting a lot of attention already. This sort of thing confounds review-centric analysis unless controlled for; they may not make it into review at all, they probably don't make it through, and even if they do a LO consensus does precisely nothing.
    – Shog9
    Commented Aug 18, 2019 at 20:22
  • @Shog9 I think I understand. In this experiment you're studying giving more impact to organic votes (as opposed to improving review), this definitely looks worth exploring. However this approach has sort of a gap on questions that have organic visibility limited by a system - ie score under -4 and triaged - I have a suspicion that these can be timely handled only via review (of course if you're OK with keeping askers oblivious you can let 'em be slowly collected by 30- and 365-days roomba)
    – gnat
    Commented Aug 19, 2019 at 8:40
  • 7
    Sure, @gnat. This is where it's helpful to remember that many of these systems only exist because of the limitations of close voting. The close and reopen queues, the Triage queue, reopen queue auto-triggers... would all be unnecessary if a single voter could have confidence that their vote would do something - but they couldn't; already by 2010 it was obvious that, particularly in smaller tags, close voting was just useless as there weren't enough voters. Reopening happened only for popular nuisances. The queues exist because we utterly broke the existing system by adding voting.
    – Shog9
    Commented Aug 19, 2019 at 15:16
  • @Shog9 lets see if I read you correctly? Would removing the voting from the close system unbreak things?
    – Braiam
    Commented Aug 25, 2019 at 22:26
8

I'm sure this is an off-topic response, but are you going to give this a trial run on Software Recommendations? SoftwareRecs is constantly under floods of off-topic questions, which has since been believed to be a consequence of the Ask Question Wizard on SO.

4
-10

we're looking to improve the efficacy of individual actions: if you, a member of Stack Overflow, vote or flag a question for closure, vote to reopen a question, or edit a closed question... How likely is that action to have any tangible effect on the outcome?

How exactly this is going to be measured? Is it % of the first vote/flag to close that end up in a closed question? How are you going to account for the ones whose first vote was cast before the experiment? How is this going to be analyzed in the case of the first edit for reopen?

It seems that you have an objective, but no plan to measure how much of that objective was achieved. This has the potential for introducing bias, since it's likely that you will measure things in a way that would confirm your previous beliefs. In other words, what questions would you ask the data?

2
  • 21
    I wrote the queries to measure the results before I wrote this announcement. But... I'm going to be a bit cagey on the precise details, because Goodhart's law probably applies here.
    – Shog9
    Commented Aug 10, 2019 at 15:05
  • 2
    @Shog9 could we suggest queries to apply, too?
    – Braiam
    Commented Aug 12, 2019 at 15:29
-18

How about add weights to close votes based on the reputation of the voter for that particular tag (e.g. gold badge = 3 points, silver = 2, all others = 1), and require a score of 10 9 to close. This will prevent users with not enough knowledge about the topic from closing the question prematurely and still 3 votes from tag experts to close it.

8
  • 14
    I guess this would work for large tags. For smaller tags, it is not uncommon that there are non (or only very few) gold badge holder. But especially in these tags, closing questions is hardest.
    – BDL
    Commented Aug 23, 2019 at 14:55
  • There is almost always one large tag for each question. Commented Aug 23, 2019 at 16:12
  • 2
    Tell that to this question
    – Braiam
    Commented Aug 23, 2019 at 19:11
  • @Braiam almost always Commented Aug 23, 2019 at 20:26
  • 12
    I agree with this until you said "score of 9." Why not stick to a score of 3? Or better, make the values {other: 1, silver: 2, gold: 4} and a score of 4? Gold's have the hammer as always, but it takes 2 silvers or 4 others to close. I kinda like it--it'd encourage you to review tags you're familiar with as you'll have more effect there.
    – scohe001
    Commented Aug 23, 2019 at 21:48
  • @scohe001 I disagree. Closing as a duplicate is less opinion based than closing for other reasons. The idea of having at least 3 or more people to agree on closing the question is correct. Commented Aug 23, 2019 at 23:11
  • The basic idea of giving more weight to users with more rep / badges in the tags is probably good (like for dup-hammer close votes). But it would need to handle corner cases for smaller tags where few people have high rep/badges. e.g. I'm the only SO user with a cpu-architecture badge, and we do get some questions about computer-architecture homework (like cache tag bits). You could argue that those aren't even programming questions, but that's a separate issue. The threshold doesn't need to be anywhere near 9, and could maybe be lower for questions with no "popular" tags. Commented Aug 26, 2019 at 1:19
  • @Braiam That question should clearly have python. So after my edit, it now has a major tag.
    – jpmc26
    Commented Sep 13, 2019 at 19:26
-24

Maybe it's too late already, but it seems sensible to give a little lead time on the announcement. Make this go into effect on Monday, to give a better chance that voters know the rules have changed before they vote. We know that some people vote, let's say, "speculatively", whether they're right to do so or not: "I kinda think this should probably be closed, but I want other voters to confirm". Another case: some folks like to look for posts where there are already 4 votes.

A sudden unknown change in the voting rules seems like it has the potential to introduce unwanted noise; eliminating that confusion would make for a better experiment, wouldn't it?

7
  • It's too late, things are getting closed with 3 votes. But it's only an experiment, and transients are inevitable either way. Commented Aug 8, 2019 at 19:41
  • 42
    This is more thought than people should be putting into their votes. If a question needs to be closed, then vote to close it, regardless of your vote mojo. This has been a diamond moderator principle for ages: don't change the way you vote just because you have a diamond.
    – Robert Harvey Mod
    Commented Aug 8, 2019 at 19:42
  • 6
    We both know that, @RobertHarvey, but the reality on the ground is that people do these things. And to get useful information, an experiment should reflect reality, not ideality.
    – jscs
    Commented Aug 8, 2019 at 19:43
  • 25
    So, here's the thing: closing is, by design, a very transparent system. Which means it's impossible to test something like this without folks knowing about it - otherwise we'd do an a/b test or something, and get better results in a lot less time. OTOH, a big chunk of folks are only going to learn about this by seeing it in action - they're not gonna read meta. Announcing it this way means everyone gets the rules at the same time - whatever skew is introduced will hopefully settle down well before the 30 days are up.
    – Shog9
    Commented Aug 8, 2019 at 19:46
  • 6
    I disagree. I cast a third close vote on a question and saw it close immediately, and my first thought was Something changed today. I should check Meta to see what it was and why.
    – Ken White
    Commented Aug 8, 2019 at 23:53
  • 18
    Something strange is afoot, old chum. Quick! To the Metamobile!
    – Machavity Mod
    Commented Aug 8, 2019 at 23:57
  • 2
    I agree that this might show some artifacts in the first days, but also there will be a (it's a new thing, let's try it out) bias towards higher activity at the start. The analysis will probably focus on the last days of the trial period. Commented Aug 9, 2019 at 13:06
-27

With this feature we have now the same required number of votes to close and also delete a question. Why not also reconsidering the threshold of delete votes? 2 instead of 3 for example.

If we are aiming to make the ability of closing question more effective, we should also do the same for deleting very bad questions. We all agree that we have a lot each day.

12
  • 37
    It's an experiment. We shouldn't be changing too many variables at the same time. And If you delete questions faster and do not make undeleting easier; you make reopening questions much, much harder.
    – yivi
    Commented Aug 9, 2019 at 10:24
  • 1
    @yivi if we change the delete we also do the same for undelete (like closing/reopening) Commented Aug 9, 2019 at 10:28
  • 23
    Deleted questions have much less visibility than closed questions. There are no queues for those questions. Even if the asker fixes their question having it first undeleted and then reopened is very, very hard. And again, changing many things at the same time for one experiment is not a great idea.
    – yivi
    Commented Aug 9, 2019 at 10:29
  • 3
    @yivi as a side note, I never expect a deleted question to get undeleted (only in very rare cases). I vote to delete a question when we really don't need it and cannot be improved like Typo question, clearly opinion based question, an nth very very poor duplicate, etc Commented Aug 9, 2019 at 10:35
  • 1
    @TemaniAfif The issue is still what yivi has said -- deleted questions have much less visibility, and you still also need 10,000 reputation to cast such votes, not to mention you get a much more limited number of delete votes even at 10k (5 for starters, +1 for every 1k after that up to 100k rep or 100 votes, whichever it is). Instead, I think it'd have been better to leave reopen votes at 5, or at most, reduce the amount needed to 4 instead of 3. Since OP almost always has the ability to cast a reopen vote on their own question, that means you really only need to get 2 folks to agree w/ you.
    – TylerH
    Commented Aug 9, 2019 at 18:52
  • 1
    Which is 1 less than the 3 other folks needed to consider the question close-worthy atm.
    – TylerH
    Commented Aug 9, 2019 at 18:52
  • 2
    @TylerH Just FYI: It's a max of 30 delete-votes at 35k rep (see 10k privileges description).
    – Makyen Mod
    Commented Aug 9, 2019 at 19:56
  • 2
    It doesn't always take 3 delete-votes to delete a post. It currently takes somewhere between 3 and 10 delete votes to delete a post, depending on the post's score, etc. Are you just proposing that range is shifted down to 2 to 9 delete-votes?
    – Makyen Mod
    Commented Aug 9, 2019 at 19:58
  • @Makyen I don't have a particular range in mind. I am simply suggesting to re-think this threshold too and make it lower following the same logic of the close votes. Commented Aug 9, 2019 at 20:48
  • 3
    So, this is an interesting idea... But it may actually make more sense to go the opposite direction. The threshold for delete votes starts at three, but scales according to the popularity of the question - so a popular question is harder to delete than an unpopular one. This ensures that questions lots of folks like require lots of folks to delete, while questions nobody likes require only a few. Seems like closing could work the same way...
    – Shog9
    Commented Aug 9, 2019 at 23:22
  • I have been asking for this behind the scenes too - maybe if roomba is tightened up that will help geet poor content off the site
    – user3956566
    Commented Aug 10, 2019 at 2:57
  • If you're going to make it even easier to delete stuff, perhaps consider a time limit between closing and deletion. Pile-on close votes are a real problem, and rapidly turn into delete votes.
    – Brad
    Commented Aug 24, 2019 at 20:46
-29

While you're testing things, please test having 5 votes to close, but as few as 3 to re-open the question.

New questions sit on the home page for a bit, and tend to attract pile-on close votes. When you're trying to get a question re-opened, it doesn't have this sort of exposure and therefore seemingly almost never happens.

7
  • 1
    What would the success criteria be for such a test?
    – Shog9
    Commented Aug 24, 2019 at 20:46
  • 2
    @Shog9 Whatever the criteria is for this test in the first place. However they plan to measure "efficacy" of the action. If we're testing things, it makes sense to try varying levels of votes. What I'm proposing here is something that might fix the balance between the difficulty in closing vs. opening, at the same time.
    – Brad
    Commented Aug 24, 2019 at 20:50
  • 6
    Ok. So, efficacy of reopen votes isn't directly affected by close votes (or vice-versa): only one type can be cast at a time. So, the current test should tell us whether 3 is sufficient to improve the efficacy of reopen votes as well as if 3 can do so for close votes.
    – Shog9
    Commented Aug 24, 2019 at 21:09
  • 1
    Adding to what @Shog9 said, it's very unlikely that 4 users or more don't see a reopen nomination, unlike in the close queue where something could enter and no one look at it.
    – Braiam
    Commented Aug 24, 2019 at 21:54
  • 2
    It's less unlikely than you might think, @braiam - though for different reasons. Askers delete their questions with some frequency - often before a review can complete. Also... Lots of people may see a question, but few may know the topic well enough to evaluate it; I've seen questions hang in review for weeks while everyone skips past them. There is certainly value in a lower threshold for reopen as well.
    – Shog9
    Commented Aug 24, 2019 at 22:16
  • 7
    "Lots of people may see a question, but few may know the topic well enough to evaluate it" This is actually my beef with questions being so easy to close. I see a lot of questions in specific knowledge domains where the question makes perfect sense and is well-formed and clear for someone remotely familiar with the topic. That doesn't stop a bunch of people for coming by and saying, "oh there's no code so I'm going to drive-by downvote and vote to close." Even in cases where specific code isn't relevant or required for a specific question.
    – Brad
    Commented Aug 24, 2019 at 22:49
  • 2
    Moderator Note: the comments devolved into personal attacks. Please don’t do that.
    – George Stocker Mod
    Commented Sep 9, 2019 at 19:56
-50

I just put here some food for thought, if someone is able to get something from it, great! Thanks for the attention, rich comments and votes.

More incentives to closing questions sounds like a bad idea in the long term to me. What makes the content growing in this site is primarily asking, and current (not future) users are becoming disencouraged to do that - no questions = no answers. Or even worse, only newcomers are encouraged, since they still think it is worth to ask their first question. Unfortunately, newcomers often ask bad questions. So, the whole system can be working like a filter towards a general decrease in the quality of questions, assuming there will be a permanent influx of new users.

Actually, the most obvious look-at-the-manual questions seem to be the ones that best fit the informal (or even formal) requirements of the culture (or rules) here. Luckily the answerer tries to teach something, but could be just shortcuts to information:

question -> manual

Once someone gets their own question closed, this act might be perceived as an 'aggression' - specially if the question is valuable / well worded / according to the rules in some sense. Being closed is not an offense to many people who realized they should have low expectations here. This is valid even for that bad OP that has almost nothing to loose just throwing away questions with no research effort. But there are elaborated questions that have some weight against (sometimes personal interpretation of) rules.

As an aside note, I personally think the best content is the rich one, not the obvious one. The great questions here are from the time SO allowed open ended questions. I may be missing something, but, in the extreme case, if two or more people consent to really learn something in depth... how does this affect the performance of the site? The fact they are called "not constructive" is ironic. Quora doesn't have even space for details (only comments). Reddit, ok, there is Reddit. SO could have an agreement with them to redirect good unfit questions automatically. [end of aside note]

The point here is that there are not only questions perfectly fit into the norm (say 10%) and bad questions (say 70%) - being a generous shameless guesser. There is much room in the middle, and askers are human beings. Take this into account for the content that follows.

From Freudian psychology, the concept of displacement explains how a perceived act of aggression leads to the replication of such behavior onto other OPs. I am just speculating a possible component in a complex web of social relations, but this could partly explain the current overall toxicity some people feel increasing in this site. None of the 10-20 people I worked personally with has the opinion that SO is a friendly place (to put in mild terms).

Other communities are not perceived this way. Anyone who uses other places like Tex, Math, Physics, anything besides SO (and maybe programming) realizes that.

I felt by myself this phenomenon playing Counter Strike some time ago. I always ignored pools about kicking players. Until the day I was kicked for no reason. It was really unpleasant having my play interrupted arbitrarily. But ok, I thought that ended there. The fact is that in the second or third pool that asked me to kick somebody I felt compeled to do that. Even without checking who was the target. That was sick, animal, that was human.

This is really a subject of research, perhaps with some components similar to the Stanford experiment, when sadism was a driving force among the moderators of the fake prison.

Of course there are thousands of useless questions, but closing a possibly good question is like condemning an inocent (or at least all the work put into asking). The threshold should avoid false positives ("close a valid question") much more than false negatives ("keep open a bad question"). It is easy to spot an absolutely useless/spam/lazy question. But if you have a single hesitation about it, then it is enough to keep the question open.

Now, a little bit of rant, just in case someone is interested in getting impressions left by the site:

People sometimes seems crazy about closing questions here. This seems specific to SO, a cultural thing. Rules are there to guide us, not to oblige. You are not obliged to hit your car in a truck because the semaphore is green. You have a good enough intuition to wait for the truck driver to fix the wheel and liberate the path.

Just look at the negative votes in other pro-OPs answers here and the ones I will also get. Unfortunately, I don't have high hopes this community can improve. Like any sect, rules above intuition and reason put people away.

Given such specific behavior found in SO, and not other communities, an interesting hypothesis is that programming too much train our brains to the point we learn to have some compiler behaviors, looking for patterns, taboo words and acting motivates for whatever reason while shielded by rules. It feels good to be able to act in the world, but it is also tempting when it is at our reach the possibility of closing questions. Maybe it is an easier reward than just leting it go, or finding something to answer.

Anyway, thank you who helps removing bad content from the site.

I think different levels of "offender" questions should be considered. Spam being the worst one, then lazy askers, ... etc ..., the minor offense would be to ask an original question without an obvious answer in a software manpage.

ps. Please notice the lots of attenuating, non-generalizing words I've put in here; I intend mostly to stimulate some ideas, despite being human and susceptible to rant.

I don't know how rich (arguably) "unfit" questions can interfere on the functioning of the community, so I really hope people are mostly thinking about spam when they say, like the most voted answer:

this gives me motivation to go back in with a shovel.

46
  • 21
    "Programming too much modifies our brains to the point we become a compiler, looking for patterns, taboo words and lacking reasoning, just pursuing the easy reward of being able to somehow impact others." Sorry, but with that kind of reasoning, you can't expect us to take this answer seriously. Frankly, it's insulting.
    – Cerbrus
    Commented Sep 10, 2019 at 17:04
  • 17
    What makes the content growing in this site is primarily asking ... in the last 6 to 8 years we have had more answers then questions, so that premises seems incorrect.
    – rene
    Commented Sep 10, 2019 at 17:11
  • 12
    Stanford Prison is probably the wrong "experiment of dubious ethics" to reference here - it came out after the fact that the atrocities observed were heavily encouraged by the folks running the show... Which ruins the proclaimed results, but in effect makes it into a variation on the Milgram experiment - where it was observed that folks would perform atrocious acts even when it violated their conscience and caused them great distress, if someone told them it was for the greater good.
    – Shog9
    Commented Sep 10, 2019 at 18:26
  • 15
    THIS, then, is a valid concern: if we (the company, the system) are in effect badgering people into hurting others, then we need to stop doing that. Unfortunately, the threshold itself does nothing to further that goal - indeed, it may work against it by in effect assuaging the guilt that voters might otherwise feel via diffusion of responsibility - in areas of the site with plenty of voters, questions still get closed quickly but no one person feels responsible for your false positives. In tags with few voters, nothing gets closed.
    – Shog9
    Commented Sep 10, 2019 at 18:31
  • 5
    It was stated in an extremely reductive fashion, but I think the underlying idea may have some merit (which, sadly, viyps sorta torpedoed via framing here) @Cerbrus. Programming does require - or, at least, benefit from - a certain mindset: attention to detail, awareness of context, precision... Many other activities require the same, of course: all of us know perfectly nice people who, nevertheless, you DO NOT INTERRUPT while they are performing a delicate task... Harder to see someone thinking though. It is plausible then that task-switching on a programming forum might be problematic.
    – Shog9
    Commented Sep 10, 2019 at 18:43
  • 3
    Sure, programming requires a certain way of thinking, but this answer's last paragraphs really aren't constructive. It implies that we're not capable of anything other than blindly close-voting questions just because one detail is off.
    – Cerbrus
    Commented Sep 10, 2019 at 18:47
  • 23
    Can we not encourage the answer filled with hyperbole, poorly fitting psychological experiments and the assumption it's going to get downvoted any way? It's not constructive. It's just pointing fingers and calling us "robots"...
    – Cerbrus
    Commented Sep 10, 2019 at 18:51
  • 7
    'What makes the content growing in this site is primarily asking, and people are becoming disencouraged to do that' - [citation needed]. 'but closing a possibly good question is like condemning an inocent' - Who's closing good questions? Even so, you do realise people can retract votes and reopen questions? As for the rest of the answer, programming didn't lead us to closing questions, the site's stated purpose is what drives us to close questions.
    – Script47
    Commented Sep 10, 2019 at 19:18
  • 5
    Most of this isn't written constructively, but that doesn't make it wrong, @Cerbrus (some of it is wrong, but not because it's not constructive. And, then there's the bits that are hyperbolic, or: wrong and unconstructive). My goal when commenting in such a situation is to help folks find ways to communicate more effectively - I doubt viyps' goal here was "let's see how many people I can piss off in an afternoon"; they had something to say, and perhaps it was worth saying, but I suspect it was largely lost on most readers here due to how it was conveyed. That's fixable...
    – Shog9
    Commented Sep 10, 2019 at 23:28
  • 3
    We'll see if viyps fixes it...
    – Cerbrus
    Commented Sep 11, 2019 at 5:20
  • 13
    Highlighting the shovel thing seems disingenuous. Surely you are aware that many people have abstained from doing cleanup they felt would have been necessary and beneficial because the system was (and now, again, is) working against its stated goals, and that the experiment encouraged them to revive that effort. Regardless of how you look at it, just the size of the close vote queue should tell you that stuff which needs to be reviewed is not. If the reviews come out wrong, that's a separate problem; but discouraging volunteers from reviewing is not the solution to that.
    – tripleee
    Commented Sep 11, 2019 at 6:25
  • 19
    Each time someone says "closing good questions", why it's that it doesn't have said questions linked in the post?
    – Braiam
    Commented Sep 11, 2019 at 8:59
  • 11
    I feel like you've not been here enough winters to really understand how we feel towards some questions. You're coming at this from a fairly innocent perspective; we've been dealing with a literal deluge of bad questions for several years now. I'm not convinced we can really talk on the same level since you're not really seeing it from our perspective, but I'll read your statements and carry on.
    – Makoto
    Commented Sep 12, 2019 at 0:05
  • 4
    The "interruption" thing is a bit hard to parse I'll admit, @viyps. Picture this: you're in a machine shop, helping someone on a drill press or whatever. They're focused - they're doing precision work, and could screw it up or hurt themselves or worse if they let their attention wander - so you ask 'em a question, and they just bark at you. Later, after they've left their station, you ask 'em the same question and they give you a perfectly civil response. We've all been in some situation like that, right? The problem with programming is... It's harder to "walk away" from it.
    – Shog9
    Commented Sep 12, 2019 at 1:51
  • 8
    There were Q&A sites before SO/SE. And there other Q&As places right now. The reason this one works so well for what we do is exactly what differentiate it from these other sites, either the failed or the existing but useless: strict rules and curation. And curation basically means closing and deleting questions, in an attempt to increase the signal/noise ratio.
    – yivi
    Commented Sep 12, 2019 at 6:59
-51

The votes cast on the other answers here show what I think is a bias of this community to participate in closing questions.

In contrast, I can attest to the numerous times I have searched on Google and landed at Stack Exchange questions that were closed and had a negative total score, and found an answer I was looking for.

In my opinion there should be some sort of system where there is push-back against closure. Let's say there is a separate voting queue where inexperienced users vote on questions proposed for closure. And then, if there are:

  • at least 100 votes from inexperienced users, and
  • they are 95% in one direction

Then the experienced users that voted in the different direction lose a significant amount of reputation.

7
  • 11
    Everyone is a, "regular user". Are you referring to any specific group; curators that are trying to keep the site clean, by chance?
    – fbueckert
    Commented Aug 20, 2019 at 20:21
  • 16
    ...a separate voting queue where non-privileged users vote on questions... This doesn't make sense. Reputation is a measure of how much the system trusts you. High reputation strongly implies that you understand how the system works and the quality guidelines for the site. It does not make sense to give reopen privileges to users who don't know when a question should be closed (or at least haven't proven it).
    – BSMP
    Commented Aug 20, 2019 at 21:28
  • 9
    @BSMP It is, however, a great way for popularity to override quality control. Which is likely exactly how it is meant to be used.
    – fbueckert
    Commented Aug 20, 2019 at 21:29
  • 15
    Your suggestion also 1) changes question closure from a matter of being on-topic into a popularity contest (as fbuecker says above), 2) gives power to spam accounts and voting rings to keep harmful content on the site, and 3) punishes people for disagreeing.
    – BSMP
    Commented Aug 20, 2019 at 21:32
  • 11
    It would be awesome if we could even find hundred users that go sit in a queue, judging posts. No matter what task they have to perform.
    – rene
    Commented Aug 21, 2019 at 12:15
  • 9
    There is already a system like you describe; it is called the Close Vote Review Queue. However, it does not include a reputation penalty for good reason.
    – TylerH
    Commented Aug 21, 2019 at 18:39
  • 1
    Half the time when I google and end up at a low-scoring question with a high-scoring answer, it's basically quoting documentation that I'd rather have found directly. Commented Aug 26, 2019 at 1:25
-65

Are we sure we want to go into that direction?

Questions being closed wrongly seems to be one of the most common complaints about this site. Making it easier to close will only exacerbate this problem.

8
  • 23
    Define "wrongly". Besides, now it only requires three people to right this "wrong".
    – Makoto
    Commented Aug 12, 2019 at 16:00
  • 39
    Most of those complaints are coming from those who don't care about quality standards, I feel. If you filter out those, you're left with a lot less. So few, in fact, that I doubt it's anywhere near the most common.
    – fbueckert
    Commented Aug 12, 2019 at 16:05
  • 11
    I see a lot of bad question being closed much more quickly now
    – Alon Eitan
    Commented Aug 12, 2019 at 16:44
  • 10
    There are two competing philosophies: one holds that any process where a mistake can happen should be as difficult as possible in order to minimize mistakes; the other holds that mistakes will happen regardless and it is better to make correcting them easy. Both philosophies have merit; perhaps it is best to have a short fence and a ready ambulance
    – Shog9
    Commented Aug 12, 2019 at 18:32
  • 36
    90% of the complaints I see about questions being closed wrongly turn out to be entirely correct closures. For the exceptions, they can be more easily fixed now too. Commented Aug 12, 2019 at 20:24
  • 2
    You forgot what questions can be reopened easier as well (usually after an edit). So it's not a problem for wrongly closed question: even if 3 guys made a mistake, only 3 others are needed and mistake is corrected.
    – Sinatr
    Commented Aug 14, 2019 at 7:07
  • True, but it would be easier to reopen too Commented Aug 16, 2019 at 5:38
  • 3
    Questions aging away from the close vote review queue is also a common problem.
    – pppery
    Commented Aug 20, 2019 at 22:34
-79

Close votes are already heavily abused, and this will only make things worse.

Yesterday I asked a clear and concise question that was downvoted to -1 before being upvoted to +1 and favorited twice. It received two close votes ("too broad" and "unclear") at approximately the same time someone understood perfectly what I was asking and provided a clear and concise answer. All of this occurred within a few minutes. Significantly, it was not a question about some code not working as expected. It occurred to me that there must be a standard C# API to do [very specific thing], and it turns out that there was, because great minds think alike. The API was just relatively new, obscure, poorly documented, and mentioned in a grand total of 9 Q&As, which of course you can't search for unless you already know its name.

The vast majority of close votes received by experienced site users are like this. It's common for good questions about obscure things to receive two or three drive-by close votes, but rare for good questions to reach the threshold of five. The mean number of bogus close votes on good questions about obscure things is probably somewhere around 1, but with a relatively large standard deviation that will result in a large number of false positives if the threshold is 3. Genuinely bad questions from inexperienced site users could easily reach a threshold of ten or more.

34
  • 14
    If they do then it shouldn't be hard to have them reopened with 3 votes again. The answerer who figured out what you meant can probably help clarify your question, or you can look at the answer and try to look at what it focuses on to do the clarification on your own. Commented Aug 8, 2019 at 20:34
  • 24
    In my personal experience most questions that are unclear are understood by some people (even worse: different people understand it differently). Having an answer that can be verified to satisfy the asker is a huge lead one can use to clarify unclear questions. More often than not the lack of clarity comes from missing requirements or missing expected output, which are typically supplemented by an appropriate answer. Commented Aug 8, 2019 at 20:38
  • 38
    I obviously don't know any info about the question you're referring to but it's a perfectly reasonable thing for some people to consider a question too broad and unclear and for others to disagree. Not everyone's standards or knowledge base is the same. That's why the system includes an ability to reopen questions in the first place. An ability, by the way, which is also now at 3 votes instead 5. So there's not really a meaningful change in the scenario you describe.
    – TylerH
    Commented Aug 8, 2019 at 20:41
  • 8
    I don't think it's fruitful to debate here the merits of the specific question of TKK. Focus on the point they are making in their answer here. Commented Aug 8, 2019 at 20:47
  • 9
    @TKK Favorites can be used for lots of different reasons. They are used by people who think a question is good, and by people who think a question is bad, and because someone lost their keys. That a question is favorited doesn't inherently imply that a question is good. It might be; it might not be.
    – Makyen Mod
    Commented Aug 8, 2019 at 20:48
  • 15
    @TKK It's a fact of life that most things are only going to be understood by some people, and interesting things are going to be understood by very few. Sure...but that's not a reason to not try to clarify it so the most people can understand it. Changing the number of close votes won't make this easier for you; if anything, it'll just ensure unclear questions get closed that much faster. "Someone understood it, so it can't be unclear" is a pretty common defense that...almost never works. Editing, however, can help immensely.
    – fbueckert
    Commented Aug 8, 2019 at 20:51
  • 23
    So, you might be right. But the solution there is probably better guidance for close-voters, since we already close a non-trivial portion of questions being asked. Making the task harder across the board is kinda like using a knife without being careful of where your fingers are: a dull knife just means you're applying more force when you do hit. Right now, closing is slow, tedious, hard to undo, and (as Josh Caswell noted in his answer) too many folks take a cavalier attitude to it because they think their vote is meaningless - all of that needs to change, somehow.
    – Shog9
    Commented Aug 8, 2019 at 20:59
  • 25
    Close votes are already heavily abused, and this will only make things worse. - there's not enough reviewers and close votes on the site with 5 close votes required. Per 100 votes, there's a potential to close 13 more questions with 3 votes compared to 5. This will let us make a big dent in the close vote queue. And yeah, it will periodically be abused, but name one feature of the site related to content that can't. Questions, answers, and comments can be used for rudeness or spam, votes can be targeted, and users can be targeted, accounts can be hijacked, and formatting can be used Commented Aug 8, 2019 at 21:04
  • 16
    Well...if you don't want your question closed...trying to help as many people understand as possible sounds pretty valuable to me. Personally, that argument sounds like, "Well, they'll never understand, so there's no point in trying", in an attempt to absolve yourself of responsibility to improve your question. Considering we're trying to help more than just the asker, that sort of perspective doesn't work in trying to build a repository of knowledge.
    – fbueckert
    Commented Aug 8, 2019 at 21:06
  • 25
    @TKK you're talking about abusable votes, and you recommend single reopen votes to fix it? That defeats the entire point of close votes in the first place. That lets any 3k user go on a reopen spree - anything from trash to borderline R/A or spam. That can be abused Commented Aug 8, 2019 at 21:19
  • 5
    Do you have examples of questions (except that anecdotal one) where close votes are heavily abused? I'm an avid close voter but I somehow never come across abused posts.
    – rene
    Commented Aug 8, 2019 at 21:19
  • 8
    @TKK Which, again, doesn't do anything to help people understand. You see, "People close things way too fast and abuse it!". I see, "My question is super niche and obscure and nobody will understand it, but that's not my problem." See the difference? By helping people understand it, you get a far better chance that someone is able to help you. You help the site, you help yourself. That sounds like a win-win for everybody in my books.
    – fbueckert
    Commented Aug 8, 2019 at 21:22
  • 25
    To add a bit to @fbueckert's point: "close as unclear" doesn't include "this question is over my head". It should only include "I understand the corresponding technology and yet I have no idea what the asker is trying to ask". And any responsible close voter will vote such. I just finished 40 close vote reviews and I skipped 37 questions because I was lacking domain knowledge (even though I filtered by language). Commented Aug 8, 2019 at 21:26
  • 10
    consider giving a read to Can a question with an accepted answer be closed as unanswerable
    – gnat
    Commented Aug 8, 2019 at 21:34
  • 17
    @TKK you should have more faith in your peers, especially those with enough rep to cast close votes. You might be surprised that most of us are not dumb. Commented Aug 9, 2019 at 8:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .