38

From my point of view, with the giant close votes queue leaving so many close votes attempts to a state of "aged away", I'd like to know if my close votes actions are meaningful or meaningless.

Consider Stack Overflow users with at least one successful closing vote round (i.e.: being among the listed close voters of a question, regardless if it was reopened or not later). Those users have a ratio of close success of:

successful_close_attempts / total_close_votes

I'd also like to compare my close vote efficacy to other users. Are there stats on the average and the median values for those success rates?

Is that part of the data that you can discover at 25,000 reputation or is that queryable via SEDE?


I only know that I have casted 1,356 close votes on SO.

34
  • 5
    That is pretty meaningless, success rate is completely dominated by the tags you visit. Commented Jul 12, 2018 at 14:59
  • 10
    No, I don't look at other users' stats. I think turning this into some kind of competition is quite unhealthy. Commented Jul 12, 2018 at 15:04
  • 24
    @HansPassant I feel my votes end up wasted 80% of the time: it's not for competition at all, it's to get an idea if I'm doing a bad job or not, if the multiple minutes I spend on each review to read it, understand it and measure if the problem is reproducible or not are worth it.
    – Cœur
    Commented Jul 12, 2018 at 15:18
  • 8
    You do have enough rep to get a birds-eye view. Visit the [votes] tab in your profile, navigate at least a month back. Anything actually closed has the [closed] annotation, deleted and roomba-ed posts are purple. You can't separate by tag, so you still don't know that much. Commented Jul 12, 2018 at 15:42
  • 13
    Here is a SEDE query that shows how much posts you managed to close and where you are relative to others (not to be used to turn this into a competition). You can compare that number with what you found on your votes tab as indicated by Hans.
    – rene
    Commented Jul 12, 2018 at 15:46
  • 1
    Just for comparison: I casted 66,837 close votes in total with 14756 posts closed, based on the SEDE query.
    – rene
    Commented Jul 12, 2018 at 15:53
  • 3
    Based on nothing I would say a "success" rate of around 20% sounds OK as the pool of questions that get a close vote is high and you only have 1 vote of the required 5 votes. Only users that primarily wield their goldbadge dupe hammer could have significant higher rates.
    – rene
    Commented Jul 12, 2018 at 16:02
  • 8
    @rene The main difference is whether your someone going around finding bad questions and flagging them, or using review queues (or similar mechanisms, like CV chat) to find questions already acted on by lots of other people (hence why they're in the queue) and acting on those. The latter will have a higher close rate than the former, because they'll consistently be casting votes on posts that already have more close votes, and that are more likely to be shown to more people.
    – Servy
    Commented Jul 12, 2018 at 16:09
  • 2
    @Servy while reviewers have that advantage, I imagine users that follow their favorite tag could be as effective, assuming the tag is healthy and those with close vote privilege have a common understanding about what they want to close.
    – rene
    Commented Jul 12, 2018 at 16:19
  • 2
    @HansPassant just so I understand correctly: are you suggesting that after close voting posts should be revisited in say 6 to 8 weeks to be delete voted?
    – rene
    Commented Jul 12, 2018 at 16:24
  • 1
    @rene I think Servy's point about using review queues/CV chat matters a lot here. For example I almost exclusively vote on questions in the close queue or socvr. I have about the same as Cœur's cvs (1600), but more than an 80% close success rate. This just shows the power of more "organized" close votes.
    – River
    Commented Jul 12, 2018 at 17:59
  • 1
    @EvanCarroll: Perhaps you are unaware that only moderators can "delete dissenting viewpoints". I cannot see them but perhaps they were phrased in a rude way? Commented Jul 12, 2018 at 18:25
  • 4
    I find this an interesting question - I've occasionally wondered, myself, how effective my close votes are... To the topic of whether it makes a difference whether one closes in a tag or a queue I can definitely say that most of what I vote to close in my main tags (ms-word, word-vba) do NOT go on hold or get deleted. And that can be very frustrating. But many that I vote on in the queue do at least go on hold. Commented Jul 12, 2018 at 18:45
  • 2
    @EvanCarroll see my edit. I think this is a generally applicable question and not too localized for a single user. Whether the statistics are actually meaningful is another story entirely.
    – ryanyuyu
    Commented Jul 12, 2018 at 18:57
  • 3
    The important takeaway is that the number is meaningless because there are so many factors that come into play other than whether or not you are voting to close questions correctly. You may have incorrectly voted to close a now closed question which would boost your score, and you may have correctly voted to close a question that did not end up closed for any number of reasons which would lower your score.
    – user4639281
    Commented Jul 12, 2018 at 18:57

1 Answer 1

14

We considered a personal 'review efficacy' page for each user. It seemed appealing to have something that encouraged people to spend more time in queues that were most intuitive for them by showing them how effective they were being.

As we started considering possible designs and implementations, we realized how much of a bad idea it was turning out to be. You're essentially saying:

  • You're really great at this
  • You really stink at this
  • You're not really any different than anyone else at this

for motivational purposes, only the first one was really shiny, the other two were more brutally honest in most cases than anything else.

But, depending on the tags you visit (as others mentioned), being really great was rather subjective; and we just wouldn't have a whole lot to show folks that weren't at least somewhat active in the tags that would get you a Generalist badge at some point in the site's evolution.

You have to go with a median from the tag group, and then a number representing the efficacy of each reviewer. So, being wrong 6/10 times in Cobol would look a lot different than being right 3/10 times in PHP, compared to respective medians. So if your goals are:

  • We want graphs where people can realistically work to make them go up and to the right pretty consistently, and,
  • We only occasionally want the lack of progress to be a bit discouraging to people that just chronically get it wrong

... it just became crazy to fathom the individual math cases, much less maintain them, and then design around that.

I still think there's opportunity to discretely tell folks hey, you're really good at this particular thing, we'd love it if you did more, and even the subtle reverse of that, but when we looked at it with numbers and graphs in our hands, they turned out to be the wrong tools.

Always open to suggestions, I'd love more enticement into (and potentially out of!) the queues, but it's a tricky problem and really easy to inadvertently hurt people's feelings if you don't nail it just right.

1
  • 1
    "being wrong 6/10 times in Cobol would look a lot different than being right 3/10 times", no I'm not wrong. Remember, close votes do expire. So that if 6/10 or 3/10 of my votes resulted in the question being closed means exactly that: there where 4 other timely close votes that resulted in the question being closed.
    – Braiam
    Commented Jul 13, 2018 at 19:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .