2

Stack Overflow Close Vote Reviewers Chat Room (also known as SOCVR) is an environment where users collaborate together to monitor, refine, and clean repetitive questions from the community.

However, I want to know if users also get informed about these processes on their post or they just receive the final decisions after their post gets closed? The process could be generally be explained better.

11
  • 20
    SOCVR operates under strict rules, laid out in our FAQ and those rules are enforced by the Room Owner team and when needed with support from the Mod team. We rely on the mechanisms and features that Stack Overflow has build in. As such any post brought up for moderation in SOCVR will see the same post notices and go through the same mechanisms. The only difference is that it will see more eyes in a short period. As SOCVR also maintains a list of pro-forma comments and guide its regulars you might in some cases find more guidance offered in comments on such posts.
    – rene
    Commented Nov 30, 2020 at 13:42
  • 10
    You've linked to the graveyard, that holds handled posts. Our lively room where anyone with the chat privilege can talk is here
    – rene
    Commented Nov 30, 2020 at 13:45
  • No they don't. Frankly I think they should(for transparency). However, that's not going to change anything. It's highly unlikely that a new user or OP makes the required edits to fit the high standards held by SO. As the name "SOCVR" suggests, it is predominantly involved in "closing" and clearing up the "close queue" rather than helping OP or improving question quality of closed questions.
    – TheMaster
    Commented Nov 30, 2020 at 13:53
  • 5
    @TheMaster the name has historical significance. We did discuss in August 2016 if renaming was needed and the consensus back then was no. Do know that our tour is more subtle and doesn't only focus on closing while that is most of the traffic we see.
    – rene
    Commented Nov 30, 2020 at 14:01
  • 10
    I think we all agree that the overall experience could be better for users who try in earnest to post a useful question, but don't get it right on the first try. The role of SOCVR in that discussion is tangential at best.
    – tripleee
    Commented Nov 30, 2020 at 14:12
  • 2
    @rene The point still stands: It's much easier to close a question than reopening it. At least that was my experience with socvr. So the name "SOCVR" is apt, I believe. If there are more eyes, Then I still believe that OP should be informed that their question is under extra scrutiny.
    – TheMaster
    Commented Nov 30, 2020 at 14:13
  • 2
    @tripleee fully agreed. The closure guidance is often not very clear of focused (ha!) and would benefit from some more actionable steps or clearer explanation of what's wrong. Who or why closed the question is irrelevant for this.
    – VLAZ
    Commented Nov 30, 2020 at 14:28
  • 9
    Getting a post reopened via SOCVR is very quick if it fulfills the site guidelines. We would hope to be able to reopen more questions, but unfortunately, my impression is that very few questions are updated after they are closed, and of course only a small fraction of those get a reopen-pls in SOCVR. (And some of those are still not eligible, but even then, I would say SOCVR is probably too eager rather than too reluctant to reopen.)
    – tripleee
    Commented Nov 30, 2020 at 14:36
  • @TheMaster Okay, it is in general hard to get questions re-opened, that can't only be attributed to SOCVR. As for your own participation in SOCVR: AFAICT the re-open-pls requests you posted in SOCVR were all handled, at least the ones you posted the last 3 months.
    – rene
    Commented Nov 30, 2020 at 14:46
  • 1
    They provide a whole new meaning to closing comments, but the queue is almost 3K posts long - that number must be reduced, and it's been way way higher than that in the past.
    – Rob
    Commented Nov 30, 2020 at 14:47
  • 2
    It would be great to make more users aware of chat and encourage them to use it. This would necessarily be part of a larger investment in chat as a thing, so unfortunately I don't see Stack Overflow (the company) doing anything about it.
    – TylerH
    Commented Nov 30, 2020 at 17:24

3 Answers 3

35

I'm a Room Owner of SOCVR.

Let's say we added a canned comment to every reopenable question we close. My experience is that most users won't do anything. At best, I've seen canned comments left for things like this (NAA, etc) maybe get a 1-2% success (anecdotally). It's not a lot. I'm also a SO moderator and I delete a lot of stuff. Very few people avail themselves of Meta to question moderator actions.

The funny thing is many of the SOCVR regulars do indeed leave helpful comments so that people know what they need to fix. Once in a while, people do listen and we respond accordingly with reopen requests. We've even had people come into the room and ask us what was wrong and they get some useful feedback. But we cannot make people participate in the site or listen to counsel. For most people, they see closure as the end of the road, and that's not something SOCVR can fix.

10
  • 3
    I'm honestly not sure how much canned comments help or not. I personally don't like them. To me, they suffer as the same problem as the close reasons - they are too broad and not very specific. They usually say something like "your question is is bad. Visit these links to find out more". Very similar to the close reasons. I also recognise that's by necessity - trying to make a custom message for each question when reviewing many a day is a big undertaking.
    – VLAZ
    Commented Nov 30, 2020 at 15:33
  • 4
    FWIW I see a lot of the canned comments get flagged as well (usually declined). Their efficacy is low, but it's better than just having non-diamond actions happen without at least explaining why
    – Machavity Mod
    Commented Nov 30, 2020 at 16:07
  • 2
    As I said, I personally don't like canned comments. I really don't know whether they do more good than harm or vice versa, though. However, I do agree that a canned comment is better than no comment at all. I wouldn't really flag them as I don't think they are against any site rule - none of the flag reasons really apply. Unless the post they are under is edited until they no longer apply but that's a rare occurrence.
    – VLAZ
    Commented Nov 30, 2020 at 16:11
  • 2
    @VLAZ - I don't like canned comments for one simply reason, they are submitted as if I another user is submitting them, when in reality they are submitted when I choose a canned reason from another trusted function of the website (review queue, etc.). The fact these canned comments get flagged speaks volumes. Commented Nov 30, 2020 at 16:57
  • 2
    @Ramhound hmm, I was actually referring to the canned comments that come outside the site - the ones from userscripts. Although, I believe some people may also have a copy/paste ready. The ones from reviews I think are better, as they are a bit more focused.
    – VLAZ
    Commented Nov 30, 2020 at 17:13
  • @VLAZ - Either way I dislike the canned responses, but it's what we have, so I use them. Commented Nov 30, 2020 at 17:18
  • Hard to remember which 'auto-reply canned comment' is provided by all review reasons but if I remember that there is one that's perfect (and how to cause it to be applied) I use that - otherwise if it's not unfair to use "no comment necessary", especially if there's a prior good comment, I use that. --- On MSE so many rep 1 users have an idea to pitch that isn't researched. If, when they check back a day or three later it hasn't been well received that's often blamed on the half dozen helpful comments and the two dozen downvoters; that they're never coming back. Rarely help is correctly used.
    – Rob
    Commented Nov 30, 2020 at 18:16
  • 7
    @Ramhound The fact that canned comments sometimes get flagged really doesn't say much about canned comments. The vast majority of times when canned comments get flagged, it's by an OP who is flagging anything which isn't "your question is great" as unfriendly/unkind. Of course, there are some comments a few people use as canned which shouldn't be posted. Those are usually addressed by moderators or other users when seen. Given that non-system canned comments do tend to be seen by other people (just because the comment is used more than once), they do tend to be addressed/fixed/stopped.
    – Makyen Mod
    Commented Nov 30, 2020 at 18:31
  • Perhaps the most important thing to understand is that SOCVR is completely unaffiliated with Stack Overflow or Stack Exchange Inc., and/or its moderation team (despite the owner of the SOCVR chatroom currently being a SO moderator). Might be a good idea to add this to the SOCVR FAQ.
    – Ian Kemp
    Commented Dec 4, 2020 at 15:43
  • @IanKemp There are several owners of the SOCVR chatroom; only two of them are currently moderators (and a few other SOCVR regulars who were not room owners were also previously elected as moderators). Regardless, I think the language of the tour is adequately clear that this is users working together rather than a site or company feature within the system as designed.
    – TylerH
    Commented Dec 8, 2020 at 18:35
28

Since SOCVR is a Stack Overflow group and this question sort of focuses on their work, this may be better served on Meta Stack Overflow (which is why I eventually moved it here)... but there are similar groups around the network designed to help get questions closed more quickly, so I think talking about this more broadly is beneficial.

In short - these groups are serving a valuable purpose and we should work to make their work easier and support what they're doing with changes that will help anyone whose question is closed. I don't see a specific need for additional transparency into the work of these groups; I see need for improvement in how the system helps users overcome question closure - and even more - improving how the UI introduces users to question asking so that fewer questions need closure.

Notifying users of closure and other system improvements

A week ago, we changed our system so that all users whose question is closed are notified of the closure and the reason for it - "Question closed" notifications experiment results and graduation - this is a huge departure from our past, where we were concerned about notifying users because we do know that having a question closed can feel like a failure and can hurt - particularly if you're desperately trying to get unstuck at work, only to have your question shut down, too. Closure can be frustrating but we want to help users overcome it, rather than having it feel like a dead end.

To do this, we want to make sure that any links in the post notice that a user sees after their question being closed lead to helpful information for the improvement of their question - there's probably some room for improvement here as this guidance is generally network-wide rather than site-specific. That said, any guide still has to be somewhat general as sometimes the way to fix a question is specific to it, so there's only so much help a guide can offer.

Think about it this way - if I have a guide about how to write a good question and it has five points - any one close-worthy question may be missing 1-5 of those points but it's difficult to make the guide specific to which points an individual question is missing, which can lead to overloading the asker with stuff they're already doing and making it difficult to identify the items that need to be added. So by narrowing the scope of any help guide to a specific subset of issue, the information can be more tailored to the question asked.

Many people who participate in reviewing questions for closure do leave comments linking to helpful resources, which will give askers a heads-up that their post needs some work, but these resources are available to all reviewers, not only specific subsets of them, so I don't see a need for special treatment for SOCVR participants only. While these may be more tailored than the main Help Center article, they may still be more general rather than specific to the question asked but, with individuals leaving these comments, they can often tailor them to the specific needs of the question - though this slows down the review process.

Currently, there are only two notifications the system sends when a question is nominated for closure and they are both due to comments being created -

  1. Questions that are possibly duplicates.
  2. Questions being closed with a custom reason.
  3. (Not a notification, but with sufficient rep, they can see on the question that it has collected close votes)

Otherwise, a user only knows their question is up for closure after it's closed or if someone leaves a comment pointing out issues with the post. I have seen discussions about notifying users about closure while its in process so that they can see why their question is seen as close-worthy and work to address that during the close process, hopefully preventing their question from being closed. I'm not really convinced that it'd be effective at getting a quick change to the post... but a lot of times, that's not even necessary since...

Many reviews age out completely

The issue that I regularly come across is that many sites have more work than they can manage in review. In a 90-day period, 69 sites had 70% or fewer of the questions nominated for closure actually finish review - this means that 30% or more aged out of review without being handled. This includes Stack Overflow, which naturally tends to be around 50/50 (which is an improvement from only 35-40% handled when five votes were required to close).

Many questions do get edited after being closed but most edits don't result in the question being reopened. For example, in a ninety day period on SO prior to the close notifications being turned on, nearly 30k questions were edited after being closed but only 6% were reopened - that said, only about half of the posts were reviewed, with 44% aging out of the reopen review queue - so, while it can be hard to close close-worthy questions, it's also hard to reopen a question that's been closed.

What I get from this is that there's a ton of stuff needing to be reviewed and not getting it and that's likely because there just aren't enough people willing to review at all. There's a core of reviewers on many sites - in some cases it's mostly the moderators - but many users with the ability to review, don't. This is why we're working to improve the review experience. If people are better trained in how to review and the review queues are easier to use, they may actually review more and more accurately, leading to a lighter load shared by more people and a higher percentage actually getting reviewed.

There's a lot of room for improvement but I don't think that transparency for the small group of users trying to manage the huge flow of close-worthy stuff on the network, particularly on Stack Overflow is the place that needs it.

Thanks to all of the SOCVR members who do work to identify and close questions that aren't a good fit for SO - and to the users who do similar work on other sites. You may not hear it often but you're appreciated.

10
  • 9
    ... I spent a good chunk of time writing this and figured someone should read it so... I moved it.
    – Catija
    Commented Dec 1, 2020 at 21:19
  • "improving how the UI introduces users to question asking so that fewer questions need closure." I know regexes have a certain reputation on this site, but I make a point to request this every chance I get -- a regex that checks and warns (or blocks!) users on SO who ask a question with a title that contains "best practice[s]", or "should"/Should I" that ends in a question mark, I think would cut down on a ton of close-worthy questions being asked. Maybe start w/ a silent flag that doesn't warn users but does record to a DB table so you can see how many would be caught over 6-8 weeks...
    – TylerH
    Commented Dec 1, 2020 at 21:51
  • 1
    "this means that 30% or more aged out of review without being handled. This includes Stack Overflow, which naturally tends to be around 50/50" I'm curious if this is just the current report's results; if this is compared to numbers from ~2 months ago before Sam started reviewing 1k to 1.5k in the CV Queue a day. I would expect the rate is actually higher than 50/50 for the last 2 months.
    – TylerH
    Commented Dec 1, 2020 at 21:53
  • 5
    It's 60/40 with Sam closing stuff. The 50/50 is from before, hence the "naturally".
    – Catija
    Commented Dec 1, 2020 at 21:54
  • 2
    Ah, I read that "naturally" as "of course" rather than as "unaltered"
    – TylerH
    Commented Dec 1, 2020 at 21:59
  • 9
    IMO, what would help the reopen queue is not auto-pushing questions into the queue due to non-OP edits. Edits by users other than the OP rarely make a question on-topic and consume the question's one entry-by-edit into the reopen queue, preventing any subsequent OP edit from automatically entering the queue. In addition, they are a large portion of the review load, training reviewers that edited questions usually are "leave closed", which really isn't how we want reviewers trained. Some few non-OP edits do make the question on-topic, but most of those are from users who can reopen-vote.
    – Makyen Mod
    Commented Dec 1, 2020 at 22:32
  • 1
    Maybe it would make sense to find out why users don't want to review. Or why they pass posts and don't want to vote on them. I know why I don't, but perhaps my reasons are different than other user's reasons.
    – Scratte
    Commented Dec 1, 2020 at 23:24
  • 3
    @Scratte I personally avoid the reopen reviews partially because of what Makyen mentioned - edits made to questions after the question was closed. I particular, cosmetic changes that leave the question the same. A lot of those come from other users but OP also occasionally does this. Since any edit automatically pushes to the review queue, it's flooded with questions that have no difference. I do make it a point to check every edit thoroughly but it's tiring and disheartening to see so many things that should stay closed.
    – VLAZ
    Commented Dec 2, 2020 at 5:56
  • I also spend little time in the Reopen queue compared to the CV queue for the reason Makyen and VLAZ put forward.
    – TylerH
    Commented Dec 2, 2020 at 15:19
  • 3
    As a regular in SO CVR, I think this write-up is great. Thanks @Catija.
    – halfer
    Commented Dec 2, 2020 at 18:37
22

No, users can't know their post is discussed in some chat room, unless they monitor this chat room.

I don't think they should know this. Those who are familiar with the system can simply be in the room and read the activity; those who aren't won't understand what it means and will just get confused and annoyed.

6
  • 10
    Ultimately SOCVR is upholding the site rules. A question that should be closed might or might not pass through the chat room. It's really irrelevant - the chat room isn't there to close questions that should stay open. So, a closure could happen with or without SOCVR's involvement - it rarely matters to the end result. We currently don't inform people that their question went through the close queue, either - I don't think SOCVR is much different.
    – VLAZ
    Commented Nov 30, 2020 at 14:22
  • @VLAZ I think it matters a lot. Compared to the close vote review queue, Questions that go through socvr is almost always closed.
    – TheMaster
    Commented Nov 30, 2020 at 14:25
  • 10
    @TheMaster I think that's a subtle example of begging the question here. I think what you're trying to put across is that these closures are wrong. Do you have examples of this? Because it's also possible that most questions that are posted in SOCVR should have been closed.
    – VLAZ
    Commented Nov 30, 2020 at 14:28
  • @vlaz I don't disagree that majority of close vote requests posted on SOCVR are valid requests and I agree with closing some of them. But You just can't claim that the end-result matters rarely. A close request is rarely rejected in SOCVR. And SOCVR plays a major part in closing questions by providing increased attention to questions that won't get that attention without it
    – TheMaster
    Commented Nov 30, 2020 at 14:34
  • 10
    @TheMaster if the question should be closed, and then it does get closed, then does it really matter whether it was posted on SOCVR or not? I'm not sure I see the difference there.
    – VLAZ
    Commented Nov 30, 2020 at 14:37
  • 14
    It is to be expected that a higher percentage of the questions that pass through SOCVR get closed. The reason is a simple, non-conspiratorial one: the bar for entry is higher, so the questions that get posted in SOCVR as being in need of closure are far more likely to be true positives. It's a lot easier to just cast a close vote/flag on a question. It's harder to open up a link to the chat room, paste in a question, write out a reason, follow the chat rules, etc. You don't do it unless you're quite certain that the question does need to be closed.
    – Cody Gray Mod
    Commented Dec 1, 2020 at 3:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .