210

For many, many years users have asked and asked and asked for changes to the number of close votes and the number of reviews per day. There's been a little bit of skepticism as to the efficacy of such changes, but so far that's been mostly speculative. So... Let's test it!

Current Problem:

We've got too many items staying the in Close Vote and the Low Quality queues for far too long before being handled. Over the past 30 days, we're averaging about 9,500 close vote and about 750 low quality reviews in the queue on a daily basis:

Review

Close Vote reviewers currently get 40 reviews a day, the Low Quality reviewers get 20 a day, and a max of 50 close votes per day.

What's Changing for the Test?

Over the next 30 days, we'll increase the number of reviews per day and max close votes to the following:

  • When the queue size is less than 1000, the max reviews per day is 30
  • When the queue size is 1000 or more, the max reviews per day is 60
  • Maximum close votes = 60 to correspond to the new max reviews per day

How will this change be evaluated?

At the end of the 30 days, we'll run the numbers again to see if these increases impact the average daily size of the queues. We are determining success the following way:

  • If we see a reduction in size by 20% of the close vote and low quality review queue, then we will assume that this worked and we'll look at leaving the new thresholds in place permanently.

  • If we don't hit a 20% reduction, then we will classify this as a failure. The assumption being that the problem is not the number of votes or number of reviews.

While these increases will impact the other queues, we're really looking at reductions in both the Close Vote and Low Quality queues.

How can you help

Pretend you didn't read this. We already have a lot of users reviewing, so keep doing what you are doing. This change doesn't mean you have to go and use all of your reviews per day, unless you really want to. We expect most users will continue their normal behavior in review.

I'll be back in 30 days to report on the results...

1

4 Answers 4

83

Update (August 4, 2016)

We've updated the settings a bit:

  • When the queue size is greater than 150, you will have 40 reviews a day
  • When the queue size is less than 150, you will have 20 reviews a day

We also updated the settings in Triage to allow a maximum of 200 posts at a time. By increasing the number of posts allowed in Triage, it will allow the possibility of more posts to be reviewed by users each day.


Update (April 26, 2016)

As mentioned in the initial answer, we were going to be working on a way to better scale the number of reviews available in the Low Quality review queue.

Effective immediately:

  • When the queue size is greater than 200, you will have 40 reviews a day
  • When the queue size is less than 200, you will have 20 reviews a day

This impacts all review queues but based on the average size of the queues, it will primarily impact the Low Quality queue.


The 30 days allotted to this test have come to an end, and the limits adjusted for this purpose have reverted to their normal values. Now, it's time to analyze the results and see what effects altering these limits had during the test period.

Results: size of review queues

Increasing the number of reviews and close votes impacted the size of the review queues. Overall, both the Close Vote Review and Low Quality queues had a substantial drop in the average number of posts during the test period.

As a reminder, in order to say this was a success, we wanted a 20% decrease in the queue size. While we crushed that in the Low Quality review queue, we did not hit those numbers in the Close Vote queue.

30 days later

  • The Close Votes had an 11% reduction in queue size, dropping from 9,500 per day to 8,400 per day
  • The Low Quality queue had a 74% reduction in queue size, dropping from 750 per day to 192 per day

Over this period, we had 1,930 distinct users reviewing in both queues. But, very few of the reviewers capped out their reviews at least once during the test:

  • 113 users maxed out their close vote reviews
  • 405 users maxed out their low quality reviews

Tangential results: automatic classification of low-quality answers

Based on Brad's suggestion , I looked at how accurately the system identifies answers that should be deleted before dumping them into review. Well, the short answer there is "not very accurately". An awful lot of stuff is being dumped into the queue that won't ever be deleted, and probably shouldn't be; this wastes reviewer time and prevents them from looking at things that've been flagged and thus stand a much, much higher chance of actually needing attention. We're going to tweak the settings to reduce the amount of noise generated by this system; we expect this alone to reduce the backlog by close to 20%.

What's next?

We've stopped the test, meaning that you're back to 20 low quality reviews, 40 in close votes, and back to 50 max close votes for the day. But don't fret we're not going to stay here for long.

This test made us realize a few things:

  1. More reviews in the low quality queue means more NAA/VLQ flags are processed out of the moderator flag queue quicker, helping with the backlog

  2. We need to do something about the close votes, but having the same number of reviews and votes doesn't work well for users who want to review and close outside of review because they'll run out of votes.

We'll be tweaking the automated quality checks immediately (as noted above). Next, we'll work on building a way for the number of reviews available in the Low Quality queue to scale more effectively.

Then, we'll start looking at increasing the number of close votes based on rep.

43
  • 37
    -1 for turning back the number of Close votes, +2 for increasing the number of close votes based on rep. Thanks for running the experiment and the feedback.
    – rene
    Commented Apr 13, 2016 at 19:40
  • 1
    @NathanOliver More people maxed out their CV reviews in the 30 days prior to the test. I grabbed the last 30 days when it was set to 40/day and were 150 users who hit the max at least once.
    – Taryn StaffMod
    Commented Apr 13, 2016 at 19:51
  • 7
    less people maxing sounds realistic. I pick the low hanging fruit when in the queue, making it as easy as possible but even this way, even after getting used to new limit, it still felt noticeably harder than 40 reviews a day. I would say 60 reviews seems to be really approaching reviewer fatigue. You probably can stretch it a little bit more by "amplifying" current CV prioritisation heuristic (which works already well btw), but only a little bit
    – gnat
    Commented Apr 13, 2016 at 19:57
  • 6
    Yup @gnat those last 20 can be tough when you've already looked through a bunch of stuff that needs to be closed. Burn out can happen quickly in review.
    – Taryn StaffMod
    Commented Apr 13, 2016 at 19:58
  • 4
    "113 users maxed out their close vote reviews" how many maxed out their close votes? I know I didn't max out my reviews, but I did max out my votes.
    – hichris123
    Commented Apr 13, 2016 at 20:07
  • 3
    @hichris123 108 users maxed out their close votes at least once during the test.
    – Taryn StaffMod
    Commented Apr 13, 2016 at 20:17
  • 1
    @bluefeet does that take into account a couple of mods that might have bothered on a couple of occasions to have a whack at the CV queue?
    – Jon Clements Mod
    Commented Apr 13, 2016 at 20:19
  • 1
    @JonClements I didn't exclude mods from these stats. Based on my quick once over, there weren't too many mods in the list.
    – Taryn StaffMod
    Commented Apr 13, 2016 at 20:20
  • 5
    All in all, I appreciate the 50% increase in the amount of reviews I could handle. Maybe a year from now I'll help out with those close votes.
    – CubeJockey
    Commented Apr 13, 2016 at 20:30
  • 2
    Separate Close Votes from CVQ votes :-) Problem solved!
    – TylerH
    Commented Apr 13, 2016 at 20:46
  • 3
    this question was asked 28 days ago, how can that be 30 days, or did you start it before the post?
    – user3956566
    Commented Apr 14, 2016 at 3:44
  • 6
    Interestingly, after an initial bump at the start fewer people hit >=40 votes/day during the last half of the test, @gnat. So yes: fatigue.
    – Shog9
    Commented Apr 14, 2016 at 4:29
  • 1
    @Yvette 30 days-ish, it ran for 4 weeks.
    – Taryn StaffMod
    Commented Apr 14, 2016 at 12:37
  • 1
    @honk Done, just hadn't gotten back around to it.
    – Taryn StaffMod
    Commented Apr 15, 2016 at 20:48
  • 4
    I agree with @gnat, there should be more priority given to posts with 4 close votes already. Perhaps there is an argument to make that this is giving bias towards robo reviewers since they are more likely to be accurate simply voting along the existing votes trajectory. However, I believe that once 4 votes are present the trajectory is more than likely accurate anyway especially given the low rate of reopen or edit after closure occurrences. As far as fatigue, I am not sure it will be a cure, but at least the effort will be instantly recognizable which plays into satisfaction levels.
    – Travis J
    Commented Apr 26, 2016 at 21:03
74

As long as we're talking about testing and the Low Quality Posts review queue, can I suggest two more things to evaluate?

The hit rate of system-heuristic-identified posts inserted into review. The Low Quality Posts review queue used to be only fed by posts that were identified by a quality heuristic (poor capitalization, short length, etc.). Then SE started feeding posts into it that had been identified by "very low quality" flags, and finally ones with "not an answer" flags.

I'm no longer convinced that system-identified posts serve a useful purpose in this review queue. I'd like to know the percentage of system-identified "low quality" posts that did not receive a flag from a human and that were later deleted in review. When I did a manual review of this, I saw very little signal in these. If that percentage was low enough, I'd argue that we should no longer have the system identify these in order to focus the queue on more relevant items.

Reducing the number of votes required to delete or dequeue a post from Low Quality Posts review. I have been pleasantly surprised at how relatively problem-free reviews in this queue have been. At present, it takes quite a few reviews to delete a single non-answer or other post from review.

I believe that it would not harm the quality of review too greatly to examine reducing the number of votes required to delete an item or mark it as acceptable. Even reducing by a single vote might be enough to bring review closer to parity with the rate of incoming "not an answer" flags.

Increasing the number of reviews per person could help, but the above might also have a significant impact on the processing rate of the Low Quality Posts review queue.

23
  • 2
    This is a first test to see if we can make an impact in the queue. I'm not sure if we will make other tweaks with this test. I will say that I'm looking at a few other things, but we want to test one thing at a time. However, I'll see about pulling some numbers of what's being fed into the low quality review queue.
    – Taryn StaffMod
    Commented Mar 16, 2016 at 18:41
  • 14
    To the second point, having users with the 20k rep to cast delete votes count for more than the 1/5 of the users necessary to delete the question would help. If those delete votes, rather than recommend delete votes, counted for, say, double, that might help.
    – Servy
    Commented Mar 16, 2016 at 18:42
  • 6
    @bluefeet - Oh yeah, best to change one variable at a time. A few moderators were talking about this stuff yesterday, so I thought I'd toss these out as possible future items to look at.
    – Brad Larson Mod
    Commented Mar 16, 2016 at 18:43
  • 6
    @Servy that make a lot of sense. Since 3 20k users can delete a bad answer outside the queue, and it takes 6 "Recommend deletion" votes within the queue, it seems pretty logical that delete votes within the queue count for 2 "Recommend deletion" votes (2/6 = 1/3).
    – ryanyuyu
    Commented Mar 16, 2016 at 18:49
  • 6
    If we aren't willing to ditch the heuristics entirely, can we at least prioritize the flagged posts (which a human has already judged to be a problem they organically found when browsing the site) over the heuristic-identified posts? (In a separate experiment, of course.) Commented Mar 16, 2016 at 21:44
  • 2
    @JeffreyBosboom As I mentioned, I'm looking at a few different things... patience you must have.
    – Taryn StaffMod
    Commented Mar 17, 2016 at 0:03
  • 1
    @Trilarion in my experience it's a friggin' problem in LQ queue, it breaks the flow and breaks it hard. This is especially noticeable when compared to CV queue which feeds me easiest items first. I wouldn't mind wasting my time, 20 (30) reviews a day isn't a big deal but the real harm is, as a result moderators get loaded by hundreds flags that could be handled by reviewers in a matter of minutes. I've seen blatant, textbook NAAs hanging in queue for hours until they eventually reach moderators - this just doesn't feel right
    – gnat
    Commented Mar 17, 2016 at 12:30
  • 2
    @bluefeet I can attest to this beeing the case heavily. About 50-60% of the posts I review in the LQP Queue are "Try this: $code". Those are answers. Either the heuristic is not very good or a ton of people are on a flagging spree.
    – Magisch
    Commented Mar 17, 2016 at 15:26
  • 1
    @Magisch Flagging spree - bad flagging spree.
    – Taryn StaffMod
    Commented Mar 17, 2016 at 15:27
  • 2
    @bluefeet I feel horrible having to OK most of these posts, but answers are answers and them beeing bad doesn't make them delete worthy. Shouldn't people who go on bad flagging sprees get a flag timeout after so many declined ones?
    – Magisch
    Commented Mar 17, 2016 at 15:28
  • 1
    @bluefeet - I have noticed that the heuristic does tend to throw a lot of "try this: (code)" answers into review, because it skews towards shorter answers. These were many of the ones that I saw when I swept through and started wondering about the hit rate of the system heuristics. That's not to say that there aren't some people out there who blindly flag any answer with "try this" in it, but the system does seem to contribute more than a few of these.
    – Brad Larson Mod
    Commented Mar 17, 2016 at 15:37
  • 2
    For example, these are all posts that were inserted into review by the heuristic this morning: stackoverflow.com/review/low-quality-posts/11669804 stackoverflow.com/review/low-quality-posts/11668147 stackoverflow.com/review/low-quality-posts/11669673 stackoverflow.com/review/low-quality-posts/11669865 and they all follow the "try this" pattern of answer. None are worth the time to review for deletion.
    – Brad Larson Mod
    Commented Mar 17, 2016 at 15:44
  • 1
    @gnat I review c and sql CVs daily. They tend to be ~60% "I instantly close this", ~10% merits closer examination and ~30% Skip or leave open.
    – Magisch
    Commented Mar 17, 2016 at 18:10
  • 2
    @Magisch 60% insta-close is just the magic of system prioritizing CV queue (explained here). If we had something like that in LQ, life would be so much easier for everyone (human flags and items with larger amount of already performed reviews get higher priority, it's not a rocket science)
    – gnat
    Commented Mar 17, 2016 at 18:15
  • 1
    Regarding the accuracy of the automated heuristics... Might wanna have a separate thread for that, since posting data in comments is not going to work terribly well.
    – Shog9
    Commented Mar 17, 2016 at 20:48
48

Alright, well, shoot. Whether it’s reverse psychology, the Hawthorne effect, or some other fun social phenomenon, I am only human, and I am too curious for my own good. So why not? I decided to go back into the review queues after being thoroughly discouraged since my initial post on the subject and see what happened.

First of all, yay. Thank you. It is nice to know that my request was heard, if perhaps a year late. Having more close votes sure does feel different, and it’s actually pretty nice. Unfortunately, this implementation actually kinda missed the point of my original post.

I'm not really complaining about a failure to close all the questions. That won't ever really be fixed. But frankly, I want to be able to close questions that I deem poor. When I run out of close votes, I stop trying to answer questions in poor-quality tags because I'm frustrated I can't close the awful ones. This actually reduces my positive contributions to the site, and I don't imagine I'm alone in that.

It’s not about the review queues

When I posted my initial request for more close votes, I scraped together a scary graph displaying the number of unclosed questions per month:

This graph shows a symptom—questions getting some close votes but never enough to seal the deal—a symptom that the close vote queue is pretty much explicitly designed to fix. With that in mind, it might seem like this issue is really about the review queues. It isn’t.

Here’s what that graph looks like today, in March 2016:

What happened? Well, Shog happened. New close vote aging logic was implemented to age out close votes that weren’t getting enough attention, replacing the old logic which let them stick around. With this in place, the effects were sudden, and the effects were total. The review queues were no longer a problem.

Reviewing posts is a chore

So, let me be honest: I do not like going through the review queues. I haven’t been doing it much lately, but that is not because things have changed, I’ve just been a lot busier. That’s not the point. I don’t know if anyone actually enjoys clicking through posts in the queue (almost all of which are terrible), but I don’t, and I would expect I am not alone in that feeling.

So then why would I use the queues? Why do I use the queues? Well, there are some motivators:

  • Stinkin’ badges. I already have both the close vote queue badges, so this is less relevant to me, but some good gamification can’t ever really hurt, especially just to get things going. (Generally, though, I think the motivation factor of these badges is overstated.)
  • Small, easy goals. The review queues are generally nearly instant gratification. Just take a few minutes out of your day, and boom. Twenty posts reviewed. This is an easy goal to work towards—it isn’t hard, but it feels good to have finished it.
  • Feeling like you’ve done your part. Related to the above, getting that “come back tomorrow” screen feels good. It says “I did my part for today”, and that’s it. I can feel a little tiny bit better about myself, and I can get on with my life.

Those last two points are especially important because they create a positive feedback loop to produce a self-sustaining system. First, the goal feels achievable, so I’m willing to invest some time to just knock out the queue. Second, I get a feeling of gratification when I’m done. This makes me want to do more, which I can do consistently because it is so easy to do each day. Et cetera.

This balance is delicate, and it almost certainly isn’t the same for everyone, but erring on the side of caution is important to maintain that positive loop because...

Protecting people from themselves

...queue burnout is a real thing that happens. I just ran through the queue and reviewed sixty—that’s right, sixty!—posts. That is a big number. And you know what? I’m not sure I would want to come back and do it tomorrow, even though I might if it were a paltry forty. Now, one might point out that I could just do twenty and call it a day, but that’s not what my brain’s reward centers crave: I want that positive reinforcement, dammit! So I am possibly doing even less by being given the option to do more.

So with that in mind, was Shog right and was I wrong? Was increasing the cap a bad idea with potentially disastrous consequences? Well... I still don’t think so. It isn’t quite that simple.

It was never about the queues, anyway

Let me select some quotes from my question and its associated comments. First, that original quote once again, from me:

When I run out of close votes, I stop trying to answer questions in poor-quality tags because I'm frustrated I can't close the awful ones.

From Jim Garrison:

After I run out I see no point in slogging through the muck looking for the occasional gem if I can't help clean up the muck. I really believe in StackOverflow as a resource and I'm not here for the rep. The vote limits are contribution limits.

From Josh Caswell:

For myself, I know that I only hit the cap when I use /review -- and (paradoxically?) that's part of the reason I don't use /review much. Burning through the queues -- and thus my votes -- doesn't take all that much time compared to the rest of the normal browsing day. Sure, I could stop reviewing early, and save myself more votes depending on the time, but then I'm putting way too much strategic thought into, as Will likes to say, mopping up the sick on the floor. (And I'm not even sure there's a displayed counter of daily close votes.)

From tchrist:

Despite being the top CV reviewer there, I "seldom" use the CV queue on ELU because then I run out right away. Yes, I get a few more but I still run out of CVs there on most days I'm an active voter. I bet if you look at the site stats, you'll see this isn't rare. I try to treasure them up and look for canidates in the 10k queue as need be. Just too many need closing to squander my CVs, which means I don't use them enough as much as I should.

The people who are hitting the caps and getting frustrated don’t want more time in the trash compactor, they want the privilege to throw the litter they find in the trash.

The people who spend time in the queues are on cleanup duty, but they are really a necessary evil. Ideally, we wouldn’t need to waste valuable human resources on validating close votes. If all close votes were cast intelligently and fairly, the close vote queue would be pointless—but we’d still need the initial person to cast that first vote. Ultimately, this is where the frustration seems to happen the most.

Close votes are a panic button

In Stack Overflow’s most popular tags, there is an extremely low signal to noise ratio, and the “close” button is a way out. It’s a way of saying “eep, this question is not worth my time or anyone else’s, please get it out of my face!” (Or for you power users, it’s pronounced mc24enter.)

The close vote is comforting to the user searching to answer good questions because it marks that question as adequately managed and has signaled to the system that it is of low quality. Theoretically, this system (along with ordinary up/downvotes) helps good questions rise to the top as the sand sinks to the bottom. Whether or not this is actually true in practice is mostly irrelevant to this particular post; the point is simple: being unable to report poor questions causes a surprising amount of psychological friction on the part of the benevolent user. The barrier is not being unable to close all the questions, it is being deprived of the way out: the panic button.

What does all this have to do with this question, again?

Alright, so the issue is that people ran out of close votes, and we increased the number of close votes people have. Alright! Good work, team. But wait, no, we tried to solve a problem but instead we made two problems:

  1. Going through the close vote queue is more of a chore than it used to be.
  2. After going through the queue, I am still out of close votes, and this makes me more likely to stop sifting through crap.

Well, fortunately for us, Shog has an answer:

Also, I recommend not reviewing until you're about done for the day - I used to leave that sort of thing for about 23:00 UTC.

Of course, that’s not really a perfect solution, either:

Not reviewing until the end of the day, dose not work well for people in different time zones. I think part of the problem is that the RISK of running out of close vote is putting people off form reviewing a lot, even when most of them do not run out of close votes.

(Plus, it interacts poorly with the psychological gamification tricks mentioned earlier.)

What am I arguing then? Well, my opinion isn’t really anything new—Félix Gagnon-Grenier’s answer to this very same question already makes my point for me!

TL;DR; We should still have more close votes than reviews.

Josh Caswell had a slightly different solution, but the spirit was the same:

Votes of any kind in each queue should be limited only by the queue's daily task limit; they should not be deducted from one's pool of that kind of vote.

Yes, this is something of a different conclusion from my post a year ago, and yes, this is a bit of a cynical view of the problem. However, I think the stats don’t lie—no matter how the queue is doing now, those questions clearly aren’t so awful that they are truly in need of closing, given that the votes just age away, anyway.

I think it is worth acknowledging that this is not really a technical problem but a social and psychological one, and understanding our (sometimes irrational!) frustrations is important to understanding what exactly the solution needs to be.

Thank you for taking the time to consider these issues, thank you for being willing to hear our feedback, and a big thank you for reading all this and taking it into consideration. It is well-appreciated. And who knows? Maybe it’s time for me to get a little more involved again.

8
  • 6
    "The people who are hitting the caps and getting frustrated don’t want more time in the trash compactor, they want the privilege to throw the litter they find in the trash." - perhaps a more "powerful" close vote (similar to the dupehammer) would be beneficial here? It seems a large part of the problem is questions languish for days until 5 close votes can be obtained.
    – hichris123
    Commented Mar 20, 2016 at 16:38
  • 2
    This matches my experience: Hawthorne effect drew me in, 60 reviews is a total slog, and then afterward I elected not to close a question because I didn't want to "waste" one of my four remaining votes. In contrast, I really enjoyed having extra LQP reviews: I cleaned up more crap in the queue and I felt better flagging crap when I came across it later because I knew it would be cleaned up soon after. Commented Mar 20, 2016 at 21:35
  • 8
    The fact that we have exactly as many CVs and reviews artificially lessens the queue, but only because people reviewing properly in it no longer have the votes to push more qs into it.
    – Magisch
    Commented Mar 21, 2016 at 7:34
  • 6
    Whats additionally problematic to this change is that while reviewing 60 CVs a day, you have not enough vote left to properly review triage / FP posts. You end up skipping more then half or beeing artificially lenient, and both of those are bad.
    – Magisch
    Commented Mar 21, 2016 at 12:53
  • 5
    So... This is an experiment. Critically, it's an experiment we can run without changing any of the logic that governs close votes, which is nice because honestly I doubt we'd do it otherwise. 50 close votes is rather a lot of close votes; I've been saying this for years, but there are very few people who can tolerate looking at that many crap questions in a day and still have the energy to pick a useful close reason for each one. It's a slog, and it doesn't much matter how you approach it. And that's fine... As long as your vote means something when you DO vote. Which is where review comes in.
    – Shog9
    Commented Mar 21, 2016 at 17:24
  • 9
    @Shog9 I think you may have missed the point, which is actually probably my fault for not making it clear enough. Let’s make it clear this time: I simply think the number of close votes should be increased to 60, but the CV reviews per day should remain capped at 40. That doesn’t require changing any of the logic (at least I would hope it doesn’t), and it accomplishes my goal. Commented Mar 22, 2016 at 5:42
  • 1
    I don't actually see how that wouldn't make the problem you're describing worse, @Alexis.
    – Shog9
    Commented Mar 22, 2016 at 21:03
  • 1
    @Shog9 Link to our relevant discussion in chat for context. Commented Mar 23, 2016 at 0:45
32

TL;DR; We should still have more close votes than reviews.


I would like to discuss an aspect of this which may have an impact on the results, which is the total quantity of close-votes being tied to the max number of close votes reviews

Maximum close votes = 60 to correspond to the new max reviews per day

This at first sight seems pretty logical, but I really wonder if it won't have an unforeseen effect.

Egotistically, I'll just think about my experience: I often (relatively, I have not been in the 3k group for a long time) cast close votes from using the site. This did not have an effect before because, even if I used a few close votes, I still had enough to actually handle 40 reviews.

However, being able do 60 reviews, I still cast close votes from outside the review queue with my close votes limited at 60.

So, effectively, having 60 reviews per day won't actually do anything about the close votes review queue. I'll still handle the exact same number of close votes reviews per day, except I will never reach the limit of reviews per day.

Can we have 80 close votes per day, please?

That is, at least for the purpose of the test, because I think that would affect the results.

8
  • 2
    Yeah, this setup kind-of encourages me to wait to go through the review queue until shortly before UTC time roles around, even more so than before. Commented Mar 17, 2016 at 5:22
  • 4
    they could at the very least give extra CVs to bronze / silver tag badge holders in their tags
    – gnat
    Commented Mar 17, 2016 at 5:26
  • 5
    See my similar request on MSE. It proposed to split the votes between review queue and regular view. It will make users use those votes in the right way. Commented Mar 17, 2016 at 7:53
  • 6
    I don't think we'll be making any additional changes during this test. The purpose of this is to see how these tweaks impact the size of the review queues. Giving more close votes without additional close reviews could potentially offset some of the work in the review queue as users would have 20 more votes to put more stuff in review.
    – Taryn StaffMod
    Commented Mar 17, 2016 at 12:07
  • 1
    Yeah, I talked about this in one of the previous discussions of increasing close votes, but if folks are less pressured to prioritize voting that can easily make the backlog much worse. There are, of course, complicated ways of avoiding that... But the whole point of this test is to see if it's even worth putting more thought into.
    – Shog9
    Commented Mar 17, 2016 at 20:51
  • I'm confused about what you're asking... you're saying we have more reviews available but also more close votes so the overall balance will stay the same? Or are you saying something else?
    – TylerH
    Commented Mar 18, 2016 at 11:47
  • 4
    "Can we have 80 close votes per day, please?" - Lordie, what with the ever increasing amount of bad and/or poorly written questions these days, that would be a reasonable amount, if not "100". Many a time, I have to take it easy as to the amount of votes to close I put in, or used to put in. Newbies (and some who have been around for a while), tend to post questions with a lot of the missing essential parts they deem as being irrelevant. Well, well, well...., we read code and not minds. They just take off and expect a magic answer. Sometimes it happens, other times it goes stale. Commented Mar 18, 2016 at 12:13
  • @TylerH Yep, that is partly what I am saying. We have (for the duration of the test) more reviews, but not more close votes. Which means that the overall balance will not stay the same. If the close votes/reviews ratio stayed the same but was augmented by 20, then the overall balance would stay the same while favoring more reviews being made. Commented Mar 18, 2016 at 18:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .