34

TL:DR: Compared to 2016, in last year of 2022 there were 6.3 times less reviews of suggested edits and 3.6 times less total reviews. Is this estimate accurate and if yes, what does such a decrease mean?


In my first several years at Stack Overflow I've been actively using reviews and suggested edits and seen others actively using these. Seeing this and reading meta made me feel that these are important features. There were active discussions and developers invested quite some effort into maintaining these features.

Some time later I started getting a feeling that something goes wrong. For quite a while I ignored this feeling thinking that it's just an old fart nostalgy, when I was youger grass was greener yadda yadda. Recently I figured that there is statistics data available to find out what's really going on, instead of feelings and guessing.

I found moderation reports for years 2016, 2018, 2019, 2020, 2021, 2022, and studied their data about suggested edits and total reviews. I was quite surprised with the trend I observed (guess I wouldn't if I paid closer attention to last years complaints about suggested edits).

There were fairly strong decreases year by year in the range of 20-30-40%, with total drop between 2016 and 2022 aggregating to massive 3.6 times less in total reviews and 6.3 times less reviews of suggested edits.

To me this looks like features I considered important in the past are essentially disappearing. And I wonder what could that mean and what could we expect next? Should we expect them to eventually fade into peripheral features of low impact?


Below are details about data I used - feel free to skip if you're not interested in detailed assessment of this study.

Here are counts I've got for total reviews in all queues:

Year Count Notes
2016 6,867,383  
2018 4,966,450 28% decrease
2019 4,346,456 13% decrease
2020 3,435,850 21% decrease
2021 2,613,983 24% decrease
2022 1,910,184 27% decrease / about 3.6 times less reviews than in 2016

Here are counts I've got for suggested edits reviews:

Year Count Notes
2016 1,864,076  
2018 1,177,058 37% decrease
2019 1,141,695 4% decrease
2020 817,980 29% decrease
2021 529,477 36% decrease
2022 295,589 45% decrease / about 6.3 times less reviews than in 2016

I tried to estimate maybe people are just unaware of reviews and it looks like this is not the case: over a million Custodian badges indicate that at the very very least 130 thousands users have tried reviews at least once (this is based on most pessimistic estimate that all badges were given to 3K users who can earn them 8 times).

Also, after observing such a sharp drop in suggested edits I decided to re-check how much it contributed to decline of total reviews. For that I calculated difference between 2016 and 2022 for all reviews excluding suggested edits. I've got about 3.1x decrease which is less than 3.6 but not substantially.

I also studied some events in site history that could potentially impact what I observed. I could not figure why decrease in 2019 looks lower compared to other years, there is a chance of a mistake somewhere in my calculations (when drafting and re-checking this question I found at least one mistake that had a substantial impact on my analysis).

I considered changes in review thresholds for suggested edits in May 2017 and for close votes in Dec 2019. Couldn't figure if these had any substantial influence on the data above.

I also considered influence of campaign rolled out in April 2018. It wasn't explicitly about reviews but maybe these were considered kind of an obstacle for a new party line, especially close reviews. Statistics of company efforts related to content curation that dropped to zero in 2018 (discussed here) suggests possible connection here, although smaller decreases in 2019 indicate that it's not really straightforward.


As was suggested in comments I additionally estimated total amount of review items that entered the queues, including those that weren't acted on. Purpose was to check if maybe this could somehow factor in decrease of reviews.

Per my observations, there was a trend of decrease from 2016 to 1021, although less pronounced than in performed reviews:

Year Count Notes
2016 4,010,000  
2017 3,650,000 9% decrease
2018 3,410,000 7% decrease
2019 3,190,000 7% decrease
2020 2,950,000 8% decrease
2021 2,770,000 6% decrease
2022 2,820,000 2% increase / about 30% less items than in 2016

I could not figure how to estimate per-queue amount of review items and because of that it is difficult to make definitive conclusion of what above means. However, from data available it looks like the most likely explanation is that this decrease reflects one in suggested edits reviews.

Reason why I think so is that suggested edits seem to be the largest part of reviews, making for almost a third of all the reviews in all queues and these suffered most decrease in reviews. And what is specifically important here is the blocking nature of suggested edits queue meaning that the fewer reviews are made, the fewer items are allowed to enter this queue.

39
  • 19
    "Is this estimate accurate and if yes, what does such a decrease mean?" - that fewer people are interested in reviewing for one. The input to suggested edits in particular is tied to its output; fewer people interested in reviewing means fewer edits enter the queue over a long period of time. The queue is constantly at the cap, so it's clear it's a throughput issue and not an input issue Commented May 4, 2023 at 14:11
  • 18
    I used to review suggested edits a lot - nowadays I barely do any as opening that queue feels like diving into a cesspool with a spoon. I suspect I am not the only one getting the same feeling. At the same time, I still consider suggested edits an important feature.
    – 0Valt
    Commented May 4, 2023 at 14:13
  • 7
    If the quality of suggested edits were better, I would suggest that the volume reviewed would likely increase, and thus the number that can be submitted would increase; hopefully resulting in a somewhat linear increase. Like Oleg mentioned though, for many the queue is not an "enjoyable" experience; I personally just tend to review edits I see on questions in my "normal usage" of the site.
    – Thom A
    Commented May 4, 2023 at 14:25
  • 17
    One thing that might help not discourage reviewers would be allowing moderators to warn reviewers about mistakes rather than suspending them from review for the first instance of bad reviewing.
    – Ryan M Mod
    Commented May 4, 2023 at 14:26
  • 22
    Half-baked idea: decreasing or removing the rep gain for approved suggested edits might reduce the volume of low-effort, low-impact suggestions in the queue, as it would mean that all the edits would come from people genuinely trying to improve the site.
    – Ryan M Mod
    Commented May 4, 2023 at 14:29
  • 6
    speaking of suggested edits, I wonder what if we tried to let them expire (just like close votes do). I recently did some preliminary study in that area but haven't yet made up my mind about what to do with its results
    – gnat
    Commented May 4, 2023 at 14:30
  • 8
    A lot of people have left in the last years, because they lost faith in the company and thus the network. One of those incidents is "Firing mods and forced relicensing: is Stack Exchange still interested in cooperating with the community?" which left quite the sour taste in many mouths.
    – Lino
    Commented May 4, 2023 at 14:33
  • 5
    🔥 burn it with fire 🔥 we used to have occasional "community events" on meta that we'd feature and then call on the community to perform a specific curation task. Simply getting people to be involved in something they ordinarily wouldn't would attract people to continue to be involved after such events.
    – Kevin B
    Commented May 4, 2023 at 14:36
  • 6
    I don't think I see the value in expiring suggested edits. Edits are something that require substantially more investment to create than a vote. Having them expire just due to lack of reviewers would discourage people from putting meaningful effort into creating them.
    – Ryan M Mod
    Commented May 4, 2023 at 14:55
  • 13
    @Andreas what if approved suggested edits didn't count for reputation, but did count toward being able to edit without review?
    – Ryan M Mod
    Commented May 4, 2023 at 15:10
  • 11
    As a final note; edits should not be a 2k privilege. It makes no sense. The first X edits for every user, irrespective of reputation, should be reviewed. Then when they reach a consistent set of approved ones, (and no defacing, damaging, etc), they are granted the privilege to edit. I made some bad edits in my first days at the site. Having them go through review was a useful learning experience. Every member should go through that. If I had skipped that because of a fast reputation gain, I could’ve made unnoticed bad edits. Commented May 4, 2023 at 15:13
  • 12
    This trend is completely unsurprising. The change from using an icon badge with a number to just a red dot happened in early 2017 and the number of reviews needed in each queue was removed from the popup shortly thereafter. The trend of reduced review participation was clear and obvious within a short time (see "Is the top bar redesign the sole cause of the suggested edit slowdown?"). It's no surprise that this trend continued into following years, particularly given the issues which happened across SE in 2019.
    – Makyen Mod
    Commented May 4, 2023 at 16:05
  • 9
    Showing the numbers in the icon badge and popup was a very significant contributor to people feeling reviews were urgently needed, causing people to do more reviews. Just the red dot is something that's easily ignored, particularly when it's there almost all of the time.
    – Makyen Mod
    Commented May 4, 2023 at 16:05
  • 12
    Multiple easy explanations: the overall site usage is declining (question rate peaked in 2014 IIRC); the queues are less discoverable (as @Makyen pointed out); there are basically no incentives and only possible punishments for using the queues and people are worn out from it; the few people (like myself) who actively curate this mess see their time better spent on tasks like identifying canonicals and re-pointing duplicate links. Commented May 5, 2023 at 1:52
  • 11
    Many people are also frustrated by getting automatic review suspensions from poor audits. While audits are a necessary means to combat robo-reviewing, there's really nothing in place to specifically offset the system choosing poor audits. The system, overall, does a reasonable job of picking audits, but it needs the addition of direct human input to the process. It needs at least A) a direct way for moderators to indicate "this post is a bad choice for an audit", and B) for users to be able to explicitly indicate "this audit was bad, please review, and if sustained remove from my record".
    – Makyen Mod
    Commented May 6, 2023 at 23:38

7 Answers 7

11

One possible solution:

Show the number of outstanding reviews on the review icon again.

@Makyen has an explanation for the drop in review queue activity:

This trend is completely unsurprising. The change from using an icon badge with a number to just a red dot happened in early 2017 and the number of reviews needed in each queue was removed from the popup shortly thereafter. The trend of reduced review participation was clear and obvious within a short time (see "Is the top bar redesign the sole cause of the suggested edit slowdown?"). It's no surprise that this trend continued into following years, particularly given the issues which happened across SE in 2019.

So, perhaps seeing that 11k reviews are outstanding right now will motivate users to review more. And seeing that number change over time might help too.

0
10

Another possible solution for the edit queue:

Tune the reputation limits

There are three relevant privileges:

  1. With 1 rep, you can suggest edits which go into the review queues.
  2. With 2k rep, you can work through those review queues.
  3. With 2k rep, you can edit any Q or A without review.
  4. With 1 rep, you can edit your own Q&As without review.

Since the review queues are constantly full, and have been for years it is time to rethink these limits:

  1. If low-rep users tend to make bad edit suggestions, maybe we should bump that privilege from 1 to 25 rep? (25 as an example of a user that is capable of giving an upvoted and accepted answer.)

  2. Lowering the rep limit for working the review queues should increase throughput.

  3. Lowering the rep limit for making direct edits bypasses the queue.

  4. I've only listed editing your own Q&A for completeness. Requiring no rep for that is already ideal both from a usability perspective and from the perspective of limiting the number of edit reviews.

Changing any of these three obviously have side effects. The goal should be to get as many good quality edits as possible through the system (directly or through the queue). It would be great if someone with access to analytics on the edit review queue can share data on the correlations between edit quality and reputation, so we can figure out if it makes sense to change any of these thresholds and to what reputation value. Or perhaps someone has good arguments for specific numbers. I don't. I do have a feeling that lowering the rep limits for edit queue access and direct edits (points 2 & 3) is a good idea.

Note: Creating a mini-reputation system for edits as suggested in an upvoted comment on the question and elsewhere on Meta would fly squarely in the face of the KISS principle and is unnecessary. That would make SO even more difficult to understand and opaque. And making good edits doesn't really require different skills than asking and answering good questions.

10
  • 4
    Yes, lowering the rep limit for 2 and 3 would be better. 200, 500 or 1000 rep would be enough. 2000 is too high, and making good edits doesn't need even 500. Users whose suggested edits get accepted more than rejected should be able to use 2 and 3. Commented May 7, 2023 at 18:25
  • Yes, I read that. I'm not sure how moderator elections take place, but something similar could be used to allow users below 2k rep if the community accepts the user in a queue-like interface after the user joins it. Some AI could be used to check the content changes to make sure it's a good/safe edit. Commented May 7, 2023 at 21:00
  • From a reviewer perspective, I'm quite interested in getting that data that you solicit. But putting myself back in the shoes of myself when I had little to no rep, more barring just makes me a little sad.
    – starball
    Commented May 7, 2023 at 21:45
  • 1
    Lowering limits is pretty much all that can be done and I kind of doubt it will persuade people to actually do things. These are janitorial duties after all, no rewards other than some stinkin' badges. Without reputation/swag gain for the actions you do, I don't think there is much of a dent to be made.
    – Gimby
    Commented May 17, 2023 at 7:48
  • @Gimby Can confirm - I was under the impression that every completed review task gave the reviewer +2, and upon recently gaining access to the review queues, was disappointed to find out that is not actually the case! I still do some reviews, especially ones where it's a new user asking a good question that just needs minor edits (like converting an image to a code block and commenting with a link to the relevant meta discussion), but nowhere near the daily limit of 40. While I do appreciate being able to contribute to maintaining the quality of the site (continued) Commented May 21, 2023 at 3:35
  • expecting the casual user to devote extensive time and effort to a task which is so relatively undercompensated (compared to answering questions, for example) just isn't realistic. Commented May 21, 2023 at 3:37
  • Adding even a small incentive (even +1 per task) for reviewers would likely cause a lot more people to jump on board. Of course, that would also contribute to the robo-reviewing problem, but if they're doing that anyway, it might be worth the gain in legitimate reviewers Commented May 21, 2023 at 3:39
  • Why exactly do people robo-review if there's nothing to be gained from it? Now that's something that makes no sense to me :/ Commented May 21, 2023 at 3:42
  • At least some of them are chasing badges, such as Reviewer
    – tripleee
    Commented May 23, 2023 at 19:16
  • I don't think lowering the rep needed to approve edits or edit without review is a good idea. When I first started reviewing suggested edits it took a few hundred to get the feel of what a truly good and helpful edit is. I think lowering the rep would reduce the overall feel of what reviewers have. For increasing the rep needed to suggest edits, I strongly support this. Anonymous edits usually aren't the most helpful/complete and 10-25 rep is certainly not too much to ask for.
    – Michael M.
    Commented May 29, 2023 at 19:56
8

Position: A decline in suggested edits and their reviews does not mean that system is any less important

Even if there really is a decrease, I don't see how that would make the suggested edit system any less important. I see its value as being based in the value of the edit system itself.

The edit system is one of the things I love about Stack Exchange's model. I can improve tagging, fix formatting/spelling/grammar/punctuation, remove noise, etc. Being able to do that makes me very satisfied (and I would be very irked without it- I know because I'm now irked when I can't do this on programming-help subreddits).

I like that the full-editing privilege is a privilege, but I also like that the system is made accessible to everyone regardless of reputation. It means that anyone can contribute to improving the content on the site.

Suggestion: Look at the decline as an indicator of a throughput problem and try to address the throughput problem

As Zoe state in the comments, and as I've stated in another recent answer post of mine, one way to look at this is as a throughput issue. Reviews are not keeping up with suggested edits. The queue is constantly full or close to being full.

If you choose to look at it as a throughput problem, then the solution needs to somehow address throughput. You could:

Now, I know there are suggested edits where it's clear the suggestor could have read more carefully the editing guidelines, or could have put in more effort to follow those guidelines. I.e. thoughput isn't the only problem right now. This leads into...

Suggestion: Don't forget about quality. Systematize teaching how to do good edit-suggesting (and reviewing)

There have been some prior discussions and related proposals on this point (some more generalized than others) (feel free to suggest additions to this list):

Another off-hand:

  • Increase the font size of the guidance in the suggested edit interface. Maybe it sounds dumb, but for me, when I see something with relatively smaller font size, by brain automatically pays less attention to it. If we want edit suggestors to pay more attention to guidance that's already built into their edit UI, ...
5

There are many ways to address this, starting with the "default" approach of officially accepting the observed trend and doing nothing to change it (call it "lazy" if you wish). I can't say that I like it, but it is has an undeniable benefit of saving company efforts, at least in the short / mid term perspective.

Of other approaches I've seen so far, two look most promising to me (these two approaches were also mentioned in comments and answers here): reverting to the old kind of progress indicator and adding human feedback for review audits. Suggestions to tune review-related privileges and introduce moderator warnings as a complement to suspensions also look worth considering, especially the last one since it seems to be really easy to implement and carries no risk.

That said, when preparing this question I stumbled over yet another idea that probably wasn't explored yet...


At some point of my research I decided to have a look at reviews at other sites to see if maybe they have similar problems. And it literally knocked me out when I saw that on other sites reviews work perfectly well—at least compared to us; these looked infinitely better. At first, I shook it off thinking that there is no way to use this observation given so much difference in scale between them and us.

Then as time went by and it felt more and more desperate observing our reviews decline, I started getting back to their perfectly fine reviews and thinking if maybe there is a way to somehow "approximate" our site to what I saw at smaller sites. The next thought that naturally popped was to compare them to our tags.

I did a quick comparison and indeed, it looks like a sensible match. Our largest tag javascript looks merely twice larger than second largest site Math.SE. Yeah, it's still larger, but not by an order of magnitude and besides, their site seems to have much wider scope (the whole math against a single programming language; think about it) which probably levels the playing field even more.

So the idea I came to is, what if we give people the option to pick a specific tag and isolate all their reviewing activity within that tag—including all kinds of reviews, a review icon indicator and last but not least, suggested edits queue limits.

This would let us get back to the review scale that was proven to be perfectly manageable at multiple smaller sites. And the additional benefit of this (also naturally following from smaller scale and narrower scope) is that reviewers would have an option to focus on working in the area they are most interested and most qualified to maintain.

I did a brief check and technically, something like this looks achievable with relatively limited efforts. I think folks not interested in digging into boring technical details of what made me think so can safely skip the rest of this post.

(side note inspired by discussion in comments) I'd like to explicitly stress than this is supposed to be strictly an optional feature, and that current system is supposed to keep working the same as it does now. This is because I believe that attempt to rework all that we've got now would involve prohibitively high effort and risks.


As promised, here is a more detailed explanation of why I think it is realistic to provide "review isolation within a tag" described above.

To start with, having the same tag filter for all review queues looks really easy, because all of them already have filtering, except for Triage. Speaking of triage, I believe that it should have tag filtering anyway. Besides, it has to be relatively easy to do since all other queues already have it; the only reason why this is not done yet is that company probably doesn't want to invest any effort into triage, no matter how small.

The review icon indicator also looks technically simple to narrow down by a selected tag. The only concern that looks worth taking into account is potential performance implications. Performance impact can be addressed by simply refreshing the tag-specific indicator at a lower rate. You know, even if it was refreshed once in several hours, I would still prefer it over the trash we have now which is useless always, no matter how fast it refreshes. Heck, I would prefer it even if it refreshed once a day.

As a last resort, we could try addressing performance concerns by limiting usage of the tag-specific review icon only to top 50 (or 100, or 200) tags - and leaving all smaller tags to a shared indicator. Summing up, this part, or some reasonable approximation of it, looks doable and even not very difficult.

Now, dedicated tag-specific limits for the suggested edits queue may look somewhat tricky. To start with, it doesn't look technically feasible to have it for all 60 thousands tags we have - but even if it was, it wouldn't make much sense. Can you imagine a "suggested edits queue limit" for a minor tag that has just a dozen questions; what kind of nonsense would that be?

I think the only way to get such limits - or more precisely, a reasonable approximation of these limits—would be to have them only for some limited set of top tags—say for 50 (or 100, or 200...) largest tags. This way, suggested edits for all smaller tags would go to a single shared queue.

8
  • 1
    How do people switch their review tag? Is it rate-limited? Or they can never switch? What if I have subject-matter expertise in multiple tags? If users can only review in one tag, who can/will review in the "small tags mixing-pot"?
    – starball
    Commented May 16, 2023 at 21:23
  • I think the idea of scaling queue items in the queue with relation to tag size / number of tags is interesting and can do with more discussion and idea generation. A Q&A post on MSE perhaps?
    – starball
    Commented May 16, 2023 at 21:25
  • 1
    @user it absolutely should be possible to switch tags (maybe rate-limited, hard to tell now). If someone changes their interest say, from java to javascript, system should be able to handle that. Besides, I think it should be possible to switch between "isolated" and current way, because otherwise system would become too complicated. Regarding posting at MSE, I am not yet ready for that because I haven't yet figured much benefits for smaller sites (notice how I wrote that state of their reviews looks so much better than our)
    – gnat
    Commented May 16, 2023 at 22:23
  • 2
    Why only a specific tag? Why not several? Incl. 'negative' tags, like other filters? For example, Python and JavaScript, but excluding Pandas and jQuery? Commented May 17, 2023 at 18:57
  • 1
    @PeterMortensen I considered allowing more than one tag (like in current review tag filters) but it turned out too complicated to figure how it would work so I decided to keep this very preliminary draft idea limited to single tag for simplicity
    – gnat
    Commented May 17, 2023 at 19:25
  • 2
    I have had "C#" and "duplicate" filters on since forever, but I still can't get myself to grind the close/reopen review queues for more than a handful or maybe a couple dozen posts at a time. The stream is infinite, and the reward zero. I don't want to be an unpaid janitor for a company who doesn't care about me but all about money. I write answers for the greater good, for the public. They can clean up their own mess.
    – CodeCaster
    Commented May 18, 2023 at 8:41
  • @CodeCaster at this point I tend to focus on reviews other than CV. I perceive close queue as separate pain point and I don't expect it to be fixed anytime soon. I sure hope for some improvement if we manage to involve more people in reviewiing within "their" tags but I'd be surprised if this led to a major breakthrough. You can have a look at Math.SE to see why I am pessimistic...
    – gnat
    Commented May 18, 2023 at 10:06
  • ...I refer them as successful example and their close queue indeed looks so much better than our, but unfortunately it still looks much worse compared to their other queues. If some tag manages to get their close queue like what they have at Math.SE, this would be a great progress - but still, I think this would be quite far from healthy
    – gnat
    Commented May 18, 2023 at 10:06
3

Here's a possible cause. Let's assume there are two populations of suggested edits. One, edits made when there are substantial improvements to be made to the question, and for which edits take a bit of time. And two, retags and other superficial edits.

Because these are put into the same system and because the two types are treated with the same importance, you are statistically more likely to get short superficial edits. Since these are annoying to reviewers, they stop reviewing. This only makes the problem worse because if the queue grows x2, a future reviewer is now statistically even more likely a superficial edit (if you're reviewing multiple).

There is another negative feedback loop coming from the supply of these edits. Let’s say you're a person who primarily makes type one edits. Because the suggested edits queue is now filled much more frequently, you are disincentivized to make substantial edits, since you commonly don't have the ability to submit them and feel like your time was wasted (as an example, I no longer make edits because of this).

These two problems are compounded by the fact that the suggested edits queue exists in a weird spot rep-wise. Most people who are on this site and contribute quickly gain rep and no longer need suggested edits. So the predominant population of people interacting with the feature are those unaccustomed to the site, which makes it more likely for type two edits to be made.

3
  • 1
    "This only makes the problem worse because if the queue grows x2, a future reviewer is now statistically even more likely a superficial edit" Huh? If the frequency/likelihood of edits is proportional to their diff size, then changing the population size shouldn’t affect the relative composition. Can you perhaps elaborate your conclusion? Commented May 7, 2023 at 16:45
  • Perhaps explain "statistically more likely to get short superficial edits"? E.g., both types are limited to a maximum of 5 suggested edits at any one time. Type 1 may not be able to submit at a higher rate due to this limit (depending on the rates of acceptance, rate of "production", and "production" time consumption for the two types). Commented May 7, 2023 at 20:21
  • Ok, yes, I made some assumptions here. Main one is that if a reviewer jump on, looks at N reviews and if they're all type two, they get annoyed and stop reviewing. Although its true that globally, the distribution of type one and two hasn't changed, the likelihood of someone drawing all type one has increased. (I think at least, feel free to correct me if I'm wrong)
    – code11
    Commented May 8, 2023 at 0:17
3

Key observation to address this question was made in comments, namely that aggregate review stats are heavily skewed not only by Suggested Edits, but also (and even to a greater extent) by Triage decline.

Indeed, Trage reviews dropped more than 9x between 2016 and 2022 - from 2,212,710 to 241,011, and this is a noticeably harder hit than even Suggested Edits, both in relative and in absolute value.

Armed by above observation I took a closer look at some other queues that had large relative decline but looked too small in absolute scale before I learned about the impact of triage. These were close reviews which dropped from 581,204 to 96,495 and Low Quality Posts falling from 758,627 to 242,261. As far as I could tell, none of these stats looks worth worrying about - or at least, not yet.

For close queue, there was somewhat troubling decline from 2016 to 2019 - although even this decline didn't look particularly dangerous because related stats on closed questions did not look like becoming much worse. Since 2019 there is a really sharp decline which looks like mostly influenced by 3CV change in the end of 2019 - and again, after checking related stats of closed questions I got an impression that there is hardly something to worry about.

For the sake of completeness, I am somewhat concerned that we may experience loss of interest to CV reviews but at this point I don't have any data to point a finger at and say, look, we've got a problem here.

For Low Quality Posts, first thing I noticed is that amount of reviews looks rather steady for years 2020-2022 meaning that data I've got doesn't really say that something bad is going on right now. Then I recalled that for last few years whenever I get to this queue if feels like doing well and in particular, system always gives me smaller daily quota meaning that queue size is under 150.

Finally, I took a look at reviews page to see what size this queue is now and it displayed 54 (that's fifty four) posts in Low quality answers queue. Folks, to me this shows that this queue is in a perfectly good shape. I can't tell what caused drop in these reviews from 2016 to 2019, maybe there ware changes in heuristics pushing posts into this queue, or maybe some throttling of users who tend to post low quality content, or maybe improved spam detection and deletion stole part of the work from this queue, or maybe something else - but anyway whatever it was, I just see no data to tell that this queue is experiencing problems.


To summarise, we've got Suggested Edits and Triage queues that look like in a deep trouble (and there are separate discussions about these). We also have close and low quality queues that show fairly large decline in stats but it didn't indicate real issues.

If we discard four above queues and recalculate stats excluding them, then what we get will be decline in 6 years between 2016 and 2022 from 1,450,766 to 1,034,828 - that is, about 1.4 times.

Granted, this trend doesn't look beautiful, but it doesn't look like an indication of a disappearing feature either. As pointed in comments, it may merely reflect overall reduced activity of users.

1
  • 3
    PS just in case if a reader feels that this answer means everything is okay - it does not. Specifically, I think that suggested edits and triage are a disaster. I also doubt that close queue is in healthy state. Though these are separate issues, out of scope of this discussion
    – gnat
    Commented May 29, 2023 at 15:59
2

One of the reasons the review queues are clogged is that some users are eager to submit very minor edits which could easily be superseded by more thoughtful editors with higher reputation.

I can suggest several changes which would reduce the clogging.

  • Drop closed questions from the review queues. Suggested edits to closed questions can still be reviewed by the original author of the post, and any reviewer who stumbles across the closed post and has enough reputation to review, but my subjective experience is that these often add more noise than actual value, and pollute the review queue to boot.

  • Create a separate queue for tag info page (aka "tag wiki") edits. These require a different amount of review effort and different skills than reviewing suggested edits to questions and answers.

  • Impose limits when suggested edits are rejected. Maybe start out with an allotment of one edit at a time, then when you manage to get that approved, raise the limit to two, etc. Correspondindly drop the limit if there are more than occasional rejections. (Some hand waving here. If you submitted five and one was rejected, I'm thinking that would be treated as occasional noise, but having multiple rejections over a short time is signal.) The curve should probably not be strictly linear; some experimentation around this could perhaps be useful.

Frankly, I'm tempted to suggest a 1k rep requirement for suggesting edits until the queue becomes manageable again, but these less draconian measures might already suffice.

(Also, can we please have a ban for anyone who writes "grammer" in the edit summary?)

5
  • 1
    err... i'd argue closed questions are of a much higher priority in terms of editing than any other. Being edited into shape is the entire point of questions being in the closed state.
    – Kevin B
    Commented May 23, 2023 at 19:54
  • 1
    separate queue for tag wikis - with separate limit - is a very very promising idea. When I checked edits queue about a month ago tag wikis took 137 slots of 500, and many of them were cloging the queue for weeks or even months (eg this one took 4 months to handle). Per-user limit for suggesting edits (similar to one we have for flags, right?) also looks worth a try
    – gnat
    Commented May 23, 2023 at 22:20
  • There's a few very reasonable curators active in the chat rooms with less than 1000 reputation. I'm not in favour of excluding them from suggesting edits. I assume there's many more reasonable editors than just them, with less than 1000 reputation, too. And everybody has to start somewhere. Having edits reviewed is part of the learning process. If you block 1k- users, you're only delaying that learning. Commented May 23, 2023 at 23:26
  • 2
    @KevinB It's not like we would stop improving closed questions. The proposal is to drop closed questions from the suggested edits queue. Editors with 2k+ rep would obviously still be able to edit closed questions into shape; the "contributions" I tend to find in the review queue are typically small cosmetic edits to unsalvageable questions which should definitely remain closed (often apparently submitted before the question was closed, but failing to address the fundamental problems). The OP would obviously still be able to accept those edits just by clicking approve.
    – tripleee
    Commented May 24, 2023 at 3:54
  • 2
    @Andreasdetestscensorship I don't disagree, but we also have to do something about the untenable size of the queue. The (half tongue-in-cheek) proposal is to raise the bar as long as the queues are not being processed fast enough, and reset to normal after that. Your comment actually provides a hint as to why I think this would be useful; the queues are full of stumbling beginners who are in the middle of learning the ropes, but not contributing much of value.
    – tripleee
    Commented May 24, 2023 at 3:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .