15

Since June 2017, the size of the suggested edits queue has been 500, meaning that once there are 500 suggested edits from all users, no one can suggest edits until at least one of them completes review.

I tried to suggest an edit today, but received an error message that the queue is full. However, the review page says that there are only 442 edits pending as of the time this question was posted, fewer than 500.

Was the size of the queue reduced today? If so, is it intentional, or a bug, and if the former, what is its current size supposed to be? (The post I linked says that the change to 500 was an experiment that was supposed to be reverted at some point, but I half-suspect it may be unintentional since it persisted for nearly four years. The previous limit was 200, for context.)

8
  • 5
    The issue you experienced is probably due to caching.
    – TylerH
    Commented Mar 17, 2021 at 19:53
  • 3
    @TylerH Nope, I refreshed the review page and the count updated to 427. Yet I'm still unable to suggest edits because the queue is full.
    – gparyani
    Commented Mar 17, 2021 at 19:53
  • 2
    IIRC, the number of pending reviews is fuzzed a bit so it isn't an exact count. Commented Mar 17, 2021 at 19:57
  • 15
    @gparyani The cache is server-side too.
    – TylerH
    Commented Mar 17, 2021 at 19:59
  • Did you hard refresh or normal refresh, @gparyani? I don't think even a hard refresh can fix caching... but I'm not an expert in these matters. AFAIK, caching is server based and refreshing doesn't help much.
    – 10 Rep
    Commented Mar 17, 2021 at 19:59
  • 1
    informational SEDE query: data.stackexchange.com/stackoverflow/query/…
    – rene
    Commented Mar 17, 2021 at 21:06
  • @rene the number of suggested edits in that graph is often more than 500!
    – Nike
    Commented Feb 19, 2023 at 21:54
  • However big it may be at the moment, it seems to be too small, every time, every few months, i suggest an edit, it says it was full. And for obvious reasons, it stays that way for a while. Commented Oct 31, 2023 at 12:46

3 Answers 3

22

The limit is still 500 - I currently see there being 490, which is pretty darn close.

Here's the site setting, which hasn't been touched since 2017 when Shog set it to 500 -

screenshot of the site setting for max items in the suggested edits review queue on SO

Here's what I see in the queue currently:

Screenshot of the review queue list showing 490 items in the suggested edits review queue

Even if we had changed it, lowering it to ~450 seems a bit odd? More and more it seems like people are not able to submit reviews since this queue is full.

I'm curious if there are reasons that this queue is full so often - I haven't looked at any data about it but it definitely seems like something we need to address - and not by changing the limit of suggested edits. If few people are reviewing these, then we need to find ways to get people to review more regularly. I find suggested edits are generally fairly simple reviews, though I'm not generally on SO - which I could understand them being more complicated to review.

30
  • 11
    I expect the new review user interface is not helping making people want to review. The big radio-button box is taking up 1/3 of the screen leaving a very narrow view screen for a side-to-side view of a post. This means one has to scroll all the time. When there's code in a post it's even worse. To be able to see the top right side of a code block, you have to scroll down to the bottom of the block first. Then scroll right. Then revert the scroll to see the nest bits.. and so it goes. I personally just clicked the X in on the top right of the window, and am now waiting for a better UI :)
    – Scratte
    Commented Mar 17, 2021 at 20:18
  • 1
    @gparyani No clue. I saw 489 for a while and then it changed to 490. I'd also point out that I'm not a 2k user. ;)
    – Catija Staff
    Commented Mar 17, 2021 at 20:22
  • 3
    @Scratte I do understand that - I know the side-by-side is a point of friction for some people, particularly on longer posts, where more scrolling may be necessary.
    – Catija Staff
    Commented Mar 17, 2021 at 20:24
  • 1
    @zcoop98 I would like to see what data we collect - I've asked internally whether we collect data about how many suggested edit attempts get blocked and how often the queue is full - hopefully that will help us get some insight. We can probably look at the top reviewers in that queue from before the change and talk to them about their experience, particularly if their numbers are a big drop from where they were prior to the change.
    – Catija Staff
    Commented Mar 17, 2021 at 20:26
  • 6
    @Catija I don't mind side-by-side at all. But I mind that 1/3 of the screen is being used for something that is unrelated to the post I'm reviewing. The new user interface has made is unnecessarily hard to review suggested edits. Try it yourself.. remember you have to check if any changes was made to code.
    – Scratte
    Commented Mar 17, 2021 at 20:27
  • 4
    @Scratte That's a fair distinction - it's less the side-by-side and more how narrow each section is. And it's the new right sidebar in review that exacerbates that issue, right? Essentially, the red/orange sections in this image - i.sstatic.net/esDc3.png)
    – Catija Staff
    Commented Mar 17, 2021 at 21:36
  • 3
    FWIW, suggested edits depend on a particularly sensitive bit of logic that tries to ensure no two people review the same edit at the same time. When that works, it should increase throughput by minimizing wasted reviews - but I've seen issues in the past where scores or hundreds of tasks will end up "locked" even after their reviewers have reviewed. A lot has probably changed with y'all's Redis configuration over the past year, so I don't want to speculate - but worth checking out.
    – Shog9
    Commented Mar 17, 2021 at 21:51
  • 2
    @Catija I hadn't noticed the red sections. I have a small screen. This is a screen shot of my full window. I've jokingly said that it's a waste of time reviewing posts 4 words per line. I can see that it's actually 8 :O For me, the orange section with the radio-button box has to go and leave the spot for the review. I can do it using my add blocker, but then I can't submit my verdict :D I was hoping someone else would be equally dissatisfied and created a user script that gave me the buttons back. No such luck so far.. perhaps they're waiting too.
    – Scratte
    Commented Mar 17, 2021 at 22:11
  • 3
    @Shog9 I just checked on our ActiveReviewItems cache (used on all queues, not just Suggested Edits). Looks like things are healthy - we have on average about 10 items being expired from there for Suggested Edits every 5 minute sync. So doesn't appear to be a big pileup there, at least nothing that would explain this.
    – Yaakov Ellis StaffMod
    Commented Mar 18, 2021 at 13:30
  • 1
    Right on, @Yaakov - that just leaves the input vs output to monitor! Should be some stuff from Jon around for that purpose.
    – Shog9
    Commented Mar 18, 2021 at 13:42
  • 1
    @TylerH it is not particularly costly on the backend. It is more: try to ensure that items that reach the queue are reviewed in a timely fashion.
    – Yaakov Ellis StaffMod
    Commented Mar 18, 2021 at 13:43
  • 1
    The problem is that the backlog itself changes behavior, @tyler - if folks have to wait on pending edits, it holds up other work.
    – Shog9
    Commented Mar 18, 2021 at 13:43
  • 6
    @TylerH that definitely touches on something that I've been thinking a ton about - how privileges are gate-kept by reputation... there's definitely people who have submitted many edits and had most of them approved but just haven't hit the 2k rep... which essentially means we're creating this issue to some degree. I'd love to see a future where privilege is more directly tied to experience with a tool, not pure reputation. So, users with <2K maybe get full edit privileges because they have >90% approved and a minimum of 50 suggestions or something like that.
    – Catija Staff
    Commented Mar 18, 2021 at 13:45
  • 2
    As you say, it's hard to know how often someone sees a post that needs an edit but has a pending edit and just... Walks away, @TylerH. However, it is possible to gauge the effect by looking at how edit submission rates vary with queue size. I suspect that if there was no hard limit for the queue we'd find some natural point at which edits stalled, similar to what is seen for bounties; but unlike bounties, suggested edits don't "time out" - so in practice that point would tend to gradually increase over time anyway, even as the rate of new edits continued to decrease.
    – Shog9
    Commented Mar 18, 2021 at 18:56
  • 3
    ...in any case, raising or increasing the limit doesn't fix anything. 500 was supposed to be a temporary measure while we fixed the massive decrease in reviewing triggered by the top-bar redesign. Obviously, that didn't really happen, not to the extent that was needed. Now the new limit is insufficient. Maybe 1K would buy a bit more time, but... Eventually it too will be insufficient unless the underlying issue is addressed.
    – Shog9
    Commented Mar 18, 2021 at 18:59
18

The number that is shown to you (in this case: 442) is the number of items in the queue that are not in the list of items being actively reviewed at the moment (items are inserted there when loaded by a user, and removed when the user finishes their review, or they expire after 3 minutes and are then removed in an every-5-minute sync). So when you saw 442, the actual number of items in the queue (including those being reviewed) was definitely higher. The diff between actual numbers and the number you will see will vary over time (based on how many people are currently reviewing).

As far as enforcing the limit on max items per queue (currently 500), current logic is: there is a query that runs once a minute checking on the number of items in the queue. This number is cached for 500 seconds (8.33 minutes). If this number is over the limit, then all suggested edits will be rejected until the next time that this number is refreshed.

So in a situation where queue was over the limit, and you submitted your edit close to the cache expiration (let's say: 7 minutes after it was calculated) - the real number of items in the queue that moment could definitely have been lower than 500. That plus the number of active reviews is how you could get the "queue is full" message at the same time as seeing 442 items in the queue.

1
  • 4
    Thanks for a detailed explanation, but why don't you show the cached number for <2k users?
    – EvgenKo423
    Commented Apr 29, 2021 at 15:51
5

How about we lower the amount of reputation needed? As every time when I start reviewing the groups I am allowed to review, I eventually get blocked by this issue. As a starting reviewer, this blockade really brings down my motivation to help improve the platform.

Reducing the amount of reputation required, would simply enable more people to work on the queue.

To make things clear, I "only" have ~615 reputation and don't suggest lowering it that low. But maybe it makes sense to switch it with create tags (1500 reputation) as in my opinion we require more reviewers, and not that many new tags.

2
  • The would require something else to change as well. 1500 rep users can't edit posts without their edits being reviewed. Do we just lower the rep that editing privileges are given at? Do we give them only two options in review (accept/reject), like in lower rep tag wiki reviews?
    – Laurel
    Commented Dec 21, 2021 at 19:21
  • My suggestion was to simply lower the 2000 to 1500. So everything that now is limited to 2000, will be limited to 1500. It could be tried with your suggestion, though I don't know how effective that would be for what we try to achieve here. Commented Dec 25, 2021 at 16:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .