6

I've been doing reviews for the past 20 minutes or so and was surprised by how many audits I saw, it was 3 over about 15 reviews. I'm currently sat at 203 reviews completed and have failed 1 audit when I didn't fully read the question(stupid mistake on my part), but that was some time ago and I've not noticed any increase in audits until now.

After reading When, if ever, will I stop getting review audits? it does appear that when audits appear is random however less should be served when large numbers of reviews, and hence audits have been done successfully.

Was it just blind chance that resulted in me having a large number of audits in a short period of time or have I done something, such as reviewing too quickly, which has lead to it?

4
  • 1
    "large numbers of reviews have been done" no, a large number review audits has been passed.
    – Braiam
    Commented Feb 12, 2017 at 14:11
  • I have done a reasonable number of reviews. I get 2 audits every 50 reviews on average but when I fail one I'm in the same boat as you ..
    – rene
    Commented Feb 12, 2017 at 14:38
  • 2
    Looks like you did 51 reviews, not 15. Different queues too, so getting 3 audits isn't particularly surprising. Reviewers are a bit liable to turn review into a grind, racing through them and only to be completed when the queue is empty. The kind of approach that forced audits to be added. Tackling, oh, 10 at a time is a healthy approach in my book. Well, if you pass the audits then you'll get away with it. Commented Feb 12, 2017 at 15:06
  • @HansPassant Oh, really? that is quite surprising, just goes to show how much all of the reviews just blend in to each other when done in quick succession, I'll take note of that and take it a bit slower in future. Or is that how many reviews I had done over the whole day at the time, I'm asking about the period of time between my first audit and my third as having done 50+ reviews over the whole day I would expect to see 3+ audits, but not expecting all audits to be over a short space of that time Commented Feb 12, 2017 at 15:08

1 Answer 1

8

less should be served when large numbers of reviews, and hence audits have been done successfully.

The problem is, completing a non-audit review does not mean that you made the correct decision. The system has no way of knowing whether your decision was correct or not unless it's an audit. (And even then, audits aren't perfect…*grumble, grumble*.)

Therefore, audits are sporadically inserted into the review queue as you go through it. This is designed to catch people who are doing what we call "robo-reviewing"—basically, not paying attention and therefore making the wrong decision. An incorrect review is worse than no review at all, so we want to catch people who are going through the review queue with their brain turned off and potentially making incorrect decisions.

Audits are not necessarily "training wheels" so much as they are "speed bumps" or "road blocks". Thus, passing (or failing) one of them doesn't indicate much about your competence as a reviewer, so we can't simply take the fact that you've passed x number of them as evidence that you're a good reviewer and therefore don't need them anymore. At any time, the high volume of garbage in the review queues is liable to have turned your brain into mush. An audit will catch this, and either wake you up or cause you to stop reviewing.

I don't have any details on how exactly audits are generated, other than they appear randomly, without respect to your reputation or anything else. Thus, I assume it doesn't matter how many things you've reviewed recently, other than having done more reviews increases your chances of seeing an audit, statistically speaking.

Rene says that, on average, you should expect 2 audits every 50 reviews. I assume he's basing this on empirical evidence, having done lots of reviews.

4
  • On average you should expect 2 audits every 50 reviews.
    – rene
    Commented Feb 12, 2017 at 14:39
  • @rene This is interesting information, although I don't feel like it's particularly at the number of reviews that I've done, although as Cody says, with your number of reviews it would be unsurprising to have only 2 audits over 50 reviews over my 20 or so over 200. Commented Feb 12, 2017 at 14:45
  • for new reviewers and just after an failed audit the rate drops to 1 every 10 or so.
    – rene
    Commented Feb 12, 2017 at 14:51
  • The general average so far today is just under 7% of reviews in triage have been audits... So @rene's figures are certainly in the right ballpark.
    – Jon Clements Mod
    Commented Feb 12, 2017 at 14:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .