27

The purpose of the audit system is to help new reviewers hone their moderation skills, and make sure everyone is paying attention.

What is the purpose of this test?

The test (known as a review audit) is designed to help new reviewers hone their moderation skills, while nudging more experienced users who don't seem to be paying close attention to what they're reviewing. ...

- What are review tests (audits) and how do they work? (answer by Tim Post)

In my opinion, the vast majority of people who are going to attempt to leave a comment on a review task are paying at least some amount of attention to what they're doing.

It is possible to find useful questions in the review queues and leave helpful comments where the commenting feature is enabled. Comments on a post don't necessarily have to be explaining a deficiency with the question or asking for clarification, but they pretty much always mean that the reviewer is paying attention.

Because the purpose of audits is to make sure reviewers are paying attention, and an attempt to post a comment is a strong indicator that the reviewer is paying attention:

I propose that an attempt to comment be counted as a pass regardless of whether the audit is known-good or known-bad.

Related:

0

1 Answer 1

17

This is good suggestion. Indeed adding a comment can be a valid response both to positive and negative evaluations.

One problem though is that if you pass the audit just by clicking the "add comment" button, without actually writing the comment, a malicious reviewer could keep pressing the button for all reviews, and never fail any audits.

A way to overcome this would be if the audit passed only after the comment had been posted. This way it wouldn't be abused so easily, because the fake comments would be easy to spot.

3
  • 11
    "malicious reviewer could keep pressing the button for all reviews" FWIW, you can already cheat on the close vote queue by trying to edit the question every time. Audits are not that good at finding malicious users (are there that many?), at least not better than moderators and flagging when suspicious behaviour is spotted.
    – E_net4
    Dec 17, 2018 at 8:59
  • 15
    I agree with @E_net4. The audit system isn't useful for handling malicious reviewers (there is an isAudit boolean property in the JSON response for each review task, so a truly malicious user could trivially write a userscript to automatically skip audits, or select the expected outcome based on the true score of the post and automatically pass all audits). There are other systems in place to handle such users. I don't think the extra complication here for the sake of some reviewers the system isn't meant to catch is worth it.
    – user4639281
    Dec 17, 2018 at 10:07
  • 4
    as tiny mentions, it's absolutely unnecessary to attempt to comment on every post in case it's an audit. There are Proof of concept userscripts out right now that display a big red "THIS IS AN AUDIT" bar on review when it's an audit.
    – Magisch
    Dec 18, 2018 at 12:05

You must log in to answer this question.