21

Today I was reviewing a question on Stack Overflow. The OP was saying that he couldn't find the documentation about a keyword and was pointing to a certain empty link. I did a brief search for it and I found this link that had all the info he needed. I tried to leave the web address in a comment while reviewing, and I got this as thank you:

stackoverflow audit outcome

I paid close attention to the situation. IMO, this is an evident blunder and should be corrected. I don't believe that a reviewer directing someone to a resource should be counted as indicating the post has a problem.

1

1 Answer 1

7

This highlights one of the big flaws of the First Posts queue: it allows reviewers to take almost any action. Some of those actions are fairly well-defined: you should upvote good posts, downvote bad ones, close inappropriate ones...

Other actions are much more open-ended: one might edit to fix a trivial typo, or to repair an all-but-unreadable post; one might comment to critique, to suggest improvements, to request clarification, or to provide tangential information.

For audits, the system must try to guess at which actions are appropriate for a given post. Comments are interpreted as corrective, though they may not always be. To avoid issues, I would recommend only commenting in review if your intention is to provide guidance or correction to an author; for all other purposes, click the "link" in the sidebar to take you to the normal question page.

Failing this audit will not count against your record in review.

17
  • ciao Shog, thanks for this in the meanwhile. Actually finding the link while reviewing is something that I do when I want to answer to the question that I am reviewing. I think that the process that you are suggesting is the only one possible, but not user friendly. Also, most probably if it was not an audit I would have been able to use the comments section properly. There is a problem in the dynamics and I understand the difficulties in solving it in a proper way. Have a good day
    – Antonino
    Dec 15, 2018 at 3:00
  • 13
    "...For audits, the system must try to guess...": if the system is only guessing, that means the system is not 100% sure. So instead of assuming the reviewer did something wrong, how about limiting review posts to those where the system does not have to guess? Dec 15, 2018 at 8:46
  • 1
    As the system has to guess and sometimes gets it wrong, that appears to be a point in favour of having an "I do not understand" or an "I do not agree" button on the audit failure message.
    – AdrianHHH
    Dec 15, 2018 at 13:07
  • 3
    No, it doesn't. It highlights how messed up the audit system is.
    – jpmc26
    Dec 16, 2018 at 4:40
  • 7
    I'm sorry, but I don't agree with this answer. Surely the fact that you're describing a workaround to a system meant to check whether people are reviewing correctly that amounts to "go out of your way not to use all of the features of the review queue to avoid the faulty test", it suggests that the faulty test should be avoided for us.
    – wizzwizz4
    Dec 16, 2018 at 7:50
  • 5
    I wonder what bad would happen if system granted pass to user who commented the audit (known good or bad one doesn't really matter). "Attempt to comment demonstrates that you pay attention while reviewing. Congratulations, you passed this audit."
    – gnat
    Dec 16, 2018 at 20:23
  • 13
    I agree with @gnat. An attempt to comment will mean that the user is paying attention, except maybe in a situation where someone just blindly leaves pointless comments on all the questions they review, but I think that would be caught by other systems regardless. An attempt to comment should result in a pass regardless of whether the audit is known-good or known-bad.
    – user4639281
    Dec 16, 2018 at 23:27
  • 5
    Post a feature-request for that please, @Tiny
    – Shog9
    Dec 16, 2018 at 23:39
  • 5
    no point in posting FR @Tiny - this feature request is exactly about this and it hangs ignored for over 4 years despite 250+ upvotes
    – gnat
    Dec 17, 2018 at 4:47
  • I didn't ignore it, @gnat. Look waaaaay down at the bottom of the answers section.
    – Shog9
    Dec 17, 2018 at 4:51
  • 4
    no status tag = status-ignored
    – gnat
    Dec 17, 2018 at 5:19
  • No status tag + answer == I'm not convinced, but willing to listen to better ideas.
    – Shog9
    Dec 17, 2018 at 15:16
  • 1
    @Shog9 I saw the reply, but I'm unsure what you aren't convinced of given the fact that this post talks about legitimate uses of comments. Do you think Antonino's use of comment here is wrong? Or do you think that there do exist legimite cases like Antoninos but they aren't enough to warrant changing the system? Because your actual thesis was that comments shouldn't be accounted for at all, you should be comments should be worth saying and that we should just get rid of the whole system. None of that is an argument against changing the impact of commenting.
    – Krupip
    Dec 17, 2018 at 22:27
  • Everyone hates failing audits, @opa. And that's what the feature-request there boiled down to: "I don't think I should fail because of a comment that might've been intended to be a helpful note". Problem is, there's no way for the system to know what the reviewer intended. That's why I said, I don't think comments should count at all - that's what we do in other queues, after all: you can comment in Low Quality, Close, Reopen, Triage and it won't fail a review or pass it, nor will it complete any other review; it's just an auxiliary thing.
    – Shog9
    Dec 17, 2018 at 22:34
  • 1
    "I don't think I should've failed an audit" isn't much of a feature-request, @opa. If someone wants to explicitly propose that comments in First Posts / Late Answers don't count as reviews, I'll support that. But, uh... I can't exactly go and implement such a change based on the overwhelming success of my own answer. I probably should've done a better job of selling it, but I was really hoping Triage would make the whole thing obsolete before it mattered. So it goes.
    – Shog9
    Dec 17, 2018 at 22:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .