33

I just want somebody to explain why "Share feedback" is an audit fail for this review on a "known good" post.

I believe that sharing feedback does not mean the post was bad, it is just a suggestion on how to improve it. In that very case, I'd leave a comment like adding a bit more information or explanation, although for the author seems obvious, for others who are trying to help it may seem ambiguous or mislead to another interpretation.

I understand the purpose of the audit is to make sure the reviewers are actually reviewing. However, this ignores the fact the people may have different opinions and points of view. That's why I humbly think the audit can't be opinion-based.

17
  • 4
    To be honest, I think the question is quite clear and the reason you failed (as I think you understood already) is that sharing feedback is usually for "bad" posts, while this one is considered "good", having 22 upvotes and 12 answers
    – Tomerikoo
    Commented Oct 14, 2021 at 14:28
  • 1
    It used to be that trying to write a comment would fail the audit, but that's since been removed. The "Share Feedback" option is to end the review by that action. The action is the wrong choice if the post is fine. Then the correct action is "Looks OK". Think of the audit-reviews as the gate-control :) It either passes or it doesn't :)
    – Scratte
    Commented Oct 14, 2021 at 14:34
  • 1
    "that's why I humbly think the audit can't be opinion-based." - The audit isn't opinion based but you still selected the wrong option, there was nothing wrong with the question, the correct action was "Looks Ok". You can always open the question in another tab to leave feedback that question should be clarified. Commented Oct 14, 2021 at 14:38
  • 3
    You may want to include in your question which of these options you were going to choose if the task wasn't an audit.
    – yivi
    Commented Oct 14, 2021 at 14:41
  • 1
    We're missing info here; did you add a comment before selecting "Leave Feedback"? Or was selecting the option your only action?
    – zcoop98
    Commented Oct 14, 2021 at 16:28
  • 4
    @zcoop98 It shows options after choosing "Share Feedback". There would a "custom comment" but since that was an audit, I didn't have the chance, was failed and suspended from reviewing right away. Commented Oct 14, 2021 at 16:36
  • 3
    Gotcha. Yeah... I'm of the mind that "Share feedback" either shouldn't fail a good audit outright, or that it should let you make a choice in the secondary modal, and take that into account. The current setup clearly has the nasty side-effect of failing good-faith reviewers for bad reasons.
    – zcoop98
    Commented Oct 14, 2021 at 16:50
  • Out of curiosity, what feedback did you plan to share with the question author? When I read the question originally and again just now, it seems crystal clear. (Disclaimer: I upvoted the question and two answers back when it was posted) Commented Oct 14, 2021 at 17:42
  • 1
    This is not a duplicate of Review audit failed after just clicking "add comment". Since March 2021, clicking "add comment" is allowed. Clicking "Share feedback" will still cause an audit fail. Commented Oct 14, 2021 at 18:56
  • @StephenOstermiller it surely is not, I voted to reopen it. Commented Oct 14, 2021 at 20:26
  • 1
    @Fabio - If you were suspended from reviews that’s an indication of failing multiple past reviews in the past. Audits are obvious, they are designed to be passed, failing enough to be reviewed banned indicated you should slow down and view the entire question in another tab Commented Oct 15, 2021 at 3:55
  • 1
    @IanCampbell When I read the entire question and saw most of as code and desired result, I thought the author could share the attempt, but this is because similar questions receive comments like "SO is not a code ordering service", some of them had even falled into bad post audits which I passed. But by reading it again now seeing the answers and the rating, okay I admit it was clear and good. It all depends on how the first see, the so-called standing ovation, the first raters rate positive, it'll hold possibly high score, and the other way around as well. Commented Oct 15, 2021 at 9:52
  • 5
    I absolutely agree with Fabio Mendes Soares that "Share feedback" on first questions review should be usable to give suggestions for improvement even when the question is legit. I just failed an audit on a "good" question (22 upvotes) that had its error message posted as a screenshot instead of text. That's something we don't want, so I wanted to give feedback about that. Commented Nov 11, 2021 at 16:31
  • 1
    I don't know why this is listed as 'Status Review' when this was already 'fixed' previously.
    – ouflak
    Commented Apr 25, 2022 at 10:52
  • 1
    @ouflak because the review system was recently overhauled, which broke it again.
    – Ryan M Mod
    Commented Apr 25, 2022 at 15:57

4 Answers 4

33

Known-good audits should only be failed by picking a canned comment

In my opinion, sharing custom-written feedback should be seen as the same as writing a comment: neither a positive nor negative action.

However, sharing any of the pre-written feedbacks would be an unambiguously negative action, and would properly fail the audit in this case (the post is fine, and doesn't need improvement).

I'm not sure at what point in the process it's triggering the failure (I didn't want to fail an audit to test this), but at least in a known-bad audit, the audit is passed as soon as you pick "Share feedback." For known-good audits, a custom comment could be either positive or negative—the system should pass a user who picks this option, as the old First Posts queue did.

1
  • 3
    Similar posts containing mostly an objective description of the desired result without a code attempting to solve have received comments or been flagged, and by seeing the rating of this one, I am being led to the conclusion that whether being good or bad is mere matter of opinion of those who rate first. It is statistically incredible that anyone who has reviewed more than 1000 posts would not fail audits atleast once just because of this tricky nuance of good-bad decision boundary. An AI algorithm trying to learn that would probably swing from one state to another. Commented Oct 15, 2021 at 10:03
9

OK, as far as I understood from you, if the post is positive or the community consensus is that the post is OK, then the right action is "Looks OK".

I personally hate that "right/wrong" or "good/bad" decision boundary, because not everything seems so binary as anybody can see. Anyway, the feedback is noted, and I will be careful in the next reviews.

10
  • 5
    "because not everything seems so binary as anybody can see" - that's why it's a community effort. It is not one person that decides that something "looks ok".
    – Gimby
    Commented Oct 14, 2021 at 15:13
  • 4
    My point is that one must match the opinion of the majority, including on issues that may lead to multiple interpretations. Similar posts that seemed clear and obvious for me were considered bad by the community. The message carried by this mechanism drives the members' behavior towards fitting in what the community judges. Commented Oct 14, 2021 at 15:34
  • 5
    @FabioMendesSoares You nailed it right there. Reviewing is much like predicting the future.. or in the case of audits investigating the past. There's no requirement that you review at all.
    – Scratte
    Commented Oct 14, 2021 at 16:49
  • It's also much like a video game. In the 80's/90's there were far too many games where you got hit once and then immediately lost a life. "Single hit death" games I call them, and boy do I think they're trash. Over time, most games have become more forgiving and gave you hearts or health bars so you could take multiple hits. The reviewing system is just that. Sometimes you take a hit, sometimes unfairly too. Just like a gamer you need to be able to shrug that off and keep playing... because if every hit makes you feel anger or frustration, you're playing games for the wrong reason.
    – Gimby
    Commented Oct 15, 2021 at 13:26
  • 4
    @Gimby I liked your analogy, but this is not a game. Indeed real world rules are not perfect and people get punished for unfair or at least not proportional actions. The fact is, those who review more are more likely to be caught in audits, and there are audits which the criteria used is purely opinion-based! We live in a diverse world, I may disagree with 95% of the people, and that still won't make me a bad person or something like that. If 5% is something the system wants to avoid, then ok, it is doing the right way, but is it? Purely binary threshold ends up in life/death boundaries. Commented Oct 15, 2021 at 19:57
  • 2
    @FabioMendesSoares: Audit reviews are supposed to be chosen to be fairly unambiguous, far from the boundary where reasonable people are likely to disagree about whether it needs closure / deletion. (Reasonable people can still downvote). When that doesn't happen, a meta post will get mods to fix it and unban. I think you're conflating "people might disagree on whether a post is good" with "stupid audit UI assumes you think it's bad if you click share feedback". I think everyone agrees that's an unfortunate or non-obvious design, and unless it's fixed, something you'll have to remember. Commented Oct 17, 2021 at 1:41
  • 1
    For everything other than audits, yes, it's ok if you disagree with people sometimes in the gray areas. As long as you agree on the major points, like that posts which barely or don't answer the question, and mainly exist to send traffic to some ad-supported website, are bad posts. (But that would be a case of disagreeing with 99% of the community, not 95%) Commented Oct 17, 2021 at 1:44
  • 3
    (It's not rare to see a meta post about a failed audit where they picked the right answer, but the audit was wrong, e.g. with a bad question used as a good-audit test, often because it was old and maybe popular but not well-written. Like a few times a year make it to the hot meta sidebar, probably more that don't. So if you're worried about the gray area where reasonable people disagree, that's something that mods can fix on a case by case basis if a bad audit catches you. Like I said, audits are supposed to be unambiguous.) Commented Oct 17, 2021 at 1:49
  • 1
    @FabioMendesSoares strong disagreement that Stack Overflow has real world rules, I will reject the notion that anything digital is real life. It is merely a reflection of real life with it's own unique set of rules and standards... just like games tend to be. I find Stack Overflow and especially reviewing to be more like video games than anything else. You have rules to follow, you have hitpoints, you will rage from time to time, you spend ammunition, you shoot at things, etc. etc.
    – Gimby
    Commented Oct 19, 2021 at 9:00
  • @Gimby okay that is your point of view, I am not supposed to see the same way you do. My point is that the purpose of this platform is to help people solve their problems (including real world ones), and reviewing improves quality, flags and filters inappropriate content. As long as you are using this ( and any) system, you must follow rules irrespective of agreeing with them or not. In video games you can kill people, that's part of the fun, but here abuses are punished because Stack Overflow wants to keep being a top source of troubleshooting. Commented Oct 19, 2021 at 10:19
5

"Share feedback" is shorthand for "Share feedback that this post should be closed." If you are fairly new to using the tool, I can imagine that you might expect "share feedback" to be used to share good or bad feedback like "add comment." Unless you think the question is so bad that it should be closed, you wouldn't want to use "share feedback."

The system puts some questions into the queue that have already been processed and for which the correct decision should be clear. The reasoning is that a question with 10s of up-votes won't need the feedback that it should be closed. If you try to close such a question, you must not be paying attention and you fail the audit.

The consequences of failing an occasional audit are less severe than they used to be. It used to kick you out of the queue right away and make you sit out of reviewing for a while. Now occasional failures are tolerated and you can keep on reviewing having hopefully learned from your mistake.

In addition to "Looks OK" you are also allowed to up-vote or comment on the post and choose "Other action." However both the vote and the comment would be discarded when you pass. I think that edits are allowed too, but I haven't tried that recently.

1
  • 14
    I failed an audit on a "new questions queue" for wanting to share feedback that screenshots of error messages are not OK. That is not a mistake, it's a legitimate feedback. Neither the how-to for the queue nor the wording "share feedback" make it apparent that it should mean "Share feedback that this post should be closed." Commented Nov 11, 2021 at 16:36
5

Having just run into a similar problem with audit, which feels like a pretty hostile system, I have a few things that seem like they could get it working better.

  • The audit could let you know what the "right" answer was - it just says "read and understand" in the message, but I have had audits that I failed and upon re-reading I did not in fact understand why I was wrong, and I'd content I wasn't.
  • Certainly for First Questions if any kind of feedback is intended to be equivalent to a close flag then the feedback option should be re-labeled because I had been using it to provide feedback. The reason I had been doing that is because it was labelled "feedback" and offered me options for giving "feedback" so I feel my response was quite logical. "Feedback (but actually secretly it's a close flag)" - which appears to be the consensus for how the feedback option works - is a weird structure. Don't force people who are already going out of their way to review things to also second-guess the UI.
  • In terms of audit questions, the one that I ran into seemed to have caught a fair few other people out as well. I would argue that if a question seems to be catching out disproportionately many people who it is given to, maybe that question should be removed from the audit list.
4
  • 5
    "audit, which feels like a pretty hostile system" => yep, completely agree, and often feels like "Russian Roulette" to me also..., even when I already "know" it's an Audit (I always open Qt's/Tasks in a separate Tab), I'm still not always sure what the "expected" Correct Action is... (Result: I don't (or very rarely) help for Reviews anymore, even if afaik I never failed an Audit, but it feels too "scary"...)
    – chivracq
    Commented Apr 14, 2022 at 12:39
  • 4
    I agree on the "Russian roulette" thing - and I have seen it suggested that the correct response is just to skip obvious audit questions, but if people working their way through a queue in good faith can't pass audit successfully, then either you aren't explaining the queue properly or there's a problem with the audit system.
    – glenatron
    Commented Apr 14, 2022 at 14:17
  • 1
    Totally agree, I click "share feedback" on a code-only answer... aiming to say "you should include supporting information" but I'm suspended... This seems to be not only "Russian Roulette" but also illogical and plain wrong. Commented May 20, 2023 at 10:28
  • @AndyPreston that was this review of an answer that seems to have an explanation that it's for Docker users and updates the Stretch repositories. It could maybe use more explanation, but it's hardly code-only, and the explanation would likely be duplicative of what's in the other answers.
    – Ryan M Mod
    Commented May 20, 2023 at 11:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .