41

Edit: Reading the comments it's clear that some folks see this as a high-rep user whining. I don't really care about getting the suspension lifted. It's just for two days.

But maybe that second post shouldn't be used for review audits moving forward.

And I do think the discussion around poor quality review audits is important. Should "leave feedback" lead to a failed audit on a "good" post? Should there be a way to report bad audits? Should the failure ratio count instead of just the failure count?

Maybe I need to turn these into discussion / feature requests posts on m.se.


I have just been suspended from reviewing based on what I believe to be poor review audits:

Screenshot of "you've been suspended" message

On the first one

This is clearly a bad answer and I don't remember exactly how I failed that audit. It was a while ago, and the only information I can see now is that I "reviewed" it.

I believe I downvoted the post, upvoted the comment that highlights part of the problem, forgot to flag, and then used the "performed another action" option. Since votes aren't actually kept on audit questions I can't look for orange buttons to be sure.

In and of itself, it's frustrating that this leads to a failed audit. Yes, a VLQ flag would have been warranted but the actions I took both signaled in the right direction.

Is forgetting to flag really worth failing an audit over? Maybe.

On the second one

I used the "share feedback" option, intending to use the canned comment that asks for supporting information.

The entire text of that answer is:

For those who do have this problem using Visual Studio 2022: switch back to 2019 as it doesn't work yet in the new version.

I don't think this is a very good answer. A link to a bug report, documentation, etc. would go a long way towards improving it. I don't believe this answer should be used as a review audit in its current form.

Other stuff

I'm curious about what triggers a review suspension. Does the ratio of failed audits to passed audits or overall reviews factor in at all?

I'm quite active in the review queues, having done something like 400 reviews since that first failed review two weeks ago on the 28th. But it looks like a person who fails two audits out of 400 reviews is treated the same as somebody who fails two audits out of two reviews.

Should it work that way? Maybe. But as somebody who spends a lot of time doing housekeeping on this site, I can say failing audits like that second one (and especially being suspended¹ over it) is very discouraging. I'm about ready to stop contributing.

(I would really like a way to flag bad audits that doesn't require me to write an entire blog post like this, but that's a matter for Meta Stack Exchange.)

(Please excuse any errors referencing text on the review modals. I can't see them at the moment due to my suspension.)


¹Full disclosure: I was previously suspended for failing review audits in the first questions queue. That suspension was quickly lifted after the audits were deemed to be poor.

21
  • 4
    The second case is similar to this report: Failed first question review audit after choosing sharing feedback? ... sharing feedback is "not allowed" on "known good posts". Pretty bad design decision, but this is how the system currently works.
    – Tom
    Commented Dec 12, 2021 at 15:09
  • 9
    It’s worth noting that, according to the answer history, the second audit has been served up four times, with two of those (including yours) being failed due to ”Leave Feedback”. I myself got this review yesterday, didn’t recognize it as an audit, but lucked out by choosing “Skip”. The “Leave feedback” issue is especially pernicious since even unambiguously useful posts sometimes merit feedback, and I usually post custom comments through the “Leave feedback” dialogue so that my review history is less ambiguous. Commented Dec 12, 2021 at 19:25
  • 24
    I’d really like the answer to this: “…it looks like a person who fails two audits out of 400 reviews is treated the same as somebody who fails two audits out of two reviews”. I received a suspension myself six weeks ago, and I suspect it fell into a similar scenario. I typically review 60-80 posts a day, and receive (and pass) maybe 2 or 3 audits each day. But I received a suspension for two failed audits 50 days apart. At minimum, it should take into account number of audits passed, if it doesn’t already, and especially given that there is a level of judgment expected in these queues. Commented Dec 12, 2021 at 20:19
  • 5
    @KINYUATIMOTHYNJIRU: Technically, there’s a difference between moderators and reviewers. Both are volunteer positions, but moderators are elected, whereas anyone (with enough reputation) can be a reviewer. That’s also independent from downvoting, which any contributor with more than 125 reputation can do. You are able to downvote. With 200 more reputation, you’ll be able to participate in some of the review queues as well. Commented Dec 13, 2021 at 4:38
  • 2
    @KINYUATIMOTHYNJIRU, I also said that I upvoted an existing comment explaining why the answer is poor. What is more discouraging, one comment a few upvotes or several similar comments piling on? In any case, I didn't fail that review audit because of the downvote. Voting (both up and down) is an important part of how SO works: "Downvote answers that are incorrect or don't provide sufficient information to be useful in answering the question." That answer suggested something that OP had already tried and said as much in the question.
    – Chris
    Commented Dec 13, 2021 at 12:02
  • 19
    The discussion on poor quality review audits has been going on for nearly a decade now. The system is awful... and nothing has been or is being done about it. It's the same story every week, which is probably why your question and ones like it often get poorly received.
    – Kevin B
    Commented Dec 13, 2021 at 15:41
  • 4
    Very often, the audit system chooses "hot" posts, and posts that were dealt with by a mod, as the signal such posts present show them as being "obvious" examples that noone should fail because 13 people agreed it was useful, or a mod noted that it was clearly spam, etc.. What this fails to capture is that more often than not, these "hot" posts are simply popular, not good. and the mod handled spam is often unreasonably hard to spot, hence it not being automatically handled by regular user spam flags.
    – Kevin B
    Commented Dec 13, 2021 at 16:12
  • 6
    [cont.] Because it's automatic, and because the review and curation systems themselves are primarily subjective, the audits are often mostly subjective... It's a game of recognizing audits, rather than you handling posts the way you feel they should be.
    – Kevin B
    Commented Dec 13, 2021 at 16:17
  • 13
    "It's a game of recognizing audits, rather than you handling posts the way you feel they should be"—exactly. I understand the value of appropriate audits, but there should definitely be a way to deal with bad ones short of posting here every time. Bad audits help nobody and they frustrate experienced users who are just trying to contribute to the site.
    – Chris
    Commented Dec 13, 2021 at 16:29
  • 12
    I lifted the suspension. That second one is borderline enough it shouldn't be an audit
    – Machavity Mod
    Commented Dec 13, 2021 at 19:19
  • 4
    Why waste precious time on reviewing, when you can be contributing answers and questions, helping people do their best at this great trade of ours? Plus, due to the arcane rules and edge-case audits, it's like walking on eggshells.
    – O. Jones
    Commented Dec 14, 2021 at 2:05
  • 9
    @O.Jones: There are a number of roles needed to keep the community running. We each contribute according to our interests and abilities. Speaking for myself, I spend far more time reviewing than I do answering. The OP has obviously done both. The audit system certainly does need updating, though. Commented Dec 14, 2021 at 3:03
  • 2
    @O.Jones, partly because there's a lot of review that needs doing. As Jeremy says, there are many ways to contribute.
    – Chris
    Commented Dec 14, 2021 at 4:25
  • 7
    @KINYUATIMOTHYNJIRU There's a lot of problems with your statement. Firstly, Chris is not a moderator. He's just a normal contributor with over 500 rep, so he got access to the "First Answer" queue (and many more). The bigger problem is that you demand that each downvote also has a comment to explain. While a lot of people (including me) do give a comment, voting is meant to be private and you don't have to explain why you do it. Commented Dec 14, 2021 at 6:37
  • 2
    @justANewbie indeed, quality voting and providing feedback are two entirely different processes which do not need to be performed by the same people. Someone who hasn't voted can provide feedback too. Someone who upvoted can do so too.
    – Gimby
    Commented Dec 14, 2021 at 11:11

2 Answers 2

13

The second answer really isn't unclear, but it arguably does need more supporting information (because the answer doesn't provide evidence that it's correct, although the voters evidently believe that it is), so I do think that you probably shouldn't have failed this. The system probably selected it because of all of the upvotes.

The first one is borderline VLQ, but it's probably not because it's more than just a link and appears to be an actual attempt to answer the question.

So, both are somewhat borderline. I agree that, in this case, the suspension seems a bit unfair.

4
  • 3
    " I agree that, in this case, the suspension seems a bit unfair." - morally perhaps, but there is nothing unfair about it happening to Chris just like it happens to everyone else. If you choose to review, you get caught by nasty audits. Unless you open each and every review item in a new tab or you have developed a sixth sense for audits.
    – Gimby
    Commented Dec 13, 2021 at 13:31
  • 9
    @Gimby, I see how it might look like I want special treatment. That's not what I want. I want to see the end of bad review audits, period, for all users. I probably could have framed that more clearly.
    – Chris
    Commented Dec 13, 2021 at 15:41
  • 2
    @Gimby RE: "Unless you open each and every review item" ... nah, just the borderline ones. Commented Dec 14, 2021 at 7:05
  • 1
    @Chris Not really, hence I respond to this answer and not to your meta post. But I'll grant you that if you want to make it neutral, you should only refer to the bad audit and not put your suspension front and center like it is in the title. It is a red herring that causes the discussion to grow tentacles.
    – Gimby
    Commented Dec 14, 2021 at 14:30
9

For some reason, it's NOT enough to just downvote a bad answer in an audit You need to flag it. If this is good or bad can be debated, but I think it's really bad that the need to flag isn't clearly communicated.

I don't agree that the first is clearly a bad answer, but I do agree that it's not suitable for an audit.

The second one is also a bad audit. The answer is ok as it is, but it not a clearly good answer. Your proposals about links to bug reports are completely valid.

And it's a design flaw that you cannot use "share feedback" for good answers. It's not communicated clear enough what share feedback means.

4
  • 1
    "For some reason, it's NOT enough to just downvote a bad answer. You need to flag it" - this needs more context. For some reason IN REVIEW it is not enough to downvote a bad answer. Outside of it is definitely perfectly fine to do so.
    – Gimby
    Commented Dec 14, 2021 at 14:32
  • 5
    @Gimby Ehm? This question is about a failed review, which is the context.
    – klutt
    Commented Dec 14, 2021 at 14:34
  • Fine, I use the wrong terminology again. It's written in an ambiguous way, it does not need to be.
    – Gimby
    Commented Dec 16, 2021 at 9:49
  • @Gimby Fixed it
    – klutt
    Commented Dec 16, 2021 at 12:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .