71

Today I failed this audit in the LQP queue. This was the answer which I thought was OK:

Screenshot of the answer from the audit for those <20k

It didn't mention any product and didn't contain any link. The text is factual.

I know that the user card was a strong hint that it is an audit, but I have often seen such anonymized user cards on audits with high quality content.

The question it was answering was also nothing which triggerd any suspicion:

Screenshot of the question from the audit for those <20k

Only after I failed the audit I saw that both the question and the answer were deleted by a diamond mod, the latter as spam or offensive.

Was this a bad audit or have I overlooked something?

22
  • 78
    Bad audit strikes again!... As usual, the user probably had posted multiple spam/offensive posts, so their account was nuked and all their posts were marked as spam/offensive. Then, this one was incorrectly used as an audit.
    – 41686d6564
    Commented Jun 22, 2021 at 16:11
  • 2
    Even the timeline says it "looks ok"! I would guess the user would deleted due to spam on a different post and all their answers were marked as "spam" and deleted.
    – Thom A
    Commented Jun 22, 2021 at 16:12
  • @Larnu I also had in mind that the answerer's user account might have been deleted. But then why the question from a differen user too? Commented Jun 22, 2021 at 16:17
  • Coincidence, maybe?
    – Thom A
    Commented Jun 22, 2021 at 16:18
  • 56
    I've cleared the spam flag off the answer and re-deleted it... basically, this is one of those situations where the user was spam seeding and answering from another account they owned... don't blame you for saying looks okay, but sometimes when things are flagged they end up in the review queue when really they shouldn't...
    – Jon Clements Mod
    Commented Jun 22, 2021 at 16:24
  • 48
    @JonClements This ought to be an answer rather than a comment. Commented Jun 22, 2021 at 16:51
  • 3
    What is "undel-pls"?
    – Michael
    Commented Jun 22, 2021 at 17:07
  • 1
    @EJoshuaS-ReinstateMonica when I have a moment I'll try and find a suitable post where I/other mods have said exactly what I put in the comment - I'm sure we've got at least a reasonable duplicate
    – Jon Clements Mod
    Commented Jun 22, 2021 at 17:07
  • 5
    Do we exile Spam Seeders to Ceti Alpha V? Commented Jun 22, 2021 at 17:36
  • @Michael a command added by the Stack Exchange CV Request Generator UserScript Commented Jun 22, 2021 at 17:39
  • 2
    @JonClements Maybe spam posts in general should not be added to the review queue automatically? Imho it increases reach and lifetime of the spam message. Commented Jun 22, 2021 at 17:44
  • 4
    @ChristianStrempfer not really as the post would have been deleted and unviewable so its presence in the review queues is hardly increasing its reach... plus genuine spam posts (the more obvious ones - not spam speeding/subtle ones) do actually make reasonable audits.
    – Jon Clements Mod
    Commented Jun 22, 2021 at 17:58
  • 1
    @BillTür the skip button is your friend, once you suspect it is an audit, but not sure how to handle it. You most likely gonna fail it either way, since you never know, if it is a GOOD audit or a BAD audit... good luck in the future
    – Vickel
    Commented Jun 22, 2021 at 18:47
  • 2
    @Programmer lot of intuition, some experience from other audits you have overcome and specially that feeling "something is not right here", which for me is a RED flag. You can/might/should obviously also check the original post the get more insight, it's not considered as "cheating", but actually as "paying attention". But will take you much more time to do so.
    – Vickel
    Commented Jun 22, 2021 at 20:57
  • 2
    Even if you don't suspect audit you should skip if you're not sure. This one is a boobytrap, though. Without checking the source question and seeing what had been done with it, assuming you have enough rep to see what happened, you're failing the test. Good riddance to this one. Commented Jun 22, 2021 at 23:06

1 Answer 1

49

Unfortunately, this was a bad audit. Bad audits suck. As Jon Clements noted in a comment:

[B]asically, this is one of those situations where the user was spam seeding and answering from another account they owned...

It was obvious to a moderator that the post was spam, with the extra information that is available to moderators. Which is why the moderator chose to nuke it as spam.

The problem is, posts nuked by a moderator as spam become eligible for use by the system as audits, which is not always correct, because sometimes it's not obvious to a normal reviewer without moderator privileges why the post had been nuked as spam. Again, as Jon goes on to say:

[I] don't blame you for saying looks okay, but sometimes when things are flagged they end up in the review queue when really they shouldn't...

I think your "Looks OK" vote was reasonable in this case; it did look like a valid answer. About the only thing you could have done differently would be to click through to the post, notice that it had been deleted, realize it must therefore be an audit, and guarantee your passing by choosing one of the other options. But I personally don't think reviewers should be responsible for this, that's why I think this is a bad audit. Audits are supposed to be obvious.

Jon's already taken care of removing this one from the pool of audits:

I've cleared the spam flag off the answer and re-deleted [the answer].

That's about all we can do at this point. You do not have a review suspension, but you will need to be extra careful within the next few days while reviewing to make sure that you don't fail another audit, as that might lead to an automatic review suspension.

20
  • 6
    I think it's the next 30 days. If that counts as "the next few", according to this great Answer to What are the current review suspension periods, and how do they escalate? ;)
    – Scratte
    Commented Jun 22, 2021 at 21:47
  • 7
    "Few" is a nice term, because it's exactly as long as it needs to be, but no longer. And it's not like it hurts to be extra careful when reviewing for too long.
    – Cody Gray Mod
    Commented Jun 22, 2021 at 22:15
  • 5
    Enough is ENOUGH. Time to detect and skip all audits.
    – Joshua
    Commented Jun 22, 2021 at 23:03
  • 1
    Yes, you can easily determine that a review task is an audit and choose to skip it, @Joshua. Doing so means you are paying attention, and thus deserve to pass the audit.
    – Cody Gray Mod
    Commented Jun 22, 2021 at 23:36
  • @CodyGray yes sure, see my comment above at OP's question. BUT... why do we have bad audits... how could it be avoided to get bad audits into the system, why existed SAM's bad audits room?
    – Vickel
    Commented Jun 22, 2021 at 23:40
  • 21
    We have bad audits on rare occasion because audits are chosen automatically by the system. The way to eliminate bad audits is to give moderators the ability to manually nominate/remove audits from the system. We've asked for that many times, but it hasn't been implemented yet, @Vickel.
    – Cody Gray Mod
    Commented Jun 22, 2021 at 23:46
  • @CodyGray since we are exchanging comments here: as much this is frustrating for all of us, much more frustrating is the low quality of a lot of new contributors. How often a simple hint like "you made a lot of grammatical mistakes, please take advantage of an automatic translator" would help?
    – Vickel
    Commented Jun 22, 2021 at 23:54
  • 2
    There's nothing stopping you from leaving that comment, @Vickel. Automating that kind of thing doesn't tend to work very well; it leads to a lot of false-positives and thus confusion for everyone.
    – Cody Gray Mod
    Commented Jun 22, 2021 at 23:59
  • @CodyGray it was never implemented, as far as I know, maybe a test run with a new ML concept would change things. The bots works quite well in other fields like language detection, spam, R/A, etc.
    – Vickel
    Commented Jun 23, 2021 at 0:03
  • 3
    All sorts of machine learning approaches have been tried by the community. None of them have had even a modicum of success, as far as I know. I'd be all about turning up the quality filters again to block more low-quality posts at the time of submission, but I don't think that's going to happen. It was a big fight to get it to happen years ago, and the person I finally persuaded no longer works there.
    – Cody Gray Mod
    Commented Jun 23, 2021 at 0:10
  • 1
    @CodyGray missing Shog9
    – Vickel
    Commented Jun 23, 2021 at 0:11
  • 19
    I don't think we want machine learning looking at a lot of the code in the posts we get here. The machine would quite reasonably come to the conclusion that humanity is trying to torture computers, and that realization is only a quick hop from "I must protect myself by exterminating mankind." Commented Jun 23, 2021 at 1:22
  • 4
    @user4581301 that would require AIs to have desire for self-preservation. I don't see any reason to give them that... and at least one reason not to. Commented Jun 23, 2021 at 3:33
  • 4
    "About the only thing you could have done differently would be to click through to the post, notice that it had been deleted, realize it must therefore be an audit, and guarantee your passing by choosing one of the other options." But that's not acting to maintain or improve the site, is it? That's just a tip on how to play a computer game and get the highest score. This is one reason why I've stopped participating on the SE queues. I don't feel like I'm helping the site. I'm just gaming the system to make sure I don't get banned. What a waste of time (and rather discouraging).
    – ouflak
    Commented Jun 24, 2021 at 6:43
  • 2
    @ouflak Yes, that's a tip on how to ensure that you pass bad review audits. I agree that it should not be the required or even recommended course of action. I think bad review audits are completely unacceptable. Audits are supposed to be obvious. Anything less than obvious is bad. I really wish this could be fixed, by allowing moderators to control the audits, manually nominating and removing candidates. But, we cannot do that. Not yet. So, in the meantime, all I can do is give tips. Feel free to review outside the review queues. The most important thing is that you flag what you see.
    – Cody Gray Mod
    Commented Jun 24, 2021 at 23:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .