15

Why is this a low quality answer? It gives a lot of information on how to solve the problem.

https://stackoverflow.com/review/late-answers/36468062

13
  • 35
    Because the audit system is awful. It was deleted for policy reasons, not quality reasons, but the audit system can't tell the difference. (it's gpt generated)
    – Kevin B
    Commented Oct 1 at 18:36
  • @Kevin B Ahh I see. I didn't know that.
    – Rich
    Commented Oct 1 at 18:38
  • 14
    The combination of perfect grammar with utterly hopeless markup is a sign that it might have come from a bot. The phraseology is also very GPT-like.
    – Dan Mašek
    Commented Oct 1 at 18:59
  • 35
    This is why suspected AI posts should not be flagged as Very Low Quality or Not An Answer.
    – blackgreen Mod
    Commented Oct 1 at 19:36
  • 3
    IDK if one should be able to tell that was AI slop but but the answer itself has obvious quality issues. formatting is broken. that should have required edits at the very least. @BDL did you mean the answer? the question wasn't in question. Commented Oct 1 at 21:26
  • 5
    I think voting "looks OK" was the only invalid move on that review. for how long did you look at the answer? how long do you usually spend on a review? Commented Oct 1 at 21:33
  • 2
    @blackgreen maybe post an answer and expand on that?! :/ I do remember seeing comments (not necessarily from Mods) which were directing to use VLQ or NAA flags on AIGC.
    – M--
    Commented Oct 1 at 22:05
  • @M-- If you happen to know where those comments are, it would be good to know so that they can be corrected.
    – Ryan M Mod
    Commented Oct 1 at 22:28
  • @RyanM needle in a haystack...
    – M--
    Commented Oct 1 at 23:55
  • 3
    I admit to being a cheat and recommend doing the same. If you have any suspicions about the post you are reviewing, pop it open in another window. If it seems to have been deleted, problem solved; vote to nuke. If not, read through the comments, question and other answers for more context. There may be something subtle or outside your wheelhouse going on in the post. Commented Oct 2 at 0:21
  • 4
  • 3
    In general, if a diamond mod was involved in the deletion there might be more to the story than meets the eye. AI generated, plagiarism etc.
    – Lundin
    Commented Oct 3 at 7:53
  • @RyanM blackgreen might know more; see this: meta.stackoverflow.com/questions/431746/…
    – M--
    Commented Oct 4 at 5:43

3 Answers 3

25

According to a comment under that answer (which only >10k users can see), the answer was deleted because the author posted AI generated answers. Afaik, it is impossible to notice that in a review queue unless you open the answer directly.

The audit system unfortunately doesn't take into account why a post was deleted. If it got deleted, the audit system assumes that it was a bad post.

4
  • 15
    "unless you open the answer directly" -- IMHO doing so is not a bad idea in general.
    – Dan Mašek
    Commented Oct 1 at 18:56
  • 1
    It seems that usually it's safer to only vote on bad answers to be deleted. I rarely if ever fail an audit on those.
    – Rich
    Commented Oct 1 at 19:18
  • 12
    This isn't quite right. The issue is that it was deleted with a helpful NAA/VLQ flag. If the flag had been declined or disputed, it wouldn't have been used as an audit. In this case, it was marked helpful by deletion, but VLQ flags also get automatically marked helpful when the post is edited, so mods wouldn't always have the option to decline.
    – Ryan M Mod
    Commented Oct 1 at 22:25
  • 1
    “Afaik, it is impossible to notice that in a review queue unless you open the answer directly.” - It’s fairly trivial to identify GenAI content if you are looking for GenAI content. If something was deleted, then I always go back to, “it can’t possibly be “ok”, now can it?” Commented Oct 2 at 11:16
23

You voted "looks OK" on that answer.

I think one should have noticed the broken formatting at least. That answer does not "look OK" in the sense that it is not fine to leave as is. It should have required editing or any other verdict.

Whether it was AI-generated or not is secondary to its broken formatting, for the purposes of this review audit.

5
  • 15
    This is correct. Although I don't think users should flag AI-generated content as VLQ, and I don't generally think mods should mark such flags as "helpful" precisely because of the potential for bad audits, in this particular case, there was nothing wrong with the audit. This answer obviously did not look OK.
    – Cody Gray Mod
    Commented Oct 2 at 0:58
  • 4
    @CodyGray so then I wonder, do these things get discussed somewhere so the knowledge is spread among moderators? It's unrealistic to think the people pushing the flags in will change their ways so it really does fall on you guys to make the right choices.
    – Gimby
    Commented Oct 2 at 8:44
  • 1
    @CodyGray If I'm not mistaken, "Edit" would also have failed the audit, so in that case there was something wrong with the audit. Commented Oct 3 at 6:40
  • 1
    Yes, @Gimby, mods can and do discuss these things in private channels amongst ourselves. But it is still a bit of an issue that not all mods read everything ever posted. Something the site has lacked literally since day one has been comprehensive documentation. Fortunately, for mods in particular, there's been enough continuity of the team that there is almost always someone around who knows and can be asked. The same, unfortunately, cannot be said of staff. Another point worth mentioning: there are virtually no uncorrectable mistakes, and things like this are how mods and the community learn.
    – Cody Gray Mod
    Commented Oct 3 at 8:48
  • 2
    @DonaldDuck That's a fair point and may well be correct. In all honesty, I don't know enough about how the current iteration of review queues work to know whether that's a true statement or not. Which I guess goes back to Gimby's point. I could probably go find out, either via Meta or by asking around, but it's not something that I have the time or inclination to do, because it really doesn't matter all that much. (By coincidence, I happened to just come across a Meta Q&A that suggests "Edit" does not fail audits. So you might actually be wrong. But I still don't know for certain.)
    – Cody Gray Mod
    Commented Oct 3 at 8:49
18

The answer ended up in the audits due to my mistake.

Answer had several pending flags which I handled in bulk while I was focusing on the AI part, and I missed that one of the flags was not a custom mod flag, but VLQ flag which should have been declined.

I have edited the answer which should (hopefully) remove it from being used in future audits.

3
  • 10
    For the record, there was a time a couple years ago at the start of the AI wave where moderators would mark VLQ and NAA flags on AI posts as "helpful", with the rationale that bringing that content to our attention was always welcome regardless of the method. With time, and probably after the 2023 strike, this position has shifted: using the wrong flag ends up doing more harm than good, in the form of bad audits and risk of leaving users unaware of the AI ban. Also, the "Not An Answer" flag is plain incorrect, since AI posts do attempt to answer the question.
    – blackgreen Mod
    Commented Oct 2 at 13:14
  • 9
    The hoops y'all have to jump through to make the flag system work well are absurd. The biggest kudos in the world for the job you all do <3; I really wish the system got the overhaul it needed to make it less of a burden, too.
    – zcoop98
    Commented Oct 2 at 15:47
  • 3
    @zcoop98 Thanks! Handling part is not that problematic, although it can cause issues like this one if you don't keep greater picture in head at all times ;) It would be better if audits would not be selected from posts that have multiple different flags cast, regardless of how they are handled. And it would be even better if we could more easily fix mistakes and change (dispute) how the particular flag was originally handled.
    – Dalija Prasnikar Mod
    Commented Oct 2 at 17:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .