-10

I've completed ~1000 tasks, and this is the first time I've ever been suspended from the review queues.

The suspension message reads:

The following tasks were handled incorrectly:

  • /review/low-quality-posts/34424050
  • /review/low-quality-posts/34431089

We suspect that you are not actually reviewing the tasks.

That is blatantly wrong. It failed me on both audits because I attempted to edit the posts. Why would I try to edit posts if I wasn't actually reviewing them? In fact, I will show that I reviewed (at least one of) them more thoroughly than the original reviewers did.

review/low-quality-posts/34424050

The first post is a high quality answer with minor formatting issues I would've liked to fix. People who "aren't actually reviewing the tasks" might see the URL and jump to conclude that it's spam. Except it's clearly a string template, and that same URL is contained in the question and was only copied into the answer. Therefore, this user's answer was definitely not spam and was wrongly flagged and deleted in the first place!

question to review/low-quality-posts/34424050


review/low-quality-posts/34431089

The second audit does contain a hyperlink to an individual's website, but it otherwise gives exactly what the question asked for. Instead of recommending deletion, I determined it was best to simply remove the link and keep the answer. Is that not reasonable? After further scrutiny, I now suspect the answer is AI generated, but I cannot objectively justify that suspicion. In any case, I don't think it's fair to audit reviewers on the ability to identify content potentially from AIs or know exactly the protocol for handling it.

Furthermore, the question to this answer was closed for "seeking recommendations", but I believe this is also a mistake. As the title states, the question is "What is embedded AI?". It primarily asks for a definition, not a recommendation, and the request for an authoritative source is inconsequential, if not completely warranted.

question for review/low-quality-posts/34431089

For these reasons, my suspension and the failed audits should be retracted immediately.

As the suspension messages says:

Poor reviews negatively affect the community. Please give each task adequate attention in the future

Poor audits also negatively affect the community's willingness to provide free content review. Please select posts used to audit reviewers more carefully.

4
  • 2
    "Poor audits also negatively affect the community's willingness to provide free content review. Please select posts used to audit reviewers more carefully." - related: Can an unintended consequence of "review audits" be fewer reviewers? (and my answer post there)
    – starball
    May 23 at 6:11
  • 1
    I don't see how the second answer could be provided to an on-topic question. There is no need to edit completely off-topic question or answers to those. May 23 at 16:20
  • @Alexei The question asked for a definition of "embedded artificial intelligence." The answer gave a definition of "embedded artificial intelligence." It's contrived, easily Googleable, and adds somewhat irrelevant information that wasn't requested - but most content from new contributors is that way. It is not unambiguous, and I've always thought it was best to give people the benefit of the doubt instead of just deleting whatever doesn't look quite right.
    – ABabin
    May 23 at 17:19
  • 2
    You may want to ask separate question whether that question asking for definition of "embedded AI" is on-topic or not. May 23 at 18:28

1 Answer 1

25

The first post is a high quality answer with minor formatting issues I would've liked to fix. People who "aren't actually reviewing the tasks" might see the URL and jump to conclude that it's spam. Except it's clearly a string template, and that same URL is contained in the question and was only copied into the answer. Therefore, this user's answer was definitely not spam and was wrongly flagged and deleted in the first place!

Nope, not quite: look at the comma after "the 'id' variable between 'Contact' and 'view'": it contains a link to https://mrjoneorganics.com/product/moringa-powder/, which I haven't clicked but sounds pretty irrelevant and spammy to me. I'm not sure where they got the text from, but that link disguised on a small punctuation character makes it clear that it's spam.

These are genuinely hard to spot in the default UI; I use a userscript that makes short links more obvious:

A screenshot of the userscript in action, making the link buried behind a single comma character much more obvious (and Christmasy, since it uses a red-and-green color scheme).

The second audit does contain a hyperlink to an individual's website, but it otherwise gives exactly what the question asked for.

Okay, let's see if that could possibly have been posted in good faith. Is the link relevant in any way to the concept of Artificial Intelligence, which is the text that the link is on?

Screenshot of website with text: Hello I am RIYA FATHIMA Digital Marketing Strategist in Kannur "Let's grow together"

...nope. Looks like pure spam.

Instead of recommending deletion, I determined it was best to simply remove the link and keep the answer. Is that not reasonable? After further scrutiny, I now suspect the answer is AI generated, but I cannot objectively justify that suspicion. In any case, I don't think it's fair to audit reviewers on the ability to identify content potentially from AIs or know exactly the protocol for handling it.

Please see Should spam posts be edited? for an explanation of why editing spam posts is not a good solution. There are plenty of reasons, but a few that are especially relevant here are:

  1. giving spammers reputation allows them to continue to post spam answers that may slip under the radar,
  2. editing spam out doesn't alert moderators to spammers that need to be stopped, and
  3. the text is generally plagiarized or AI-generated (this looks AI-generated). The point isn't to identify if it's AI-generated or plagiarized or whatever; the point is to identify that it's definitely spam, flag it, and remove it entirely, as spammers are pretty reliably not writing high-quality original answers.
22
  • 4
    All of this said: if you acknowledge that you'll keep an eye out for sneaky spam links and flag spam posts rather than trying to edit them going forward, I'm happy to consider the audits as having accomplished their education goal (yeah, the condescending messaging is unfortunate, sorry about that...) and lift the review suspension.
    – Ryan M Mod
    May 23 at 6:36
  • 3
    I'm glad you buried that olive-branch of forgiveness in a comment so it wouldn't stop people like me from upvoting this great answer!
    – Cody Gray Mod
    May 23 at 7:01
  • 3
    Question of the day: Is @CodyGray starting a review (un)suspension war with their fellow mod?
    – rene
    May 23 at 7:05
  • 3
    There are plenty of users to review suspend, @rene; I wouldn't have to work nearly that hard. Also, it's genuinely difficult for me to find anything Ryan does or says that I disagree with.
    – Cody Gray Mod
    May 23 at 7:07
  • 1
    Do we have a feature request to make links hidden like that (or in general) more obvious by any chance? May 23 at 7:29
  • 3
    Not much we can do, @Abdul, as the <blink> tag is deprecated and not supported by modern browsers. If we were going to do something, it would probably make more sense to block the posting of these single-character links...
    – Cody Gray Mod
    May 23 at 7:35
  • 3
    @CodyGray I was more thinking along the lines of adding an icon. Although yes it does make more sense to just block single character links, although would that work with zero width spaces (Probably need to filter those out or something)? May 23 at 7:37
  • 1
    @ABabin - Properly identifying ChatGPT content is now something reviewers should be able to identify. I was immediately able to identify that the second answer was generated by ChatGPT. May 23 at 12:12
  • 3
    @ABabin - We are talking about the comma turned into a link not the url between the href. For a community not to delete spam, it must be flagged as spam by more than 2 users. The embedded link, the comma, was spam for moringa powder. May 23 at 12:53
  • 1
    @ABabin Unfortunately, not without clicking the link at the top of the review and editing it outside of the review. May 23 at 14:14
  • 1
    @SecurityHound Do you think it is fair to audit and suspend people on whether or not they've learned to jump through the right hoops in order compensate for the weaknesses of the platform they're providing free labor for? Is it against the rules to merely consider editing a post that doesn't outright scream that it contains easily removable spam?
    – ABabin
    May 23 at 17:04
  • 4
    @ABabin To be clear, I think that the site should have a read-only show-markdown feature as well (not unlike the suggested edits queue). Might be worth asking about if nobody has already. May 23 at 17:59
  • 2
    @SecurityHound Now that I think about it, when you open a review audit post in another tab, wouldn't you see that it was already flagged and deleted? Doing that for every review makes the whole audit system completely pointless.
    – ABabin
    May 23 at 18:33
  • 1
    @ABabin doing that for every review does make it completely pointless. There are those who feel audits are a "teaching" tool, however at best they teach you that the audit system, and reviewing as a whole, is deeply flawed.
    – Kevin B
    May 23 at 18:45
  • 3
    I don't think of the audits as a teaching tool, but they are one tool to flag people who may need pointers in one specific area. In general I think the audit system is pretty poor and could use improvement, especially if we will need a large number of additional reviewers for Staging Ground. These audits happened to be correct, but they didn't explain the logic to the reviewer who hit them, so they did not provide any education until the reviewer came to Meta for an explanation.
    – Ryan M Mod
    May 23 at 20:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .