24

Yesterday I reviewed a good looking answer with No Action Needed, but it turned out to be a review audit that's supposed to be reviewed for removal because it is a copy of another answer.

Here's the review audit (the -3 votes was obviously not shown when I reviewed it):

The question is, how am I supposed to know that this answer was copied from another one.. when the only information I see is the answer I'm reviewing and the question it's answering below it?

Am I supposed to open the question of every answer I review and make sure there aren't any similar answers already posted on the question?

I read many meta question about deeming reviews as incorrect, but I couldn't find one about this specific case.

18
  • 2
    The fact that the user who posted it has been deleted is a good indication that something fishy is going on.
    – ivarni
    Commented Feb 23, 2018 at 8:07
  • 1
    @ivarni even that wasn't clearly shown when I reviewed it (if I remember correctly)
    – Omar Einea
    Commented Feb 23, 2018 at 8:09
  • 3
    Am I supposed to open the question of every answer I review and make sure there aren't any similar answers already posted on the question? that doesn't make sense. Not every one, but in cases like this, yes. I find the review tools lack all the information and it's often better to go to the question itself.
    – George
    Commented Feb 23, 2018 at 8:16
  • 8
    Users get banned for failing many audits. Everybody gets one wrong once in a while, review has a strong knack for turning people into zombie clickers that reject a flag from a concerned SO user without looking. Focus on all the other ones you failed. Commented Feb 23, 2018 at 8:20
  • @ivarni True, but often this is used to hide the fact that the post is from a high rep user (at least in First Posts and Late Answers). Besides, shouldn't the review be on the post only not on the author. Commented Feb 23, 2018 at 8:23
  • @George That wouldn't have helped here as it can be seen in the screenshot that there aren't any other answers. Commented Feb 23, 2018 at 8:23
  • I understand that, I do go to the original questions sometimes and I do get some incorrect reviews every while. But I feel this one is a bit unfair.
    – Omar Einea
    Commented Feb 23, 2018 at 8:24
  • @ThomasSchremser This is a screenshot of the review, they only show the answer that's being reviewed, however going to that actual question would show the rest of the answers
    – George
    Commented Feb 23, 2018 at 8:26
  • 8
    Remember that you weren't banned for failing this audit. You were banned for failing all the others audits you failed previously. Do not make a big deal out of this. You were probably banned only for a couple of days. You'll come back after that and be more careful with your reviewing.
    – yivi
    Commented Feb 23, 2018 at 8:26
  • 3
    @OmarEinea I learned to see the "signs" of when the system tries to audit me. Good indicators are "hidden" users or just in general questions that don't seem to belong (e.g. extremely well-formatted questions in Triage). It's absurd, but the best way to get through audits is to learn when the system is trying to audit you and just tell it what it wants to hear.
    – MechMK1
    Commented Feb 23, 2018 at 9:13
  • 2
    The best way to avoid getting caught by an audit is to always go look at the question and answers in a new tab outside of the review. We all run across bad audits from time to time, it's just something that will happen when they are automatically generated. Over time you'll develop a sense for when something is out of place and get better at spotting them.
    – ivarni
    Commented Feb 23, 2018 at 9:14
  • 2
    @DavidStockinger got it. trick the system when it's trying to trick you.
    – Omar Einea
    Commented Feb 23, 2018 at 9:14
  • 7
    If I see a well-written answer by a 1-point user, I become suspicous. Stack Overflow has a learning curve. Very few people can write good SO answers right away. So a well-written answer by a 1-point user is likely to be an audit or plagiarism. Either way you should go to the question and see what's going on. And if turns out to be plagiarism, flag for mod attention! Commented Feb 23, 2018 at 9:15
  • 2
    @OmarEinea Exactly. The audit system is far from perfect, and the sheer number of daily complaints about it only reaffirm that. Just learn how to game the system and everything will be fine.
    – MechMK1
    Commented Feb 23, 2018 at 9:16
  • 6
    And while you feel that you are gaming the system, what happened was that the system manipulated you into paying more attention while reviewing. Everything seems to be working according to the plan. :)
    – yivi
    Commented Feb 23, 2018 at 9:19

2 Answers 2

15

If an answer is copy-pasted from another answer, you can't see that in review.

However, there are a few things to note.

  • First, No Action Needed is almost never the right choice. It was added because people would upvote answers just to get the review done. Giving them No Action Needed limited the damage - but when you feel this is the right choice, consider using Skip instead.

  • Second, a well-written answer by a new user is extremely rare. Stack Overflow has a learning curve. Very few people know how to write good answers right away. So if you see a well-written answer by a new user or an anonymous user in /review... do go to the question page and check.

    • If the answer you're reviewing isn't there, it's an audit.
    • If the answer you're reviewing is there, Google!! There's a good chance it's a copy-paste from official documentation or from another answer - either on the same page or on a similar Stack Overflow question.

So - this particular audit was nasty, because it was a copy-paste of a good answer. You couldn't have known without visiting that page. But there were a few signs that it was actually a bad answer.

So in general, the advice for reviewers is: be paranoid. Not just about audits, but about plagiarism in general.

14
  • 5
    The solution is not "see through the audit system and learn to spot audits". If that's the solution then why do we have audits in the first place! Plagiarism is not an issue that user moderators are expected to handle. It should be handled by diamond moderators.
    – Lundin
    Commented Feb 23, 2018 at 10:04
  • 3
    For mods to handle it it first needs to be flagged by... regular users. That's one of the reasons there are review queues in the first place.
    – ivarni
    Commented Feb 23, 2018 at 10:07
  • 2
    @Lundin This is where my advice comes in, to use Google. If the answer you're reviewing isn't on the original question page, you're done because you now know it's an audit. But if the answer is there, you're not done, because it can still be plagiarism. And the reviewer needs to check for that. Commented Feb 23, 2018 at 10:15
  • 3
    @ivarni Except, you can't detect if it is plagiarism in reviews, so reviews is a horrible tool for that.
    – Lundin
    Commented Feb 23, 2018 at 10:16
  • 2
    @Lundin If, as a reviewer, you suspect plagiarism... then you should go the extra mile. See if you find evidence of plagiarism. That usually means you have to go beyond the /review page. Commented Feb 23, 2018 at 10:19
  • 2
    @S.L.Barth The point is, there is nothing in the reviews that helps you spot plagiarism. We are not expected to watch for it except when doing wiki tag edit reviews. If you come across it and can prove it, great, flag it. But this is not the normal procedure for reviews.
    – Lundin
    Commented Feb 23, 2018 at 10:22
  • 2
    @Lundin Of course you can. The text in the answer is right in front of you and google is only a tab away. If you're not prepared to do that then maybe boycotting reviews like you suggested in your answer is the way to go for you.
    – ivarni
    Commented Feb 23, 2018 at 10:27
  • 2
    @ivarni This has never been a criteria for reviews, you are making this up right now. Link to the policy where it says that we are supposed to look for plagiarism during first post reviews?
    – Lundin
    Commented Feb 23, 2018 at 10:44
  • @Lundin There is a case for the OP copy/pasting in the First Post review guidelines. "The original user posting the answer as the exact copy of someone else's answer (similar to a thanks)" . I'll grant it's not exactly the same as checking for plagiarism. I think checking for plagiarism should be in the FP and LA guidelines. Commented Feb 23, 2018 at 10:52
  • 1
    There's nothing in the guidelines that explicitly state that you should look for plagiarism, nor is there anything in the guidelines that explicitly say you should flag posts written in another language. The FAQ only describes the UI and I guess they expect people to use common sense. Not looking for copy-pasted plagiarism during a review is a very strange thing in my opinion, and I can promise you that I'm not making that up.
    – ivarni
    Commented Feb 23, 2018 at 10:56
  • @S.L.Barth That's only the specific case of tag wiki reviews that I already mentioned being a valid exception.
    – Lundin
    Commented Feb 23, 2018 at 10:57
  • @ivarni Then where does it end? Should we also do a background check on the poster to see if they are a known spammer? Check if the post is a cross post on any other SE site? Those requirements aren't reasonable for a review.
    – Lundin
    Commented Feb 23, 2018 at 11:01
  • If a known spammer posts an otherwise valid question, why is that an issue?
    – ivarni
    Commented Feb 23, 2018 at 11:03
  • 1
    @Lundin As far as I'm concerned, it ends after a Google search. If a search on the web doesn't turn things up, it's probably original content. (Although I've Skipped my share of reviews even after seaching, just to be on the safe side and not accidentally validate plagiarism!) And you don't have to search for every post - just the ones that raise your suspicions. Commented Feb 23, 2018 at 11:03
10

There is no way for you to tell by the review only. The answer is perfectly fine and no action needed is the correct way to review it. You have done nothing wrong.

Plagiarism is not an issue that user moderators are expected to handle, apart from flagging it in case it is spotted. It should be handled by diamond moderators and the correct approach is to flag for diamond mod attention.

We are not expected to look for plagiarism in reviews (except for the specific case of wiki tag edit reviews), simply because they are completely unsuitable for the purpose of spotting plagiarism.

You can tell that it is a review by cheating the system, such as opening the link to the post and checking if the answer was deleted. But we should really not be expected to do this for every review, just to dodge the audits.

The problem is simply that SO review audits are horribly broken and have been so for many years. SO will not fix the problem. The best thing you can do is to boycott reviews in order to put some pressure on SO to finally fix the audit system.

Not the answer you're looking for? Browse other questions tagged .