56

I've just failed an audit in /review for a very stupid reason. I was shown an answer, by a new user, to this question. The answer was very good, but I decided to be sure, and opened the question in a new window. And I saw, that this question has an ABSOLUTELY IDENTICAL answer by a different user, and no answer by the user in the audit.

So I thought: "What the heck, the new user just made copy-pasta from a good answer, and then it was quickly deleted!" So I flagged this answer with the other reason and wrote "That's an exact duplicate of different user's answer". And then clicked "Done".

Bang! I'm told, that

This was a high quality post and you should have considered leaving it as-is or even upvoting.

And the new user's name changes to the nickname of the user whose post I thought was copy-pasted.

So, my questions are:

  1. Is this normal behavior, that I'm not shown a true "new" user name?

  2. If it's normal, then how do I find out whether I'm checking the original answer or a copy/paste? Because I've seen this several times outside of review, when a new user just copies someone else's answer.

5
  • Assuming you were in the Low Quality Post Queue, it wasn't a low quality post. Did you click 'Looks Good' or Recommend for Deletion? I am little confused as to how this played out Commented Jul 22, 2014 at 12:55
  • 5
    @staticx it was a first post
    – OGHaza
    Commented Jul 22, 2014 at 14:26
  • 2
    Thats just how review queues are intended to work: do the absolute minimum required by the queue and don't try to look around and find anything else possibly wrong with that post. Look at it as if it is in total isolation and the world around it doesn't exist.
    – PlasmaHH
    Commented Jul 23, 2014 at 13:26
  • possible duplicate of Audit failure with reasonable actions
    – Barett
    Commented Jul 30, 2014 at 22:54
  • @Barett, yeah, that looks like the same issue. I did some search before posting question, but didn't find it. However, it doesn't have any answers at all.
    – Olter
    Commented Jul 31, 2014 at 5:25

5 Answers 5

33

Yes, this is entirely normal behaviour, to make sure you are paying attention to the post, not the user. Imagine a post by Jon Skeet being picked, for example.

If it was a copy-paste job, you'd see both answers when you followed the link.

27
  • 28
    If you don't have 10k rep, you can't see deleted answers. How do you distinguish between audits and deleted answers (possibly deleted between when you got the review and when you check the question)? Commented Jul 23, 2014 at 3:29
  • 3
    Then you judge just the content. If it isn't an audit, you said 'looks good' to something that is already gone (no loss), if it is an audit, audit passed! Commented Jul 23, 2014 at 6:47
  • 6
    Who is Jon Skeet? :)
    – Stefan
    Commented Jul 23, 2014 at 13:30
  • 21
    @MartijnPieters, the most prolific and valued users of this site are people who like to dig into things and understand them. They will not "just judge the content" if there's something else they think they can discover, and if something doesn't add up then it will smell fishy, at best. So, thanks, but your advice is naive. It only stands as the best advice because failing a single audit is not a big deal, but it's not a positive way in which to modify user behavior.
    – Jason
    Commented Jul 23, 2014 at 13:39
  • 2
    @Jason: Sure, but in the context of reviews with <10k points, those are your options. Commented Jul 23, 2014 at 13:44
  • 13
    @MartijnPieters: I agree with Jason that this does not feel particularly constructive. I realize audits are tricky to get right, but certainly here it feels dysfunctional: the user invested more care than a normal reviewer and is punished for it. Commented Jul 23, 2014 at 13:48
  • 7
    @MartijnPieters, if the primary goal of reviewing is to pass audits, then that's a good option. If the goal of reviewing is to review well, then best to ignore the failed audit and move on, and trust that you won't run into so many badly designed audits that it actually impacts your ability to do it at all.
    – Jason
    Commented Jul 23, 2014 at 13:51
  • 7
    @MartijnPieters what incomplete evidence? They acted on the evidence they were given. The fact that some of that evidence had been faked by the site tricked them into making a mistake. i.e. Stack Overflow lied to them and as a result they pressed the wrong button
    – Tim B
    Commented Jul 23, 2014 at 13:51
  • 15
    To be explicit, Stack Overflow claimed (contrary to reality) that the post was a new user's first post. Maybe the OP should not have believed that, but if it really was a new user's first post, then the OP did the right thing. And suppose a new user does start copying answers of others, and gets answers deleted because of it. What's preventing one of those answers from showing up as an audit?
    – user743382
    Commented Jul 23, 2014 at 13:56
  • 1
    @Jason: The audit system is designed at catching out those that are just clicking through the queue as fast as possible to get badges. Anyone that pays attention is welcome to detect that there is an audit going on. Commented Jul 23, 2014 at 14:01
  • 13
    You simply cannot defend the audit system as without flaws. This is a flaw, simple as that. There are many, and we see posts about them in Meta all the time. I can understand the advice to just live with it and move on, I cannot understand the advice that you should act differently in some way just because of audits. That's the last thing you want. That's the whole point of how the SE system was designed, to encourage people to contribute in ways that they feel is constructive and let popular opinion sort out what's best.
    – Jason
    Commented Jul 23, 2014 at 14:01
  • 3
    @Jason: Who says I am defending the audit system? I am advocating a practical approach because there are flaws. The basic conclusion is that the OP conclusion about a copy was wrong, we can explain why it was wrong and how not to draw that conclusion in the future. In case of real plagiarism the post would still be there. Commented Jul 23, 2014 at 14:02
  • 1
    @MartijnPieters, that is an excellent point. This is an audit that rewards robo-reviewing, because only on taking your time does the answer appear suspect.
    – Jason
    Commented Jul 23, 2014 at 14:03
  • 5
    Why not just not show the user name in the review at all? Commented Jul 23, 2014 at 14:13
  • 1
    @MartijnPieters Are you saying that when an answer is deleted, that answer is never used as an audit? I'm not sure about my own review history, but I have seen questions here on Meta about users who failed audits, when they thought the post looked good? Or do you mean that in the event of real plagiarism, posts will be deleted more destructively?
    – user743382
    Commented Jul 23, 2014 at 14:26
16

This could be really and truly solved by making "flag for plagiarism" an explicit option in First Post review, and count it as a Pass if the reviewer selects that on an anonymized high-quality post used for audit.

14

I ran into the exact same problem when I started doing reviews. In fact you will find a meta post very similar to yours discussing it.

All I can say is, you are not alone. It's stupid but it's the price we pay for reducing the number of robo reviewers...

Here's my post on it: High percentage of audits combined with audits failed for bad reasons putting me off reviewing

3

Audits will deliberately fake most details to ensure you're judging as much by the post content, and as little by the metadata like author, score, accepted, etc as possible. This breaks somewhat in cases where that metadata is actually crucial, like here.

But there is a workaround that will never fail: in a new tab, open the "link" link to the right of the post under review. If it goes to a deleted question, this will be obvious; if a deleted answer, the link will not scroll down at all (unless you have 10k, in which case the deletion will again be obvious); otherwise, it will scroll to the top of the answer being reviewed, where you can then look around at the rest of the page to determine the actual situation — in particular, see if there's any actual duplication going on, or any other audit-related funny business.

0

The problem doesn't seem to be the audit-process, but the review-process.

If the audit is faking metadata to ensure that people review based on post and not on metadata the obvious solution would be to not include the metadata for any review, i.e. just show the post itself.

Obviously that may slow down reviews, which is a downside. Hopefully the better quality of reviews will make up for that - but it has to be tested.

(The comment should likely be included; those are a bit problematic - sometimes they highlight an actual problem, sometimes they are wrong.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .