20

I just failed my first review audit (\o/) regarding the First Post Queue and I don't really understand why (actually I think I do because the returned message is explicit but this question aims more at improving this kind of first post review audit).

So I was introduced to this question where a new user (1 rep) seemed to have posted 2h ago a very good answer. I even found that the answer was a bit too good for a first post so I decided to open the topic in a new tab to quickly check all the answers on a neat interface (I find the review one a little bit less clear which is normal since you have more options).

And guess what I found ? He had copy/paste the exact same answer as the accepted one (originally written by Jerry Coffin). I didn't investigate further since it seemed clear to me that the new user was attempting at gaining reputation through the answers of others.

The result of the audit is "you should have considered leaving it as-is or even upvoting". You bet I didn't : I flagged the answer and if I could have double-flagged it, I would have done it !

So in the end, am I paying my overzealous first post review moderation ? How could those particular audit could be improved ?

I can think of one way : do not select an "awesome first-class guru-like" answer as the first post of a new fake user. I do agree that this kind of first post review audit is necessary and can only improve the awareness of first post reviewers. But the algorithm responsible for the selection of the question/answer couple should take into account the technical length (or simply the length :) ) or the "fake" answer which will be presented to the reviewer.
Another way this can be done is by checking the reputation of the original answering person. In my case, Jerry Coffin has over 239K rep, so obviously the answer was great ! The algorithm should select accepted answer from low rep user (or even combine the two : low rep user with low length answer content) so that the answer is just good enough to be plausible as a first post.

Screen : enter image description here

7
  • 2
    How else would you have a known-good audit case, if you couldn't select from good posts by known users? Who else is going to open up the question in a new browser and not realize that the names are the only thing different between the review and the actual post?
    – Brad Larson Mod
    Commented Aug 18, 2014 at 22:27
  • @psubsee2003 Indeed I searched for similar question before posting mine in Meta StackOverflow but didn't thought someone could have addressed the same issue in Meta StackExchange so thanks. At least "the struggle is real".
    – m4rtin
    Commented Aug 18, 2014 at 22:39
  • 3
    @m4rtin the reason it was addressed on MSE as the question was asked when MSE and MSO were the same site. Commented Aug 18, 2014 at 22:40
  • 1
    @BradLarson My point is that a "known user" (say what ? 100K+ rep ? 200K+ rep ?) indeed knows SO very well and you can feel it in its answer (good formatting, technical accuracy, quotes from trusted sources, etc etc). And anyone that is not new to SO knows that fresh new user doesn't write like that, at least not on their first post. So this thing smells from the beginning. And who would open the question in a new tab and not realizing the name are different ? Nobody : I saw that the answer I was reviewing was posted 2h ago and the accepted answer was posted Aug 14 at 5:14, so...
    – m4rtin
    Commented Aug 18, 2014 at 22:41
  • 1
    I suppose my question would be, "Why would you care who wrote it? Couldn't a stellar answer be written by someone who just came onboard to the platform? Why look at who wrote it instead of what was written?"
    – Makoto
    Commented Aug 18, 2014 at 22:54
  • 3
    "...I just failed my first review audit..." - that's one of many (speaking from experience). If you feel the question is a poor audit question (emphasis: "audit"), then double vote on it. For example, cast two down votes - the first will down vote the question and remove the question from the audit pool; and the second will restore the question's score. It ensures you don't penalize the person who asked the question. If you feel the question is a poor question, then single vote on it. For example, cast one down vote - it will drop the question's score and remove it from audit pool.
    – jww
    Commented Aug 18, 2014 at 22:56
  • @Makoto I've been moderating on other forums and old habits die hard ("who wrote that ?"). But SO is not a forum so you're right in the end, no matter who wrote that stellar answer, I shall only consider its content ! (+@jww Hum good to know, thanks.)
    – m4rtin
    Commented Aug 18, 2014 at 23:08

1 Answer 1

13

Please take a look at Review Audit shows an answer that's duplicate on Uber-Meta.

Basically, you did everything exactly right except you failed to recognize the audit.

To quote myself in my answer to the question I linked

The only "failure" in your logic was that if it was someone trying to score free rep and copying & pasting the answer, you would have seen both the upvoted original and the 0-score duplicate answer under the question. It is a misunderstanding that I've seen other new reviewers make, but posts in the review queues are posted before reviewing, not after, so you are reviewing questions & answers as they appear live on the site (except for audits, which are tweaked slightly to disguise the fact that they are audits)

Basically there is not supposed to be anything tricky about audits. The link will take you to the exact answer you are supposed to be reviewing, not a random spot in the post* so if you click the link and see the same exact answer that has a lot of upvotes and is written by someone other than the user identified in the post, then you can be 100% confident that it is an known good question audit, and you can go back and upvote the post to pass the audit.

* - There is 1 exception. A known bad audit often will be a deleted post, so if you do not have 10K rep and the ability to see deleted answers, then the link will take you to the top of the question (so if you click the link in the audit and go to the top of the question, then it is very likely an audit of a deleted post).


As for your feature request, it is completely unnecessary. You should evaluate all reviews (and audits) on the perceived quality of the answer, not who wrote it. If you just recognize the signs of an audit, then you don't need to worry about filtering audits based on who wrote it or what their rep is, in fact that should make the audits easier as you should be able to spot the fact that it is very likely an audit on the first read.

6
  • 1
    I'll accept this answer (didn't see it was already you back on this Uber-Meta thread) and recognized I failed to see it was an audit. I treated it a a genuine answer and "over-thought it" as you said. Thanks, point made.
    – m4rtin
    Commented Aug 18, 2014 at 22:59
  • 2
    Well, but there is something tricky about audits: they lie to you about who posted an answer. I'm still ambivalent as to whether there's any point to it. Commented Aug 19, 2014 at 22:57
  • 3
    IMO that is tricky: an actual copy/paste first post is a terrible first post; how can a sub-10k user differentiate "actual good answer w/ fake name" from "copy/paste first post that's since been deleted"? Commented Aug 19, 2014 at 23:29
  • @MichaelEdenfield no one said that copy and paste answers aren't bad answers (no matter if it is a first post or not). That isn't part of the discussion. Audits are designed to be easy to spot if you are paying attention. So you click the link, go to the answer. See it was posted by someone else and realize it is an audit. Commented Aug 20, 2014 at 0:05
  • @KubaOber audits are design to do nothing more than catch users who aren't paying attention when reviewing and just robotically click "Looks Good" or upvote everything while reviewing in an effort to get badges. They are hardly perfect but they do serve that purpose. You will get bad audits that are not obvious, but more often than not you should be able to spot the audit as an audit and handle it appropriately to pass the audit. Commented Aug 20, 2014 at 0:07
  • @iStimple how can it be cheating when there is nothing to win. The entire point to reviewing is to fairly review each post that you are given or decide to skip it if you aren't sure you can review it fairly. Clicking through to the actual answer can be extremely helpful when you see a borderline post because seeing the entire comment discussion, and the question and the other answers can often provide context for the answer that could explain why the particular answer is good or bad. Commented Aug 20, 2014 at 0:09

Not the answer you're looking for? Browse other questions tagged .