23

I failed an audit but I think it might be unjustified. I voted "Looks okay" but it told me the answer was deleted because of spam or something offensive. I can't see what's wrong with this answer.

https://stackoverflow.com/review/low-quality-posts/31729560

Audit

13
  • 2
    My guess is that the author posted an extensive number of links to their own site. Hard to say now that the user is deleted.
    – BDL
    Commented May 9, 2022 at 12:57
  • 14
    Looks OK apart from the link at the bottom. Unfortunately it's hard for normal users to differentiate between <link to benign site you haven't heard of before> and <link to site that this user is spamming>. Best approach when you see a link to a site you've never seen before is to check out the full question page and the user profile to see if you can narrow down whether it's spam or not. Commented May 9, 2022 at 12:58
  • 7
    looks ok to me the link is a reference.
    – nbk
    Commented May 9, 2022 at 13:12
  • 8
    @nbk the linked site seems to be very modest quality: certainly not an authoritative "source and reference". Most likely the answerer had been trying to promote their site. Commented May 9, 2022 at 13:48
  • 4
    @snakecharmerb that makes sense to me. Nevertheless I think audits should at least be clear to the reader paying attention. I don't think many people would mark this as spam.
    – jrswgtr
    Commented May 9, 2022 at 13:50
  • 1
    Does this answer your question? How is this answer spam? Commented May 9, 2022 at 13:52
  • 10
    That answer is really strange. Why would someone need an external source to write such simple code? And how come that answer uses the exact same text numbers as another answer (Array Sum & Average in Java), but no one else uses them? Could it be that the answer is just a re-written code of that other answer in order to post the link to a site? This all looks a bit weird.
    – Tom
    Commented May 9, 2022 at 13:57
  • 22
    @Tom that's probably the reason the answer was deleted; the problem is, without all that additional context, a reviewer wouldn't be able to tell, and shouldn't be asked to as a review audit
    – Gus
    Commented May 9, 2022 at 14:08
  • 4
    @Gus The review provides that context. The answer text is there, the link is there and the other existing answers are there using the "Other answers (8)" tab. So, what context do you miss?
    – Tom
    Commented May 9, 2022 at 14:30
  • 5
    Sure, you can go hunt it down, with available information, but the audit should provide the context in the audit. So, it's missing all the context that you have to go independently dig up, on some other page @tom
    – Gus
    Commented May 9, 2022 at 14:31
  • 4
    @Gus It has all the context in the audit. It obviously can't crawl external links and copy their content into the review as well and that's the only missing thing. So, again, what context do you miss? Also, I don't follow this "a reviewer shouldn't be required to actually do review" argument. When one doesn't want to do that, then they just can ignore the review queues and it would be fine.
    – Tom
    Commented May 9, 2022 at 14:33
  • 1
    @snakecharmerb i doubt that, i believe that he posted the link more than once, but a mod can tell us more if they wanted
    – nbk
    Commented May 9, 2022 at 15:32
  • 1
    A similar meta question (3 hours younger). Commented May 10, 2022 at 11:22

2 Answers 2

32

Unfortunately, you can't see the wider context, beyond the question, but that link is the reason why the account was destroyed as a spammer.

The account was one of three that were posting nothing but low-quality answers that all included a 'reference' to the same site. In other words, they were not here to answer questions, they were here to seed links to their site.

Generally speaking however, there are a few things I hope our community is watching out for when they see a post with a 'reference' link like this:

  • If the reference link is there to point to documentation or an article that helps expand on the answer, then that's great, but it should be a reputable resource. If an account is posting links to a single, often new or obscure site, it is probably spam instead.

  • Was the linked post created around the time of the answer? That'd be another red flag indicating that the answer could have been motivated by a desire to promote the linked post, more than to help solve the question.

  • Another red flag here is: the code is, from a technical point of view, very, very, very simple. Why would you need a reference link for that kind of post?

For this audit you wouldn't be able to verify the account any more or check if the domain name was used in other posts. The fact that there are no posts at all on Stack Overflow can be another signal, however, that the link is obscure. If it was reputable, you'd expect more links.

If it isn't an audit, and you'd find that only specific accounts are creating posts with similar links, please let us moderators know with a custom flag, asking us to investigate if the users are linking to the site in good faith.

Because these posts may be harder to recognize as bad content, I've pulled them from the audit pool.

19
  • 26
    This is why we shouldn't use spam as audits on the Low Quality Review Queue.
    – Braiam
    Commented May 9, 2022 at 16:46
  • 12
    @Braiam: the vast majority of spam posts in the LQ review queue make excellent audits. Lets not let a few harder ones spoil a good thing. Commented May 9, 2022 at 17:45
  • 5
    The vast majority is wrong. The queue isn't a place to evaluate spam. It's not a few harder ones, they shouldn't be there at all. Spam is for the first question/answer queues.
    – Braiam
    Commented May 9, 2022 at 18:01
  • 11
    @Braiam: numbers please. If you are making such claims I'd love you to back that up with some proof. Commented May 9, 2022 at 18:02
  • 4
    Further, why shouldn't LQA reviewers be on the lookout for spam? Spam should be flagged in any queue it's seen in. Reviews indicating that spam content is not problematic are always wrong, regardless of the queue.
    – Ryan M Mod
    Commented May 9, 2022 at 19:15
  • the answer looks still ok, i gueesed it in my comment right, but that was after in knew that t was a spam flag that removed it, but it is for newbies and oldies impossible to guess that that is spam
    – nbk
    Commented May 9, 2022 at 19:44
  • 7
    @Braiam: I pulled up all the audit posts used in the low quality queue for the last 7 days, removed the 'positive' reviews (posts that should not be deleted), and found 5 posts for this specific spammer. Outside of those 5 there are 116 unique posts that are just about all spam posts, (we filter out RA posts due to the risks of exposing reviewers to extreme materials). Those 116 posts are, in my opinion, very clearly posts that should be deleted. These are highly successful audit posts that the majority of reviewers manage to pass. Commented May 9, 2022 at 20:13
  • 7
    @Braiam: In those 7 days, out of 9 failed audits, 4 were for posts by this spammer, so I pulled all such posts from the audit pool. I see no evidence whatsoever that, as you claim, the 'vast majority is wrong', those other 5 audit failures were by users that can’t have been paying attention to the post under review. Commented May 9, 2022 at 20:19
  • 4
    My single data point could beat up your single data point! Commented May 10, 2022 at 5:28
  • 1
    "Is the post perhaps copied, verbatim, from the linked resource?" Well, I wouldn't encourage anyone to click on spam links in order to check that...
    – Lundin
    Commented May 10, 2022 at 9:36
  • @Lundin: not all links are spam links. We don't always know something is a spam link until you check the link. Commented May 10, 2022 at 9:59
  • 4
    @MartijnPieters My point is that in this case we know that something is a spam link. Or at least you did when you deleted this. Therefore we shouldn't enforce other users to go click on a known spam link.
    – Lundin
    Commented May 10, 2022 at 10:01
  • 2
    @RyanM because, I don't know if you have tried the queue recently, but you are supposed to "Identify, then improve or delete low quality answers." by doing one of three actions: Looks Ok, Edit, Recommend deletion. Notably there's no "flag as spam" there. So you are asking reviewers to stop their workflow to click outside the queue and do actions outside the queue. Audits are supposed to audit actions inside the queue, with the information that the queue provides.
    – Braiam
    Commented May 10, 2022 at 12:01
  • 3
    "If you are making such claims I'd love you to back that up with some proof" how can I back proof without internal insight of information I'm not privy to? I would need not only all the post deleted as spam, also need all the review audits and the results of such audits, but also experiments with several groups of users using different levels of information presented to the queue. Or, you know, I could just pull up all the post on any meta about users complaining that they failed an audit that was spam and in the LQRQ.
    – Braiam
    Commented May 10, 2022 at 12:04
  • 1
    BTW "Have you actually counted how many were actually correct with their complaint?" all of them are correct. The answers weren't NAA's or VLQ, which is what that queue is supposed to evaluate. They were spam. So I don't need to count.
    – Braiam
    Commented May 11, 2022 at 14:19
16

Now if the audit system was working in a sound manner, it should automatically remove anything manually deleted by a moderator from audits. In this case only moderators see the greater picture and although the post is fishy, it's not an obvious cause for deletion.

Sure, the answer is a bit fishy and there are ways to tell that this was manually deleted spam - just visit the original post to tell - but if we require that from reviewers then why aren't we given a link to the original post during review?

We shouldn't force or expect reviewers to go click on possibly malicious spam links on an already deleted post in order to review something. Or to keep a tab on "possible spammer URLs" for that matter, that is surely not something that human reviewers should keep track of.

You did no obvious mistakes here; you are simply getting punished for doing boring unpaid volunteer work for a private company, when you could have done funnier things in your free time.

18
  • The posts were removed automatically because we spam nuked the account. Only later did I manually re-delete the posts precisely because that removes the posts from audits. Commented May 10, 2022 at 10:18
  • The posts that I removed from the audit post pool were troublesome not because the link is particularly dangerous, it's just a low-quality online ad vehicle that the owners wanted to drive traffic to. There really was no harm in visiting those links by a few reviewers. The issue was more that the posts were not good audits for catching roboreviewers. Commented May 10, 2022 at 10:20
  • 7
    @MartijnPieters Not harmful in this specific case. But next time it could be some site with malware/phishing/disturbing content. Requiring reviewers to click on such links is a bad idea. It would be yet another reason to refuse doing reviews, in addition to the broken audit system repeatedly accusing reviewers incorrectly.
    – Lundin
    Commented May 10, 2022 at 10:25
  • We'd be happy to remove posts that are problematic from the audit pool, if someone flagged for that or posted here on Meta. In the majority of audit posts, there is no need to click on any link as they are crystal clear even without following the link. There are loads of late answers that are in a middle ground, like stuff that is completely copied from the source: <link> at the bottom, and there really is very, very little risk here of any such sites being actively malicious. Commented May 10, 2022 at 10:33
  • The audit system is otherwise not broken. The vast majority of audit posts do their job just fine, there is no 'repeatedly accusing reviewers incorrectly', there is an occasional wrong audit post. I gave numbers in the comments on my answer, and that there were 4 posts is an exceptional spike because we spam removed an account with a large number of posts. Commented May 10, 2022 at 10:35
  • 1
    Take into account that if you get your perception about the number of bad audit posts from people complaining on Meta, that you are seeing the exceptions, not the norm. The vast number of audits are being passed successfully, or if someone failed one, because they really were not paying attention. Commented May 10, 2022 at 10:37
  • 3
    @MartijnPieters It's bad enough if the link leads to for example porn, snake oil medicine or fake news from dictatorship trolls. As for the broken audits, could it be that my experience from them does not come from meta but from the 5286 reviews I have done over the years? Admittedly I have mostly boycotted review work for several years now, because I don't fancy doing free, boring busy-work for the benefit of some US private company. But it's not like they have improved the audits during the past 5-10 years or so.
    – Lundin
    Commented May 10, 2022 at 10:52
  • 1
    Yes, you have done a large number of reviews. But, you stopped doing reviews a few years ago, and as a single reviewer you still don't get to see the full scope of how many audits there are, the frequency of bad vs good audits, etc. Also, the human mind is incredibly bad at gauging frequencies, as negative experiences stand out in memory disproportionally. The fact that you haven't actively reviewing anymore for at least 3 years now may well have skewed your perception even more. Commented May 10, 2022 at 11:30
  • 5
    @MartijnPieters Be that as it may, there's approximately one post about review audits on meta every 1-2 days. And as everyone who's worked with quality/support knows, for everyone actually speaking up about a problem/disputing an audit, there's some ten or so people who just suffer in silence. meta.stackoverflow.com/questions/tagged/review-audits meta.stackoverflow.com/questions/tagged/disputed-review-audits. This is frequent enough to question the existence of audits in the first place, as they are clearly distracting everyone away from actually improving the site.
    – Lundin
    Commented May 10, 2022 at 11:37
  • No, it's not every 1-2 days. If you look at this search, where I filtered out negatively scored and closed posts, you'll see it happens far less often, and most of these are not actually incorrect audits. This is out of hundreds of audits per week per queue. Commented May 10, 2022 at 12:39
  • 1
    @MartijnPieters Why would you filter out negatively scored or closed? These all ought to be tagged "support" if anything, since they are unique situations. Meta voting on support issues is senseless in general - what does it even mean? "I agree/disagree that you are posting a support issue"... riiight.
    – Lundin
    Commented May 10, 2022 at 12:43
  • because not every audit dispute is valid and some concern the same issue. I've used score and question status as a proxies for validity and duplication, because it is usually exactly why the post was downvoted and / or closed. Commented May 10, 2022 at 13:23
  • 2
    New to meta, but isn't the main point here that computers are really good at detecting a situation like 'OP has posted 10 other answers today with link to same site', whereas detecting the same thing is very laborious for a human. Shouldn't either (a) the posts not be in the audit queue, or (b) there should be a summary of any machine-detected spam score for links and specific reasons if available? Commented May 11, 2022 at 4:08
  • @MartijnPieters yeah, lets filter out the complains by a popularity contest. People downvote post even if they are valid concerns and close questions because they touch the same issue. And even then the list includes 300+ posts! There has to be something there to be so frequent. Also, just for fun the opposite query. Yes, there's some that are invalid, but there are valid on the bunch.
    – Braiam
    Commented May 11, 2022 at 14:24
  • 1
    @Wildcard I personally have no problem with moderators having an opinion too in things, they are still community members and not company employees. As for "long history" I have no clue about what that's supposed to mean.
    – Lundin
    Commented May 12, 2022 at 6:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .