46

After roughly two weeks of smooth reviewing I failed another audit and got me a review ban. It was one of those posts that I find quite borderline: An answer with a small list of short things relevant to the question and a "further read here" link. It is certainly not a good answer, and probably deserved downvoting, but I would not call it "delete-worthy" as it seems to be a genuine attempt at summing up what the link describes. This is after all what we tell the people to do in link only cases: we want the core of the link content here, and the link is there for reference and "more in depth reading". Removing the link should give an answer too (and in this case I would argue it would have been an attempt to answer, though a bad one that should have been downvoted).

Now I find these disagreements between me and the community somewhat annoying, and it kills the motivation to do reviews. Then I thought, maybe thats exactly what they should do.

I seem to disagree with the community in that I don't think bad answers should be deleted, but rather downvoted. Based on that, should I better step away from reviewing and let only those people do it that have a better agreement with the community on such things?


By popular demand, here the review item I failed last on. Note that the purpose of this question is not to discuss whether this is a good review item or not, or whether I should have identified it as an audit and played along, or whether I should have agreed with its deletion or not. I have anyways already forgotten how that answer looked like and can't see it anymore (so far for the "stop look and listen" part).

The purpose is mainly stated in the title: Assume you sometimes disagree with the community. Should you do reviews?

Review screenshot:Failed review


Because some people wondered what my outcome of this is: Combining all the comments and answers below, I think that as long as you are not 100% (or almost) in line with the community, you should not participate in reviews. Failing audits is a strong indicator that you are not, and it is not a loss if some people that only agree in 99% of the cases don't do reviews, the community is big enough.

12
  • 4
    If you can, could you post the audit you failed? Sometimes the audits can be borderline or in a few cases, downright wrong. Commented Jul 11, 2014 at 11:07
  • 8
    @Qantas94Heavy: It is a deleted answer, so would only be visible for 10k+ users, which would reduce any target audience if we start to discuss it. It is also not the point of this question to dispute a review audit, but more general to assess that if you have a different opinion than the community about some stuff.
    – PlasmaHH
    Commented Jul 11, 2014 at 11:22
  • 8
    The purpose of asking for the audit was so that people can see how far off the general community's view on reviews you were. Of the few reviews I've taken a look at, they seem alright to me (at least within the normal range of community opinion). Commented Jul 11, 2014 at 11:32
  • 1
    Could it be possible to see what others have approved or rejected ? Maybe my interpretation has value, but I think the value would increase if I could know other community member's interpretation ... Commented Jul 11, 2014 at 13:52
  • 2
    @PlasmaHH if relevant a 10k can view and post a screenshot of the item in question. Commented Jul 11, 2014 at 14:06
  • @Qantas94Heavy even if it was wrong does bringing it up here actually fix anything for the future? I'm partially thinking about a recent audit I failed for an answer that looked good but was apparently downvoted/deleted because the answerer mis-guessed what the questioner was trying to do. I didn't do anything at the time beyond grumbling because I thought the audits were randomly selected and not individually fixable. Commented Jul 11, 2014 at 14:12
  • 12
    "... it kills the motivation to do reviews..." - +1. I pretty much gave up on queue maintenance after I failed an audit and asked a question about it. It wasn't the failed audit that irritated me; rather it was the personal attack that followed. Who needs the aggravations....
    – jww
    Commented Jul 12, 2014 at 8:30
  • 6
    +1 I quit reviewing for a day as soon as I get an audit instead of real post. But audits help to fight against robo-reviewers. They don't. Crappy reviews, lousy approved edits still take place.
    – vaultah
    Commented Jul 13, 2014 at 18:54
  • The answer didn't answer the actual question, it was just a generic list saying you can insert an element into the array at the array's start, end, or anywhere else. That does not "look ok". Deletion reason aside, it was a bad answer to start with.
    – Cerbrus
    Commented Jul 14, 2014 at 8:54
  • @Cerbrus: As I stated above, my opinion is that is was a very bad attempt to answer the question, which deserves downvote into oblivion, but is not a reason deleteworthy. For the spam flag appearantly present, that link returns a 404, so hard to tell from my end. But as also stated above, the community has their opinion, and I have my opinion on that question, and the goal isn't for either side to convince the other, but to recommend an action in case of such disagreements.
    – PlasmaHH
    Commented Jul 14, 2014 at 9:01
  • Spam or not, it was still a link-only answer (Which is a valid deletion reason). The list was as generic as it could get.
    – Cerbrus
    Commented Jul 14, 2014 at 9:07
  • 3
    I failed an audit for voting down a question I thought was rubbish but which the community apparently liked. I gave up reviewing and went and voted the question down for real. How this makes me a robo reviewer or indicates reliably that I'm not paying attention, I don't know. Votes are supposed to be secret and conscience-based on one's own interpretation of usefulness so how I can fail on that? I also consider the message printed to be extremely rude. I'm providing free labour and you're gonna shout at me?! Exactly how many kittens got hurt??? So, yeah... Commented Nov 12, 2014 at 20:18

5 Answers 5

18

I think that question entirely depends on the set of audits you failed. The question you mention above sounds like it was deleted for being alleged spam - maybe it was 100% clearly spam and your opinion massively differs from the community consensus, but more likely it could be that the user had posted half answers across a number of posts trying to plug their product/website and you'd never know from looking at a single post in review.

If your views really are largely misaligned from those of the community (or at least, the janitorial community) perhaps you should stop, after all, the point of reviewing is to help keep SO clean and heading in the direction we want it to. If 90% of us are tunnelling west, you're not going to get far trying to chip away to the east.

The queues are work at the best of times, if it's causing you frustration it's definitely not worth doing.

I've never failed a suggested edit audit, but the queue is outrageously frustrating for exactly the reason you mention - I will reject, 3 other people will approve, over and over and over and over. Hence why I left the suggested edit queue being many months ago.

7

Most of the audit questions I've ever seen are pretty clear cut, often egregious violations of community standards. I'm sure there are a few borderline cases, but if you failed several audits then it's likely that you're out of step with the community.

If you find it difficult to understand what the general consensus is, or if you're unwilling to help implement it, then reviewing may be a waste of time and an exercise in frustration. A more effective tactic would be to identify a number of cases where you disagree with the current practice and make your case for changing the standard in a meta post.

I seem to disagree with the community in that I don't think bad answers should be deleted, but rather downvoted.

Don't stop trying to help just because of a rough patch. Instead, take a little time out and look at all the audits that you failed, not just the one that resulted in a ban. Try to learn what the community thinks is wrong with them and figure out where/why you differ (sounds like you're already doing that). Then decide if you can contribute productively to the review process. I think getting people to do that is the real goal of the ban.

While reviewing, you always have the option to skip those reviews that you're not sure about, or where you know you disagree with the consensus.

8
  • 6
    " not just the one that resulted in a ban." The problem here is that this is almost impossible to do, since they are buried in the review history and hard to find, if there at all. For things like that single one I failed in the last two weeks that lead to the ban, it is displayed ni the ban message, but also there you can not see it because it is a deleted answer.
    – PlasmaHH
    Commented Jul 11, 2014 at 15:32
  • 1
    We might be able to help more if you posted the link to the question, even though not everyone will be able to see it. There's a disproportionate number of 10K users here in meta.
    – Caleb
    Commented Jul 11, 2014 at 15:40
  • I will do that in a while, but I would rather like to have people discuss for a moment about this in the general case, as this isn't so much about why that answer was deleted, but more about the general implications of having a harder time due to disagreeing with certain things in the community. Likewise I am happy that we don't have tag-only-edit reviews, but as soon as we get them I would skip instead of reject to not fall into an audit. This is much harder to do for known disagreements in VLQ so it might be better for everyone to not do reviews anymore.
    – PlasmaHH
    Commented Jul 11, 2014 at 15:45
  • 3
    You pass 12 million audit reviews and fail at 2, maybe even questionable ones. Bottom line is, you invest a lot of work and valuable time to do high-quality reviews and fight the robo reviewers, earn nothing but a few measly badges from it, and finally get banned because you 2x were not in line with what was the expected "right" action. That's how SO motivates people to do more reviews. Seems to work ... not.
    – JensG
    Commented Jul 12, 2014 at 11:47
  • 3
    @JensG except you don't get banned for "two [consecutive] audits after passing 12 million of them". Personal experience. Commented Jul 13, 2014 at 5:36
  • 3
    @JanDvorak: Except, you do, sometimes. Personal experience.
    – PlasmaHH
    Commented Jul 13, 2014 at 18:04
  • 2
    I just got a ban last week for my first failed audit in a couple months. I try to review often, and have gotten much better at being in line with the community. Looking at the question, I should have known it was an audit, and just agreed with it (it had a score of 10), but I honestly thought it should have been deleted cause it was a link only. (Like @PlasmaHH, it's lost to the queue now)
    – krillgar
    Commented Jul 14, 2014 at 1:54
  • 2
    Yea, this audit was pretty clear-cut.
    – Cerbrus
    Commented Jul 14, 2014 at 8:59
5

I find when I fail an audit, it's often because I'm reading more carefully than past reviewers.

One time someone posted an answer with nothing but a link to a tool they made and a short description. It was both a link-only answer and spam. However, the question was essentially asking if such a tool existed and this post answered the question as well as any answer could. If reviewers were careful, they would flag the question for being off-topic and leave the answer be. However, this wasn't the case. The question came to me in an audit and I failed it upon deciding there was no action needed.

To me, the discouraging part is not failing the review, but seeing such valid answers with several commenters raising their noses with "Link-only answers are discouraged on SE...". It looks as if they are willing to spend time crafting a well-structured, full-sentenced comment with hyperlinks but don't have the interest in simply reading the whole question and answer.

The best I can do is choose to not let these instances bother me too much. I can read and understand the rules of moderation to know when I'm right and the audits are wrong. Enough people are careless here that I know that my presence is helpful to the review process. It's the same reason why I try not to skip reviews. If I'm having a difficult time handling it properly I'm sure the next person is going to give it less attention and is likelier to do the wrong thing.

9
  • 2
    There are good reasons for raising ones nose at link only answers, not least because links break. That doesn't make the answer spam, but during the review, you might like to edit it into shape, so it passes further reviews easily. Commented Jul 13, 2014 at 9:24
  • 2
    @Oddthinking: Depending on the queue, trying to edit something in a review makes you fail the audit.
    – PlasmaHH
    Commented Jul 13, 2014 at 20:11
  • 1
    @PlasmaHH: Wha? That's crazy. Which queues? Commented Jul 14, 2014 at 2:23
  • 1
    @Oddthinking: I don't have any examples at hand, but it probably is similar to commenting: meta.stackexchange.com/questions/188885/… meta.stackoverflow.com/questions/256727/… meta.stackexchange.com/questions/179651/…
    – PlasmaHH
    Commented Jul 14, 2014 at 8:35
  • 1
    @Oddthinking are you saying to edit the link-only answer into shape? In general, I don't think this is the job of the reviewer and in particular, this answer was perfectly in shape upon review.
    – drs
    Commented Jul 14, 2014 at 16:09
  • @drs: I'm confused by your comment. The job of improving answers is the shared responsibility of the entire community, especially anyone with enough privileges to review. The answer in question was not in shape because it was a link-only answer. Commented Jul 14, 2014 at 17:49
  • @Oddthinking Reviewers editing a link-only answer into shape is a lot more work than the +2 it receives, and it doesn't teach the OP how to form proper answers. Furthermore, it requires the review to have a good deal of experience in the subject of the question.
    – drs
    Commented Jul 14, 2014 at 18:40
  • @Oddthinking, I guess we disagree about what answers are in shape for questions that ask for a link.
    – drs
    Commented Jul 14, 2014 at 18:41
  • @drs: Looks like we disagree on a lot of the fundamentals. Commented Jul 15, 2014 at 3:36
1

I'd say take the time off and resume later. You're part of the community and your interpretation has value. Selecting a dissenting choice in just a couple questions seems pretty minor to me. I'd only consider quitting if you consistently fail future audits (by which time I assume the system will give you a long ban, anyway).

4
  • 5
    I think the ban time is increasing. 2 day ban, fail one audit, 7 day ban, fail one audit, 30 day ban (don't know whats after that)
    – PlasmaHH
    Commented Jul 11, 2014 at 13:48
  • 3
    I don't think the rule is that simple. I have failed a few audits, and never got any kind of ban. If you only failed a couple of audits where the decision is borderline, or if you ran into an example where the audit system has known problems (like I did here: meta.stackoverflow.com/questions/262231/…), I wouldn't worry about it too much, and continue reviewing after the ban if it's something you still want to do. Commented Jul 11, 2014 at 14:00
  • @PlasmaHH the longest ban is 30 days, and IIRC the window used for deciding if how long the ban will be is short enough that your next ban after a 30 day ban would only be 2 days again. Commented Jul 11, 2014 at 14:04
  • @RetoKoradi: exactly this is what I am trying to find out: if I still want it to do.
    – PlasmaHH
    Commented Jul 11, 2014 at 15:40
0

It's not like the majority is always right. Sometimes minorities even have a better idea. On the other hand if people just step back, who knows what the majority wants then - if people just go away the statistical basis for knowing how the community judges gets thinner and thinner.

You should continue and give your best. Everyone here is part of SO and only we all together are SO.

There is a reason that a single failed audit is tolerated. Rules cannot be formulated so clearly and exhaustive that they solve every case. It's always partly gut feeling.

If however your motivation is killed by several failed audits in a row - better stop for awhile. Nobody is forced to do something he/she doesn't like.

Summary: Stop as long as you do not feel good about it, but do not let yourself be influenced so much by the perceived opinion of others. As contributor you have a right as well to define what SO should be.

(Actually even with the audit that you failed something could be wrong. Maybe indeed it is a border case and many fail it. What to do with controversial audits may be another discussion.)

4
  • 1
    Part of the problem seems to be that sometimes "several failed audits in a row " means one per two weeks, or in other numbers, one out of 500 or so reviews is a failed audit, and who knows how many passed audits this includes.
    – PlasmaHH
    Commented Jul 13, 2014 at 9:42
  • @PlasmaHH I don't know enough about the inner workings of the failed-audit-automatic-ban system only what everyone says that a single failed audit is harmless. I can just assume that they come fairly regularly and that there is some intelligent thresholding, like average of last X audits above Y is a ban or so. Maybe the system should display more information when a ban is imposed how many passed/failed audits there were before. Commented Jul 13, 2014 at 9:53
  • From what I heard here, once you got yourself a ban (which gets really fast if you tend to skip audits instead of getting a success), you are in a spiral of death. After a ban is lifted, just a single failed review gets you into a ban again for the next month or so. And even after that month, the amount of failed reviews necessary for a ban seems to be rather low.
    – PlasmaHH
    Commented Jul 13, 2014 at 14:00
  • @PlasmaHH Okay didn't know that there is not much forgiveness inbuilt. I guess then you have to take the ban as a sign that the system doesn't fancy your review capabilities. Short of discussing the audits you failed (maybe they are faulty) or changing the ban requirements (maybe they are too tough) there is nothing you can do then. Commented Jul 13, 2014 at 18:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .