-6

Background

Recently a SO user graciously provided his opinions and tips on how EF can improve performance as a reviewer. As part of his participation in encouraging EF to improve, SO user used his powers to suspend EF from conducting reviews.

The stated rationale for implementing the suspension: "You appear to be blindly accepting every edit you see".

This rationale is factually incorrect (and the SO user was gracious enough to admit this mistake after being informed of it) nevertheless, the review history still reflects a (misleadingly) high accept/reject ratio and it is easy to see how a reasonable person could make such a mistake after casual review of the published review history.

Problem

It is possible for a reviewer to use "skip" instead of "reject" as a means of bypassing questionable content that does not meet the quality threshold for acceptance. Nevertheless, this is not reflected in the review history, and that can give the (false) impression that the reviewer is not paying attention and "blindly accepting every edit".

Proposed Solution

Include "skipped" reviews in the published review history, along with "rejected" and "accepted" reviews.

Rationale

Including "skipped" reviews in the published review history will give a more accurate reflection of the accept/reject ratio that might otherwise look high upon superficial inspection of the review history.

7
  • 5
    I don't think you realize just how cluttered that would make some histories. I certainly wouldn't be interested in seeing pages filled with inaction - that's rather pointless to look at. They could include the number of skips in the reviewer stats on each item, though. That sounds much more reasonable.
    – animuson StaffMod
    May 2, 2014 at 15:14
  • 1
    //They could include the number of skips in the reviewer stats on each item, though. That sounds much more reasonable// Excellent suggestion. The basic premise is that a more complete picture of the underlying statistical data can help prevent an unfairly skewed appearance, for reviewers who use "skip" as a substitute for "reject".
    – dreftymac
    May 2, 2014 at 15:17
  • @animuson: I've retitled the post to more accurately reflect the purpose behind the feature request. Many thanks for your suggestion.
    – dreftymac
    May 2, 2014 at 15:19
  • 5
    To be clear, I did not ban you because you only had apparent approvals in your recent review history. That only drew the attention of users who flagged your reviews, and caused me to look into things a little deeper. Moderators are fully aware that some people skip many reviews and only approve obvious cases. It was only when I found specific reviews that never should have been approved: meta.stackoverflow.com/a/253188/19679 that I chose to ban you from review and warn you about this. Could I have used better wording in the message? Perhaps, but the point was the specific reviews here.
    – Brad Larson Mod
    May 2, 2014 at 15:32
  • 1
    @Brad //To be clear, I did not ban you because ..// This post is just a feature request. No one is questioning your actions here, and you were deliberately not named in order to keep the feature request generic. If you have any issues you would like to discuss about prior actions, it seems reasonable to take the discussion to a more appropriate context.
    – dreftymac
    May 2, 2014 at 16:02
  • 2
    Sorry, my point was that the addition of a feature like this wouldn't really affect our decisions in regards to review, since that wasn't what led to a review ban here. We don't (nor does the system) ban people from review strictly based on review stats, but on specific reviews themselves. Review statistics that lie outside the norm may draw attention, but it's the reviews themselves that are the actionable elements.
    – Brad Larson Mod
    May 2, 2014 at 17:44
  • 1
    //Sorry, my point was that the addition of a feature like this wouldn't really affect our decisions in regards to review, since that wasn't what led to a review ban here.// As stated previously, no one is questioning your actions here, so the outside context of your motivations (or other potential actions by other moderators) are not really relevant. The primary issue is that an accept/reject ratio can reveal one part of the picture, but you yourself acknowledge that further statistics and inquiry are appropriate. This feature request simply acknowledges that fact.
    – dreftymac
    May 2, 2014 at 18:27

3 Answers 3

9

I think animuson hit the major point in his comment: this would really, really clutter up people's review histories, and it makes very little sense to mark "I chose not to do anything!" as an event on your profile.

"You appear to be blindly accepting every edit you see" sounds like a canned response to me. (I could be wrong, but my next point applies either way). Regardless, it of course doesn't actually mean literally every edit, and honestly I don't think the number of reviews you skipped is relevant here. If you skipped 100 reviews, correctly evaluated 3, and incorrectly evaluated 15, you deserve a review ban. That's 15 edits that shouldn't have gone through that are now displaying on the site. How many you skipped in the interim is totally irrelevant; there's still bad content out there because you weren't paying enough attention.

So no, I don't think this is a good idea.

P.S. "You" in this post refers to the generic "you" (ie. anyone who has this problem, not just the OP of this meta question).

1
  • Regarding clutter, this could be addressed by the traditional "search/sort/filter" approach found in many web user interfaces. This, however is secondary to the fact that statistically complete data could be presented without a cluttered user interface. Thanks for the feedback.
    – dreftymac
    May 2, 2014 at 15:23
6

It is possible for a reviewer to use "skip" instead of "reject" as a means of bypassing questionable content that does not meet the quality threshold for acceptance. Nevertheless, this is not reflected in the review history, and that can give the (false) impression that the reviewer is not paying attention and "blindly accepting every edit".

There is a big difference between clicking on "skip" when you find an edit questionable and always clicking on "skip" instead of "reject". The first one should result in a quite normal skip/reject ratio, with a few skippeds. The second one results in a review list of only "accepts", which is what you are talking about.

If you only accept and never reject anything, you aren't a good reviewer. The review system is a place where all questionable actions are listed, so people can quickly reject/indentify/remove the incorrect ones. The fact that people can also accept things is just a by-use of the review system.

Adding the skipped count of reviews will not add anything value to whether someone is a good or bad reviewer; the accept/reject ratio keeps the same.

It can only be usefull for the Community bot to be able to see the difference between a reviewbot that is only clicking on accept and a user which doesn't know how to use the review system. If it isnĀ“t done yet, I think that the skipped count should be added to this alghoritm, but I'm almost sure it already does.

4
  • //If you only accept and never reject anything, you aren't a good reviewer// This seems contestable, and a suitable basis for reasonable and considerable disagreement. Nevertheless, there is a difference between "never rejecting" and using "skip" as a substitute for bypassing questionable content. The latter is what is at issue in the OP, not the former. Nevertheless, thanks for sharing this perspective.
    – dreftymac
    May 2, 2014 at 15:33
  • 1
    @dreftymac but if you don't skip every reject, how can somebody claim that "you blindly accept everything"? If you do only skip questionable content, there will be a lot of non-questionable wrong edits left to give you a normal accept/reject ratio.
    – Wouter J
    May 2, 2014 at 21:09
  • Not sure we are talking about the same thing here ... if some content is unambiguously worthy of rejection (e.g., blatant vandalism) then it seems everyone can agree that "reject" is the appropriate action. However some content requires a judgment call, and reasonable people can disagree in those cases. It is the cases where reasonable people can differ where "skip" seems appropriate (e.g., the reviewer is tired, or outside the realm of his or her expertise).
    – dreftymac
    May 2, 2014 at 21:15
  • 2
    @dreftymac that exactly what I am saying. Except you claim that this results in a very high accept/reject ratio, while I think it doesn't make any significant difference on that ratio.
    – Wouter J
    May 2, 2014 at 21:22
1

Skipped editions really should just not count in the statistics.

Recently, I was bashed with this:

You appear to be blindly approving every edit you see ...

which is not true. The truth is, that those edits I don't understand (areas not of my competence) I skip, while the edits in the areas I do understand all have seemed OK.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .