33

I've noticed a consistent trend in the Suggested Edits queue wherein I'll see a proposed edit that looks fairly bad, click through to look at the context, and come back to vote against only to see that it's been approved. When this happens, nearly 100% of the time the three approve votes all came from users with less than 5k rep, who are presumably new to SO. The edit audits don't really help because the audits are so blindingly bogus that only the most oblivious robo-reviewer would get caught.

I propose requiring that at least one of the votes to approve an edit come from a user with a high reputation (maybe 7.5k or 10k), and perhaps additional logic when several 3k users approve while Jon Skeet rejects. The Suggested Edits queue is always so small that this wouldn't clog it and would help squash some of the lousy proposals (and perhaps even improve their signal-to-noise ratio).

21
  • 11
    This is assuming that high-rep users will actually see and vote for the edit... which means it could easily be in the queue indefinitely.
    – deceze Mod
    Commented May 8, 2014 at 8:49
  • 6
    @deceze I check the edits queue regularly, and it's usually very small or empty. Commented May 8, 2014 at 8:52
  • 11
    I've been thinking for a while now about requesting a raise in the rep barrier for approving suggested edits, this looks like a happy medium. +1
    – user247702
    Commented May 8, 2014 at 9:10
  • 8
    How about this - the rep of all approvers must total > some number, and must average > some other number. Commented May 8, 2014 at 10:55
  • 2
    meta.stackoverflow.com/questions/252856/… contains my thought on this problem. Commented May 8, 2014 at 12:36
  • 3
    @BobJarvis I like the total rep idea. Imposing a lower limit on the average could cause an obviously good edit to go unapproved, despite high rep approvals, because a large number of low rep users also approved it, dragging down the average. Commented May 8, 2014 at 13:43
  • Having 5 4.5kers vote to approve isn't the same as having 2 3kers and a single 10k+. Commented May 8, 2014 at 13:55
  • 6
    More than high rep, we need people with high success rates and previous votes. Commented May 8, 2014 at 15:54
  • 4
    I'd like some stats on this before I'd agree that this is a good idea. Personally, I think we just need to get better audits - Add "too minor" audits to review queue Commented May 8, 2014 at 19:22
  • 8
    I doubt 7.5k or 10k reputation has to do anything with quality of review . So I think it's better to add measuring of review fails (we have such mechanics right now, when system is proposing fake edits). If it's too high - then restrict user from reviewing for a while.
    – Alma Do
    Commented May 8, 2014 at 19:33
  • 1
    @AlmaDo That already happens - users get banned for failing too many review audits. Commented May 8, 2014 at 19:40
  • 19
    The fact that i have high rep doesn't mean i make a good reviewer.
    – Kevin B
    Commented May 8, 2014 at 19:41
  • 2
    A while ago I proposed that we take into account flag weight (I'm aware this isn't displayed to the user, but I believe it still exists) more instead of rep as that is probably a better indicator that someone has taken the time to review correctly.
    – Jack
    Commented May 8, 2014 at 20:13
  • 5
    @KevinB high rep doesn't necessarily mean you'll be good at anything, but we give high-rep users all sorts of different, powerful privileges anyways, because it's a good-enough approximation of how trustworthy a user is. It's not perfect, but it's good enough. I don't see how restricting edit-review privileges to fewer users could be any worse than the current situation, where it's given out to too many un-trustworthy users.
    – user456814
    Commented May 9, 2014 at 4:15
  • 2
    More than a few times I've said to myself, "How the hell did this edit get approved?!" Most of the time what that happens, it's very late at night (in my timezone...) Commented May 9, 2014 at 5:06

2 Answers 2

5

I just went through the suggested edit queue, and then backed through it to check the rep of the other reviewers. It's true that all of the approve votes came from users with less than 5k - actually less than 3k - rep.

However, all of the reject votes also came from users with less than 3k rep.

So it looks like most of the reviewers have "low" rep, whether they approve or reject. That means there's no reason to think that "low" rep reviewers do any worse than "high" rep reviewers.

On the other hand, this also means that there is reason to believe that if you required high rep reviewer votes, the queue might get clogged. That doesn't seem like a good thing to risk.

I think a better solution would be to have the invalid edit audits require actually reading through the edit, instead of essentially all being cases of vandalism.

2
  • 3
    Would the mix of reviewers change if edits did not get approved so fast? Commented Jul 10, 2014 at 3:43
  • 1
    @PatriciaShanahan: I can't speak for the high-rep users, but after only a couple of months, I can say that I'm already partly frustrated by how the edit review queue works. I still do reviews, but it seems likely that I'll get tired enough of it that I'll stop completely at some point. So if high rep users stopped reviewing, I can completely sympathize. Commented Jul 10, 2014 at 8:03
3

I'll give a similar suggestion inspired mostly from the comments: Require a minimum total rep, with a minimum number of people. Also require a minimum total rep held by the lower half (in rep) of the reviewers, so that:

  1. Higher rep people aren't going to guarantee its passage regardless of others' thoughts, and
  2. Lower rep people aren't able to plague the site with bad approved edits.
3
  • 3
    reputation is not a good indicator whether you a good or bad reviewer.
    – user2140173
    Commented Jul 10, 2014 at 7:16
  • 4
    How would you judge aptitude at reviewing edits? Especially algorithmically, because this will have to be computer-enforced.
    – Claudia
    Commented Jul 11, 2014 at 3:46
  • data.stackexchange.com can give you some data about the suggested edits approvals and rejections.
    – user2140173
    Commented Jul 11, 2014 at 7:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .