Skip to main content
replaced http://meta.stackoverflow.com/ with https://meta.stackoverflow.com/
Source Link

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
  
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
  
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign.
  
That's all these numbers are - signs. A fast reviewer is not necessarily a bad one, but most of us cannot review correctly at breakneck speed. A reviewer who uses Skip a lot is not necessarily a good reviewer, but it does suggest they know when to withhold their judgment.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editorsFAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
  The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
  The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign.
  That's all these numbers are - signs. A fast reviewer is not necessarily a bad one, but most of us cannot review correctly at breakneck speed. A reviewer who uses Skip a lot is not necessarily a good reviewer, but it does suggest they know when to withhold their judgment.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery. 
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for. 
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign. 
That's all these numbers are - signs. A fast reviewer is not necessarily a bad one, but most of us cannot review correctly at breakneck speed. A reviewer who uses Skip a lot is not necessarily a good reviewer, but it does suggest they know when to withhold their judgment.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

added 249 characters in body
Source Link

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign. That's
That's all these numbers are - signs. A fast reviewer is not necessarily a bad one, but most of us cannot review correctly at breakneck speed. A reviewer who uses Skip a lot is not necessarily a good reviewer, but it does suggest they know when to withhold their judgment.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign. That's all these numbers are - signs.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign.
That's all these numbers are - signs. A fast reviewer is not necessarily a bad one, but most of us cannot review correctly at breakneck speed. A reviewer who uses Skip a lot is not necessarily a good reviewer, but it does suggest they know when to withhold their judgment.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.

Source Link

A high ratio of Approve:Reject could indeed be an indicator of bad reviewing, at least on Stack Overflow.

Assuming that the quality of edits suggestions is normally distributed, there probably is such a thing as an Approve:Reject ratio that matches the distribution of good and bad edits.

But.. there are a few things that throw a monkey wrench into this machinery.
The first is that we cannot see how many edits a reviewer has Skipped. This is a variable that would need to be accounted for.
The second is that Stack Overflow recently started rate-limiting the amount of edit suggestions that a user can make. This has helped to prevent edit sprees and retagging sprees, which has (probably) changed the distribution of good and bad edits.

A better variable, however, would be how fast a reviewer reviews, and how often they choose Skip. Fast reviewing is a bad sign; Skipping a lot is a good sign. That's all these numbers are - signs.

I think your real question is how to be a better edit reviewer, and what early warnings you can use to see if you're on track. I'm not sure about what early warnings we can give, but for what it's worth, I proposed a FAQ for editors once. Several users have contributed to it. Although it is aimed at editors, it may be useful for reviewers as well.