Skip to main content
deleted 1 character in body
Source Link
TylerH
  • 21.2k
  • 22
  • 229
  • 328

My first reaction, mostly based on the title of this question, was that there is no way anyone would even be able to do 1500 reviews in a single day without severe fatigue and subsequently lots of errors (or simply doing robo reviewing). But that was coming from a 40 reviews a day limit perspective.

Thinking more about it I changed my mind and think this is actually an eye opener. Doing lots of reviews at once with high quality is possible. I'm very thankful to this moderator for the reviewing work and hope that in the future more reviews can be done by others as well (if they want to).

Here are the details: Additionally to gnat's review I reviewed a few (~50) of the reviews of that moderator manually and I found that they are all high quality. A couple of times I would choose rather "needs clarity" instead of "needs focus" but that might be personal preference and once I could salvage a closed "recommendation" question by removing the recommendation part and leaving something answerable. With an average speed of 20 reviews per ~10 minutes for many hours (!) this person shows that one can deliver high quality reviews in the order of hundreds per day if one is experienced and abstains from robo reviewing.

I'm a bit concerned about the reduced consensus and missing audits. The moderator is basically doing regular curation work here, not real exception handling. The diamond does shortcut the usual 3 close votes threshold. I would feel a bit safer if we could still have the consensus feature also in thesesthese cases, where moderators aren't really in their moderating role somehow. It's not urgent but we might not always get such high quality work.

But the main point is that this proves in my eyes that hundreds of accurate reviews per day are possible and that there are people actually willing to do them. In order to accommodate for these people we should increase the limit for trustworthy users (large rep, experienced reviewers, good audit history). A limit of 40 reviews per day is too small, even if not all of the trustworthy users will exceed the limit, some might and therefore it's worth to change it. Take home message: We should trust experienced users (users like this moderator only without a diamond) more. They know what they are doing.

As a small thing to accommodate for different amount of reviewing time during the week, maybe the limit should be weekly, not daily. That would allow for more flexibility.

Last, but not least, the question of long term fatigue and impact on motivation is still unclear. The moderator is just now doing a couple of hundreds of reviews more but we don't know what the long term impact on this moderator is. I hope he will be fine. Part of the 40 reviews a day limit is surely also protection of the reviewers themselves. I'm a bit worried there it might lead to burnout soon and I hope the moderator takes care of himself there.

My first reaction, mostly based on the title of this question, was that there is no way anyone would even be able to do 1500 reviews in a single day without severe fatigue and subsequently lots of errors (or simply doing robo reviewing). But that was coming from a 40 reviews a day limit perspective.

Thinking more about it I changed my mind and think this is actually an eye opener. Doing lots of reviews at once with high quality is possible. I'm very thankful to this moderator for the reviewing work and hope that in the future more reviews can be done by others as well (if they want to).

Here are the details: Additionally to gnat's review I reviewed a few (~50) of the reviews of that moderator manually and I found that they are all high quality. A couple of times I would choose rather "needs clarity" instead of "needs focus" but that might be personal preference and once I could salvage a closed "recommendation" question by removing the recommendation part and leaving something answerable. With an average speed of 20 reviews per ~10 minutes for many hours (!) this person shows that one can deliver high quality reviews in the order of hundreds per day if one is experienced and abstains from robo reviewing.

I'm a bit concerned about the reduced consensus and missing audits. The moderator is basically doing regular curation work here, not real exception handling. The diamond does shortcut the usual 3 close votes threshold. I would feel a bit safer if we could still have the consensus feature also in theses cases, where moderators aren't really in their moderating role somehow. It's not urgent but we might not always get such high quality work.

But the main point is that this proves in my eyes that hundreds of accurate reviews per day are possible and that there are people actually willing to do them. In order to accommodate for these people we should increase the limit for trustworthy users (large rep, experienced reviewers, good audit history). A limit of 40 reviews per day is too small, even if not all of the trustworthy users will exceed the limit, some might and therefore it's worth to change it. Take home message: We should trust experienced users (users like this moderator only without a diamond) more. They know what they are doing.

As a small thing to accommodate for different amount of reviewing time during the week, maybe the limit should be weekly, not daily. That would allow for more flexibility.

Last, but not least, the question of long term fatigue and impact on motivation is still unclear. The moderator is just now doing a couple of hundreds of reviews more but we don't know what the long term impact on this moderator is. I hope he will be fine. Part of the 40 reviews a day limit is surely also protection of the reviewers themselves. I'm a bit worried there it might lead to burnout soon and I hope the moderator takes care of himself there.

My first reaction, mostly based on the title of this question, was that there is no way anyone would even be able to do 1500 reviews in a single day without severe fatigue and subsequently lots of errors (or simply doing robo reviewing). But that was coming from a 40 reviews a day limit perspective.

Thinking more about it I changed my mind and think this is actually an eye opener. Doing lots of reviews at once with high quality is possible. I'm very thankful to this moderator for the reviewing work and hope that in the future more reviews can be done by others as well (if they want to).

Here are the details: Additionally to gnat's review I reviewed a few (~50) of the reviews of that moderator manually and I found that they are all high quality. A couple of times I would choose rather "needs clarity" instead of "needs focus" but that might be personal preference and once I could salvage a closed "recommendation" question by removing the recommendation part and leaving something answerable. With an average speed of 20 reviews per ~10 minutes for many hours (!) this person shows that one can deliver high quality reviews in the order of hundreds per day if one is experienced and abstains from robo reviewing.

I'm a bit concerned about the reduced consensus and missing audits. The moderator is basically doing regular curation work here, not real exception handling. The diamond does shortcut the usual 3 close votes threshold. I would feel a bit safer if we could still have the consensus feature also in these cases, where moderators aren't really in their moderating role somehow. It's not urgent but we might not always get such high quality work.

But the main point is that this proves in my eyes that hundreds of accurate reviews per day are possible and that there are people actually willing to do them. In order to accommodate for these people we should increase the limit for trustworthy users (large rep, experienced reviewers, good audit history). A limit of 40 reviews per day is too small, even if not all of the trustworthy users will exceed the limit, some might and therefore it's worth to change it. Take home message: We should trust experienced users (users like this moderator only without a diamond) more. They know what they are doing.

As a small thing to accommodate for different amount of reviewing time during the week, maybe the limit should be weekly, not daily. That would allow for more flexibility.

Last, but not least, the question of long term fatigue and impact on motivation is still unclear. The moderator is just now doing a couple of hundreds of reviews more but we don't know what the long term impact on this moderator is. I hope he will be fine. Part of the 40 reviews a day limit is surely also protection of the reviewers themselves. I'm a bit worried there it might lead to burnout soon and I hope the moderator takes care of himself there.

Source Link

My first reaction, mostly based on the title of this question, was that there is no way anyone would even be able to do 1500 reviews in a single day without severe fatigue and subsequently lots of errors (or simply doing robo reviewing). But that was coming from a 40 reviews a day limit perspective.

Thinking more about it I changed my mind and think this is actually an eye opener. Doing lots of reviews at once with high quality is possible. I'm very thankful to this moderator for the reviewing work and hope that in the future more reviews can be done by others as well (if they want to).

Here are the details: Additionally to gnat's review I reviewed a few (~50) of the reviews of that moderator manually and I found that they are all high quality. A couple of times I would choose rather "needs clarity" instead of "needs focus" but that might be personal preference and once I could salvage a closed "recommendation" question by removing the recommendation part and leaving something answerable. With an average speed of 20 reviews per ~10 minutes for many hours (!) this person shows that one can deliver high quality reviews in the order of hundreds per day if one is experienced and abstains from robo reviewing.

I'm a bit concerned about the reduced consensus and missing audits. The moderator is basically doing regular curation work here, not real exception handling. The diamond does shortcut the usual 3 close votes threshold. I would feel a bit safer if we could still have the consensus feature also in theses cases, where moderators aren't really in their moderating role somehow. It's not urgent but we might not always get such high quality work.

But the main point is that this proves in my eyes that hundreds of accurate reviews per day are possible and that there are people actually willing to do them. In order to accommodate for these people we should increase the limit for trustworthy users (large rep, experienced reviewers, good audit history). A limit of 40 reviews per day is too small, even if not all of the trustworthy users will exceed the limit, some might and therefore it's worth to change it. Take home message: We should trust experienced users (users like this moderator only without a diamond) more. They know what they are doing.

As a small thing to accommodate for different amount of reviewing time during the week, maybe the limit should be weekly, not daily. That would allow for more flexibility.

Last, but not least, the question of long term fatigue and impact on motivation is still unclear. The moderator is just now doing a couple of hundreds of reviews more but we don't know what the long term impact on this moderator is. I hope he will be fine. Part of the 40 reviews a day limit is surely also protection of the reviewers themselves. I'm a bit worried there it might lead to burnout soon and I hope the moderator takes care of himself there.