59

I found a case of robo-reviewing in documentation.

A few examples:

And that are only the first 2.5 of 7 pages of his documentation backlog

Most approvals are fine, because the content itself is fine, but there are disproportionally many bad approvals in here.

This is especially problematic with most proposals just needing a single approval and often being longer around (as there are discussions needing to be resolved first) and thus robo-reviewers being able to actually do harm.

I maybe do a lot check the backlog of proposals (to check nothing gets wrongly approved), but I also don't catch everything and miss a few bad reviews … a bad rejection is not that bad, but a bad approve is much worse for the documentations quality.

What can I do about them and what shall be improved in the system?

I currently have a feature-request: Documentation proposals: There are comments, please do not approve straight ahead! but this only will mitigate the issue a bit.

8
  • 27
    I've seen hundreds of these. This is Inded an issue. A good portion of the reviewers do nothing but approve without even looking at the change. Jul 26, 2016 at 3:36
  • 27
    Doesn't help that when you go to approve/reject you can't see if anyone else has already reviewed. So if you left rejection comments, next reviewer can't see them until after they pull the trigger. Mybe this is by design so that everyone is supposed to be objective...not sure
    – charlietfl
    Jul 26, 2016 at 4:20
  • 1
    @charlietfl with comments I mean comments below the individual examples, not the rejection comment. [but that the next reviewer cannot see it is a bit annoying too. … But that's not directly related here. It gains nothing when someone always approves and rejects in case there already was a rejection.]
    – bwoebi
    Jul 26, 2016 at 4:21
  • Right...understand that part...but there are also rejection comments that aren't visible until after the fact. Not sure if robo reviewer might think twice or not if they could see that someone else rejected
    – charlietfl
    Jul 26, 2016 at 4:22
  • 3
    I think this can be later implemented similarly to the review on SO, that is adding "this is only a test" reviews from time to time.
    – Ruslan Bes
    Jul 26, 2016 at 10:41
  • 13
    "Review ban from Documentation for the next 24 years" would be the appropriate punishment for them! Jul 26, 2016 at 12:09
  • 6
    That didn't take long. Jul 26, 2016 at 17:19
  • 1
    Here's another example: stackoverflow.com/documentation/proposed/changes/70834 It's really disappointing and discouraging that SO team haven't addressed this important issue yet.
    – P.P
    Aug 1, 2016 at 9:34

1 Answer 1

-2

Part of the problem is that it's harder to know what's going on with Documentation. The layout alone is difficult to understand, and I personally have trouble working out which section the information is being added to. It's entirely possible the guy has never used Documentation and the edits all look good to him. I know I avoid reviewing Documentation because I've never used it and I don't really understand the goal of it (or what to accept and what to reject).

Possibly we need a kind of "induction" process for people who are new to the format of Docs. Bear in mind it's absolutely possible to gain review privileges for a system you've never used before.

2
  • It's good that you avoid reviewing Documentation. More people should follow your example -- if you don't understand the format, purpose, or the subject matter of an edit don't review it. That said, I don't think the solution is to add more processes. The answer, IMO, is to disallow reviewing for Documentation until the team has figured out what it's for. Jul 17, 2017 at 15:07
  • @MikeMcCaughan Possibly, but are you suggesting Documentation can continue to be edited while not requiring reviews, or just that it stops altogether? Stopping only the reviews would be very harmful if the SO team then used the current data in Docs as a base for the new system (which, it seems, they intend to do). The current reviews may not be great, but they're better than nothing. At least we have some sort of gate slowing the flood of terrible edits.
    – Clonkex
    Jul 17, 2017 at 23:05

Not the answer you're looking for? Browse other questions tagged .