18

I recently unapproved a change to the Documentation for a tag that I have a lot of experience in.

The approved edit was approved by 3 out of 4 reviewers. The only reviewer to "Reject" the edit was coincidentally also the only reviewer that actually had any reputation for that language tag.

The other 3 reviewers predominantly had lower reputation overall (~500), which is not in itself proof they lack expertise, but literally zero of them had any reputation for the relevant language under review.

Can (should) we improve the documentation review queue to avoid this scenario?

  • You might say the system worked because I had a notification in my inbox that the review was approved, so I could manually revert it
  • The edit itself was almost a good one; it needed reviewers with more experience
    • Part of me believes the edit may continue to be controversial (in which case we should leave it unapproved and improve upon it until it is not so controversial)

What I propose:

  • A small change (from my POV) to the Documentation review queue: Only users with at least 10 reputation in the language under review are allowed to approve/reject Documentation edits
    • This is literally 1 upvote to an answer for a question under that language tag

This will additionally help prevent "robo-approvers" because you are required to have at least a tiny bit of interest in a language before you can review someone's proposed Documentation edits.

15
  • 2
    This won't prevent robo reviews, but will certainly help - agree with this
    – CalvT
    Commented Jun 29, 2017 at 12:51
  • 3
    This has been suggested several times. The counterpoint is that many small tags may be un-reviewable (no one qualifies to review them out of the pool of reviewers) and also that you don't need tag expertise to approve grammar/typo changes. On thing I would really like to see is the rep requirement go up. If you can't approve edits on main until 2k we should do the same for Docs. Commented Jun 29, 2017 at 13:03
  • @NathanOliver, your concern in the first half is exacerbated by your suggestion in the second. If the rep requirement is raised, it will be more of a problem for small tags to get reviewed.
    – code11
    Commented Jun 29, 2017 at 13:22
  • @NathanOliver: Perhaps the logic should become more complicated regarding reviewing eligibility. Depending on how big a tag is, how much rep a user with that tag has, and how many users have any rep with that tag.
    – AndyG
    Commented Jun 29, 2017 at 13:23
  • 2
    @code11 I'm not suggesting both things happen. I'm just suggesting that we only allow 2K+ people to review docs changes. That still leaves a large pool of reviewers (if they can find the review button) Commented Jun 29, 2017 at 13:23
  • Depends on the low freq tag. Some of them have nice auto-upvote mech. And we do not do "good" documentation every day, it should be ok from my low freq tag experience. Commented Jun 29, 2017 at 13:53
  • We should define whats Low Freq Tag is exactly . So we can see what are the LFT that have documentation, and withone have documentation edit in the year. It will be close to 0 depending on how you define LFT. Commented Jun 29, 2017 at 13:57
  • Maybe if a tag has over x number of users with over y reputation, then only said users can approve documentation edits?
    – CalvT
    Commented Jun 29, 2017 at 13:59
  • 12
    If it is such a "small" tag that there aren't enough users here with 10 reputation in that tag, we probably shouldn't be documenting it because we clearly don't have enough collective expertise.
    – Cody Gray Mod
    Commented Jun 29, 2017 at 14:00
  • 5
    All that's asked is that reviewers in our hypothetical small tag have 10 points - a single upvote on a single answer. Surely that's not a high bar, even for the low volume tags. Commented Jun 29, 2017 at 14:16
  • 1
    Given that you can only create a tag in Documentation if the main site tag is popular enough, that might actually be a reasonable filter. Low-traffic tags on SO have problems with people being unable to vote for synonyms, etc, but those are often tags that are below the thresholds required to create Documentation tags. Commented Jun 29, 2017 at 17:25
  • 1
    @NathanOliver "The counterpoint is that many small tags may be un-reviewable (no one qualifies to review them out of the pool of reviewers) " Moot point, since you need at least 5 users with score 1 in the tag, for the documentation even exist.
    – Braiam
    Commented Jun 29, 2017 at 18:34
  • @Braiam Those are users who are interested in having Documentation for the topic. Different from being committed to reviewing changes for the rest of their life.
    – user6655984
    Commented Jun 29, 2017 at 22:29
  • @Walt wait, isn't the button called "commit": "Be the first to commit to building the tomcat8 Documentation."
    – Braiam
    Commented Jun 29, 2017 at 22:34
  • 4
    This has been brought up numerous times: Minimum tag score for adding documentation
    – jpmc26
    Commented Jun 29, 2017 at 23:15

0