324

TL;DR

Noticed your rep change? It’s because we’re deploying an update to the rep in the Documentation Beta as was announced a couple weeks ago. The biggest change: there is now a “minor” contributor level where example upvotes give you +1 instead of +5.

Read on for the nitty gritty of how Documentation rep now works:

Gaining Reputation

  1. Whenever a change is approved, the author receives +2 reputation, unless...
    • They were the last editor to all the topics modified in that change or
    • The change rolls a topic (or topic(s)) back to a previous state
  2. If an answer cites a topic or example, and that answer is upvoted each contributor gets a one-time +5
    • A user is considered a contributor if they have ever net added 20 or more characters to the topic or example (more on contributors below)
    • This only happens once per answer-user-pair, multiple links do not stack
    • This only happens on the first upvote after a documentation link has been added, no user can gain more than +5 reputation per-answer from Documentation
  3. Whenever an example is upvoted, major contributors to the example get +5 reputation and minor contributors get +1 reputation
    • Minor contributors are everyone who has added at least 20 characters to an example as part of a single change
    • Major contributors are the creator of the example plus everyone who has added at least 350 characters to an example
      • The 350 characters may be split across several changes, provided that each change adds at least 20 characters

Losing Reputation

  1. If the topic(s) affected by a change are rolled back to before it was applied, the +2 from approval is removed
  2. If the answer upvote is reversed, the answer is deleted, or the documentation link is edited out then the +5 from citation is removed
  3. If the example upvote is reversed, the example or topic is deleted, or the contributions the user made are rolled back then the +1 or +5 is removed
    • If enough contributions are rolled back so that a user is no longer a major contributor but is still a minor contributor, the +5 is instead converted to a +1

In keeping with Q&A, if the reputation has been live for 60 or more days it is “locked in” and future deletions, removals, or rollbacks will not affect it.

Frequently Asked Questions

  • Q: How are characters counted for minor and major contributor status?
    • A: Whitespace and formatting are removed and a per-character diff calculated. The number of characters contributed is: <# of characters added> - <# of characters removed>
  • Q: Why can’t deletion make you a contributor?
    • A: We found through manual evaluation of the data that while most deletes improve content, they do not improve it as much as the typical addition. Accordingly the one-time +2 rep reward from their deletion being approved felt like an adequate and appropriate reward.
  • Q: Why do changes have to be 20 or more characters to affect contributor status?
    • A: Many of the changes smaller than 20 characters are minor cleanups (spelling, punctuation, etc.) which, like deletion, feel adequately rewarded by the +2 on approval. They do not “stack up to” contributor status because it doesn’t make sense for repeated spelling or punctuation improvements to ever equal more substantive additions.
  • Q: Why 350 characters to be a major contributor?
    • A: This number is about 3/4ths the typical size of the initial revision of an example. We believe adding this amount of content to existing examples is about equal to the effort that creators put in, and is thus worth the same potential reward.
  • Q: Am I still considered a contributor to a topic if my example is deleted?
    • A: If your only contribution to a topic is through the deleted example, you will no longer be considered a contributor. Whether that impacts your reputation depends on how old the rep gained from the example is, as detailed above.
  • Q: What happens when an example is moved?
    • A: You may lose your contributor status to the previous topic the example was on, and may gain it on the new topic, depending on your other contributions to those topics. Moving an example is not treated like a deletion, so you will not lose reputation.
  • Q: What happens if someone upvotes an example they’ve contributed to? Or upvotes an answer that cites one of their topics?
    • A: A user can never give themselves reputation. Upvoting an example you’ve contributed to (or an answer that cites Documentation you’ve contributed to) may reward other users, but never yourself.
  • Q: What rep is subject to the daily rep cap?
    • A: All reputation except the +2 from approval is subject to the daily rep cap.

Inspecting Contributors

There’s a a dev-y page that shows a bunch of detail about topic and example contributors. You can find this page at /documentation/contributors/topic/{id} and /documentation/contributors/example/{id}.

Example link for the C Hello World topic, and one for the Original “Hello World!” in K&R C example.

Contributor page

The top two sections show all the current contributors to the topic or example, and include links to all the changes they have made. These changes include those that did not affect their contributor status, and are meant to give the full context of their contributions.

The table includes everybody who has ever been considered a contributor, the dates and levels of their contributions, and if their contribution ended why it ended.

We’re exposing these details to make it easier for folks to help us find bugs and edge cases in the reputation system.

89
  • 26
    Do downvotes still have no effect other than sorting?
    – Kevin B
    Commented Sep 14, 2016 at 17:34
  • 34
    So the recalc has already happened?
    – jscs
    Commented Sep 14, 2016 at 17:34
  • 172
    @the12 "You've eliminated all incentive to update docs" if reputation is your only incentive then you don't have to edit the docs. There will be still users who will edit it to make something better and not just for reputation.
    – Rizier123
    Commented Sep 14, 2016 at 18:44
  • 21
    Well, I have lost a little less than 2k. Considering I had 4.5k, that's ~45%. That hurt a bit. But I have expected something similar. Will is not broken, we can still participate in doc suggestions. ;) Commented Sep 14, 2016 at 19:00
  • 90
    I feel cleansed. Commented Sep 14, 2016 at 19:00
  • 37
    @the12 Well there are many users who care about reputation and many who only care about reputation and not about the quality of the docs. So attracting many users does not mean that is good for docs at all.
    – Rizier123
    Commented Sep 14, 2016 at 20:12
  • 29
    @the12 I stopped making edits because of the new review system and the way it allows for all edits to be robo-reviewed by people with no rep in the tags - that was the final nail in the coffin for me. So the reason for 0 pending edits, which I saw yesterday too, may have nothing to do with rep change. Commented Sep 14, 2016 at 20:38
  • 19
    "Noticed your rep change?" Couldnt you just keep our rep and update the new upvotes? Commented Sep 14, 2016 at 23:00
  • 91
    Q: Should we still disconnect Documentation rep from Q/A rep? A: Hell yes.
    – kjhughes
    Commented Sep 14, 2016 at 23:01
  • 18
    @RosárioPereiraFernandes: It's been belabored time and time again that any changes to the rep system for Documentation would happen retroactively.
    – Makoto
    Commented Sep 14, 2016 at 23:06
  • 45
    That beta label ain't just there to look pretty, @Hack-R.
    – Shog9
    Commented Sep 14, 2016 at 23:36
  • 100
    I've lost 4k reputation :'( This isn't fair, I've spent at least 5 minutes to contribute to How to use console.log() !! Commented Sep 14, 2016 at 23:46
  • 24
    @Hack-R it's been made horribly explicit that rep is going to be recalculated. See also. Commented Sep 14, 2016 at 23:48
  • 25
    It seems super counterproductive to think of it as 'losing' rep. That rep was never yours. If a bank accidentally gives you money, they take it back - and that's fair. If the new rules mean you get less rep, and we assume the new rules are more fair, then you were benefiting from an unfair system: to argue you should keep it is to argue for unfairness. Commented Sep 15, 2016 at 0:23
  • 60
    This is the first time I am so much happy after loosing the reputations. Commented Sep 15, 2016 at 5:06

9 Answers 9

193

I feel like the calculation "The number of characters contributed is: <# of characters added> - <# of characters removed>" is a bit flawed.

I don't care too much about the reputation, but it's a bit disheartening to see that I'm not even a minor contributor after having made this overhaul to a topic.

Why not just count added characters, and ignore deleted characters (rather than penalize them)? It seems strange that re-writing a paragraph counts as no contribution.

11
  • 78
    Like I said in the other comment thread, this basicly means don't bother fixing stuff, just keep adding more. Which in turn means we're in for a whole lot of cluttered through.
    – Magisch
    Commented Sep 15, 2016 at 6:33
  • 41
    This should definitely change. It was okay since adding topics was important in the first place - but this is not going to make documentation a good place removing content is as important as adding new one - if not more (same goes in coding big programs)
    – Mafii
    Commented Sep 15, 2016 at 6:41
  • 38
    "It seems strange that re-writing a paragraph counts as no contribution." Do it in two steps. First delete everything that is wrong. Wait until edit is approved. Then add the right information in a second edit. Never, ever mix additions and deletion in the same edit.... Okay I admit, this is just a joke. I fully agree with you. It's bad and they know it. They have been warned about it several times. Commented Sep 15, 2016 at 9:31
  • 8
    I agree with this post entirely, especially with code. If someone is doing something the long way, and you make it simpler, then you won't be considered a contributor with the new system
    – JD9999
    Commented Sep 15, 2016 at 9:41
  • What are some ways not penalizing deletions would be gamed? Perhaps it would encourage rewriting code and/or paragraphs for no reason other than hitting the 350 char limit? Commented Sep 15, 2016 at 11:01
  • 10
    @JD9999 Exactly! It's as if reducing code clutter and refactoring was ever a bad thing... Code golf users will be pretty sad.
    – CPHPython
    Commented Sep 15, 2016 at 11:03
  • 9
    Haha, I love that this is the "Lines Of Code Count" problem writ in a different medium, that people have been fighting against for decades and decades.
    – Kzqai
    Commented Sep 15, 2016 at 18:23
  • 3
    I completely agree that the "net" method has problems. But regarding what goes wrong if you just count additions, I think the most common issues tended to be reordering edits (where simply moving a paragraph or code block is counted as a major "addition") or formatting (a big block of text or code is removed, and re-added with markup, etc.)
    – Jaydles Staff
    Commented Sep 15, 2016 at 23:23
  • 7
    @Jaydles: A "Move" can be taken into consideration. Strip all formatting from a topic, then count the occurrences of [a-z] before, and after. Get the difference per letter, add these up, and I'd say you have a more accurate count of characters changed. "I am Yoda" > Yoda, I am! should return 0. Then again, "I am Yoda" > "Yo, I'm da man!" should return just 1, which is arguably too low.
    – Cerbrus
    Commented Sep 16, 2016 at 10:18
  • @Jaydles Then you need to employ some more expensive calculation, eliminating appearances of any consecutive 30+ bytes (or such) which are equal in additions and deletions, after stripping whitespace, from the calculation. [Even with markup there shouldn't be that few bytes not matched by the consecutive 30+ byte sequences or the section is either very, very big or markup just abused.]
    – bwoebi
    Commented Sep 17, 2016 at 11:13
  • 5
    Probably the "correct" approach would be using Operational Transforms to calculate minimum number of operations to transform the input into the output, and give reputation based on that. Moving a paragraph would be one operation, just like adding or removing a character. A complex rewrite would be many small operations - and commensurately rewarded. Commented Sep 22, 2016 at 7:56
118

I think that the rep recalc is pretty widespread at present, there are many users who have had the script fix their account.

Considering how much reputation some people seemed to have amassed from minor edits, this was much needed. I like that the issue was raised, acknowledged, and addressed in a very efficient manner; and also the result seems to be fair.

With this in place, hopefully we can all move on from worrying about the Java Arrays example breaking the internet and get back to focusing on the actual reason for SO Docs: canonical content.

23
  • 21
    The Java Arrays example is still so huge, there is danger that when it collapses, people will get stuck under the debris... (But at least probably less people will try and poke it...)
    – ppeterka
    Commented Sep 14, 2016 at 19:16
  • 2
    there are more features in the pipeline that should be able to assist with that type of example - now that is something I'm eagerly waiting for...
    – ppeterka
    Commented Sep 14, 2016 at 19:30
  • 34
    I'm a little worried about how the new incentive system encourages making examples even larger and doesn't reward housekeeping at all. There will be nothing canonical about these, just lots of pitfighting over edits.
    – Magisch
    Commented Sep 14, 2016 at 19:43
  • 4
    @Magisch as I understand it, you receive the +2 for an approved edit. It just limits the +5 that you were getting for every upvote on the example, which makes total sense
    – Lamak
    Commented Sep 14, 2016 at 19:44
  • 4
    @Lamak Yeah. Assume that you can choose by artificially making an example longer and getting hundreds of rep or do the right thing and get maybe 2. That sets a terrible incentive. But I guess thats just one of these things that need to be seen in live to be changed
    – Magisch
    Commented Sep 14, 2016 at 19:45
  • 3
    @TylerH 2 rep for an edit is so insignificant as to be meaningless considering we're talking hundreds for edits that make the example longer artificially. If it was 20 or 30, maybe, but thats still a big maybe.
    – Magisch
    Commented Sep 14, 2016 at 19:47
  • 2
    @Magisch every type of incentive can be gamed, doesn't mean that this change isn't the right one
    – Lamak
    Commented Sep 14, 2016 at 19:47
  • 4
    @Lamak The right change should reward proper editing, that is in most cases removing just as much from any given example as adding. Making added characters a metric was fundamentally wrong imo. I'd venture so far as to say for nearly all examples most edits adding lots of text are actively harmful. Now these are the only thing the system rewards anymore.
    – Magisch
    Commented Sep 14, 2016 at 19:51
  • 6
    @Magisch and it is rewarding them. Do you think it would be ok to edit minor things on an answer or a question in SO and get rep for every upvote there?. I still feel dirty for receiving rep for some upvotes on an example I edited lightly
    – Lamak
    Commented Sep 14, 2016 at 19:53
  • 4
    In essence that means we'll be seeing lots and lots of fluff added to examples already way too big which will mean more browser clutter, less performance, and less overall usability. In the current system, pending a quick change for the opposite direction, docs is poised to die the death of a thousand cuts.
    – Magisch
    Commented Sep 14, 2016 at 19:53
  • 10
    "focusing on the actual reason for SO Docs: canonical content." Isn't the actual reason for docs missing content?
    – null
    Commented Sep 14, 2016 at 23:03
  • 10
    "canonical" is overused and misused (to mean... decent? comprehensive?) It's pretty unlikely SO Docs will ever be canon for a great many topics; @null is correct, the problems being targeted here are more in the missing / incomplete / incomprehensible docs camp.
    – Shog9
    Commented Sep 14, 2016 at 23:44
  • 9
    @Shog9: "the problems being targeted here are more in the missing / incomplete / incomprehensible docs camp" And yet, there's no reason to expect the system to accomplish that. Things with dysfunctional documentation get that way usually because it's not widely known and/or used. So how will Docs.SO change that? Commented Sep 15, 2016 at 0:42
  • 8
    Do you guys have a plan? @Shog9♦ What if "Simply lowering the bar to changes" does not go a long way? I sometimes ask questions on SO because of missing documentation. I rarely get good answers to those if any at all. I lost faith in SO docs not because of a broken rep system but because it doesn't work as advertised. Why would somebody go to docs in "6 to 9 months" to write down what would be necessary to answer a question that exists today? The bar for answering a question is already on the ground, how could docs have a lower one?
    – null
    Commented Sep 15, 2016 at 8:31
  • 2
    " canonical content." Canonical, example centric content, you do not find elsewhere already in sufficient quality. And as a wish: well organized canonical, example centric content which doesn't exist elsewhere. Commented Sep 15, 2016 at 9:33
29

The reputation system is now more balanced. The biggest flaw I still see is the length based dependence of the reputation gained. This will incentivize bloat, i.e. overly verbose contributions with a low density of information.

This problem will aggravate over time, when it becomes less about building up content but more about improving quality of existing content.

This may seriously impede the success of Documentation!

Problem: I find it difficult to come up with anything better. Rating edits seems to be overkill.

Best alternative I have: Blame the contribution line by line to contributors, then distribute reputation for upvotes based on percentage of lines one contributed to, normalize by total number of contributions to single lines. This assumes that every line is equally important, but at least it doesn't scale with the length.

9
  • 2
    rating edits in edit approval queue? Commented Sep 15, 2016 at 19:11
  • @uoɥʇʎPʎzɐɹC Yes, why not. If you already look at the edit and decide if it's worthy to stay you could as well classify it along the way. Not sure though that three to five ratings would be enough to get statistical accuracy. But it could work. Commented Sep 15, 2016 at 19:41
  • a major two char edit should have more than a spelling fix. Commented Sep 15, 2016 at 19:42
  • Gonna flesh this out as a feature request, I'll need help though. Commented Sep 15, 2016 at 19:43
  • @uoɥʇʎPʎzɐɹC I like the idea. If you propose it I will definitely support it. You could for example show a display of a screenshot of the current Documentation review queue, then one how you imagine it to be so that a classification (cosmetic, minor, major) is possible and then propose that we test this system against their system, then manually judge who is better and then keep the better system. It's worth a try. Commented Sep 15, 2016 at 19:45
  • willing to write it together? ;) Commented Sep 15, 2016 at 20:18
  • @uoɥʇʎPʎzɐɹC I can do it tomorrow if you haven't done it by then. ;) Commented Sep 15, 2016 at 21:04
  • google docs and just use the collab feature? Commented Sep 15, 2016 at 23:32
  • Similar suggestion is documentation-shares-the-work-make-the-rewards-shared-not-multiplicative Commented Sep 22, 2016 at 2:52
21

It would have been nice to keep track of how much reputation was lost for an individual user, since we do have wildly varying reports of how much reputation was lost. It wouldn't be something that'd be public since reputation loss is generally private, but it would have been a nice-to-have.

All in all, I do feel like the major concerns about how much rep Documentation is contributing to the ecosystem at large can mostly be soothed, since it's not a whole lot anymore.

3
  • 11
    Figure on > 50% of all reputation being eliminated today, skewed heavily toward the top rep-earners.
    – Shog9
    Commented Sep 14, 2016 at 23:39
  • @Shog9: Suppose one day a user got 50 points from documentation edits, and 150 points from up-votes on Q&A, and also got 6 answer up-votes that generated no benefit because the 200 cap limit was reached. Further, suppose that the documentation recalc eliminates 40 of the 50 documentation points. Does the user still get credit for the answer up-votes that would've counted had the doc points not been granted? For the hypothetical user, would the net score not change, even though the contribution from doc edits decreases? (I rather assume that the answer is "Yes", but reassurance would be nice.) Commented Sep 15, 2016 at 5:20
  • 3
    Yes, @jonathan. Recalc means the world is rewritten under the new rules - the rep for that day is as it would be if the cap had been reached slightly later due to the answer votes.
    – Shog9
    Commented Sep 15, 2016 at 5:59
13

Q: Why 350 characters to be a major contributor? A: This number is about 3/4ths the typical size of the initial revision of an example. We believe adding this amount of content to existing examples is about equal to the effort that creators put in, and is thus worth the same potential reward.

This doesn't solve much when the initial revision was smaller than 350 characters and then it was expanded into a much larger example. So while in most cases the reward for the creator is justified, there are certain scenarios where it isn't.

For example: Inner Join - SQL Server (initially 152), Insert Into - SQL Server (initially 65), Return multiple values using std::tuple - C++ (initially 115), and Get and invoke method Reflection - C# (initially 251).

Disclaimer: I expanded the Insert Into example.

4

Tangentially related to this, but I received an email notification about this post, saying "the following item was added to your Stack Exchange global inbox"... I'm on Android, and have the SE app installed. Clicking the link in the email did not launch anything, neither the app or the browser.

I manually opened the SE app, and either couldn't find any sort of "inbox" functionality, or there was no notification of this post. I swiped from the right to the left (or clicked on the number icon in the upper left corner) and say what appear to be older notifications, but no mention of this post.

Should I have expected this to all work within the Android app?

1
  • Ask another question for this. Commented Oct 13, 2016 at 13:10
2

There are minor changes that contribute substantially to a topic. I think it should be up to the reviewers to see if it's a minor edit, like a spelling correction, or a substantial edit worth reputation points.

The 20 character addition limit will just cause people to game it, rewriting sentences to be slightly longer, etc. Here's an edit I made that is really small (-2 +2 characters), but it has a fairly big impact.

Here's an example of a change big enough that the original author shouldn't get reputation points any more, but because the diff tool wants to find similarities, it passes the 20 character limit just by me using the same function name. The previous version didn't even compile.

10
  • 13
    "I think it should be up to the reviewers to see if it's a minor edit" I disagree. Reviewers get it wrong, too often.
    – Cerbrus
    Commented Sep 15, 2016 at 6:55
  • The algorithm also gets it wrong too often, but which one is better? Some combination?
    – Drathier
    Commented Sep 15, 2016 at 6:57
  • 3
    I don't think that your first link has a big impact, you just renamed some a variable Commented Sep 15, 2016 at 6:59
  • @SZenC I didn't rename a variable. I changed the example to use the middle variable instead of the left-most one, to illustrate that there are no pointer relations between them in either direction, avoiding a possible misunderstanding. See the edit comment at the top.
    – Drathier
    Commented Sep 15, 2016 at 7:09
  • 5
    @FilipHaglund: I'd hardly call that a "big impact", especially since the significance isn't explained in the example.
    – Cerbrus
    Commented Sep 15, 2016 at 7:17
  • @Cerbrus In this case, a, b, and c, are all independent -- changing one will not change the others isn't enough?
    – Drathier
    Commented Sep 15, 2016 at 7:19
  • The significance of the edit.
    – Cerbrus
    Commented Sep 15, 2016 at 7:42
  • Perhaps the system will get to automatically mark edits as major, but those marked minor (or too small to be marked at all) will be decided by a reviewer vote. This way people can't just make everything a major edit, and the fine line between no significance or minor is left to the discretion of the reviewers.
    – Cas
    Commented Sep 15, 2016 at 7:52
  • 2
    @Cerbrus The diff and characters added metric to judge worthiness of a edit is so inaccurate as to be meaningless entirely. Even if reviewers got the classification wrong 90% of the time, it'd still be better then the current system.
    – Magisch
    Commented Sep 15, 2016 at 8:06
  • @Magisch: I agree that the current system is <redacted>. Still, I trust reviewers even less.
    – Cerbrus
    Commented Sep 15, 2016 at 8:10
2

The new reputation points system is better than the previous.

However, you should consider something to improve the remarks/syntax/parameters sections.
In most of the cases these sections are empty.

2
  • 2
    Ok, so you deleted your other answer but this one seems to end on a partial word, and not be completely formatted.
    – Louis
    Commented Sep 15, 2016 at 18:39
  • definitely. did not expect so much from a zero upvote answer Commented Sep 15, 2016 at 19:33
-4

I feel like Documentation has the potential to become something really powerful and useful but I'm afraid that this will reduce contributors to it, although I agree that the reputation was indeed too much and that would probably lead to low quality posts on the long run.

Maybe a good way to incentivate people to contribute more to Documentation may be add more badges as rewards for being constant. I'm not sure if this would be a good idea though because people may start to exploit it like for reviews.

I am trying to participate anytime I have the opportunity myself but personally I don't feel comfortable yet on how the browsing works. I guess I'll get used to it eventually.

Not the answer you're looking for? Browse other questions tagged .