186

Documentation. Also known as a gold mine for people who love to get easy reputation points. Right now it's being (ab)used by many users, some of whom are using it to exponentially increase their reputation points.

Currently, upvotes on examples give all contributors +5 rep. There are no limits on the amount of rep you can gain from an example, except for the standard 200 rep per day limit. This system is severely broken.

For example, I contributed one sentence to the Python 'Hello world' example, to revert an edit made by another user (which should have been rejected) and add a single (relatively unimportant) sentence. Since then, I have received more than 300 reputation, tripling my Stack Overflow reputation points. And I continue to get reputation, even though the sentence I added is no longer there! I don't deserve this.

My SO rep has exponentially increased since I made a contribution to documentation.

Why does reputation even matter that much? Surely a bit of inflation caused by documentation can't hurt?

Yes, theoretically reputation is just a form of imaginary Internet points. But it still determines privileges on Stack Overflow. We don't want a bunch of low-rep users with high-rep features, do we? Also, in the professional world, employers do use your Stack Overflow reputation as a rough gauge of your programming ability, but Documentation is making it look worthless.


I think that this system is severely broken at the moment, and we need to introduce some more reputation limits to fix it. I have a few ideas which would reward substantiate edits while reducing the possibility to farm reputation points. (Note that some of these ideas were seeded by Jon Ericson and sokin's answers.)

Idea 1: Only reward editors whose contributions still make up a substantial part of the example.

Basically, an editor should only be rewarded if the content they contributed makes up more than x% of the whole example (e.g. around 10 or 20 percent). How 'their content' is counted is debatable, but possible systems include:

  • Counting based on whoever last modified a line (e.g. git blame)
  • Counting based on the number of words which are still part of the example

Whether or not the example creator always receives reputation points or not is also debatable.

Idea 2: Introduce a reputation points cap per-example

There should be a reputation points cap of (e.g.) 50 reputation per example, so single edits won't produce huge amounts of reputation. Subsequent edits could possibly allow a higher cap.

Alternative

The reputation points cap could only be introduced when a large number of people edit an example. E.g. once more than 5 or 10 people edit an example, further editors cannot gain any more than 50 rep for that example. Previous editors can keep any excess reputation points they have accumulated.

Idea 3: Scale reputation points awarded per-upvote, based on number of editors

If an example only has 5 or so editors, each person will be awarded the full 5 reputation for every upvote. But, once an example reaches 10 or so editors, each upvote may only be worth 3 reputation; and when there are 20 editors each upvote will only be worth 1 reputation.

This way, the amount of reputation received from upvotes is proportional to your contribution to the example.

Alternative

Instead, reputation per upvote could be scaled by the amount you have contributed to an example: if you have only contributed a few words, you only get a few reputation points; if you wrote most of the documentation, you will get a larger share of the reputation points awarded. This would probably be a better way to scale reputation points, but is more complex.

Idea 4: Do not award reputation points if your contribution has been overwritten or removed

This bit is obvious - if your contribution isn't part of the example any more, you shouldn't be getting any reputation for it.


Hopefully if some of these suggestions are implemented, reputation points will be a better representation of your activity and helpfulness on Stack Overflow - not some huge number caused by a couple of tiny edits to documentation. Of course, a reputation points recalc will probably be required to bring things back to normal.

24
  • "Currently, upvotes on examples give all contributors +5 rep." Are you sure this is still correct? A few days ago I tested by editing this example and have not received any reputation. Maybe they silently fixed this? Commented Jul 28, 2016 at 11:44
  • @Bjørn-RogerKringsjå You have to do a "substantial" edit. I think right now that is 100 characters. Not 100% sure on that number but a simple indentation or small typo fix is not supposed to give you rep anymore. Commented Jul 28, 2016 at 11:48
  • 1
    @NathanOliver Ah, okay. So I guess it's partially fixed. :) Commented Jul 28, 2016 at 11:50
  • 95
    Plenty of people want the reputation removed from Documentation - I would also approve of such action. Or if these imaginary internet points are so important - let Documentation have separate "reputation" points. This was suggested several times as well. I second each of these options. Commented Jul 28, 2016 at 12:02
  • 23
    @MariaDeleva rep is NOT needed for documentation, because the good contributors want to write good docs without earning some bs. this: meta.stackoverflow.com/questions/329459/…
    – JonH
    Commented Jul 28, 2016 at 13:09
  • 7
    @MariaDeleva If the point are imaginary and not all that important, why so much fuss over them?
    – DavidG
    Commented Jul 28, 2016 at 13:26
  • 2
    The "imaginary internet points" bubble is about to burst, yet life will move on and you'll still wake up in the morning. Try not to let it get to you. EDIT: @DavidG easy for you to say, you're at 30k.
    – Shark
    Commented Jul 28, 2016 at 13:26
  • 1
    @Shark Yes, but I've been here for 4 years. If I had really been bothered, I would have been a lot higher :)
    – DavidG
    Commented Jul 28, 2016 at 13:30
  • 1
    Also could change the incentive method to award points to the act of editing, rather than an edit giving you points till the end of time. This would decrease the payoff of getting a small change in. But we shouldn't remove all incentive from small changes. Sometimes a single character, may in fact, have an impact on code you write. Commented Jul 28, 2016 at 13:50
  • 2
    The simple solution is to have Q&A rep and Documentation rep be separate totals. This also makes sense, because they're two different things. Competency in one, does not imply competency in the other. Commented Jul 28, 2016 at 13:52
  • 1
    @hatchet It's already been said by SO staff that having a second set of rep would have been a huge amount of work so this is highly unlikely to happen.
    – DavidG
    Commented Jul 28, 2016 at 14:34
  • 3
    I'm not really sure what this is contributing that hasn't already been said. Ideas 1 and 3 are covered by my Documentation shares the work make the rewards shared not multiplicative, which you're obviously aware of because you linked to Jon Ericson's answer there. 4 is implicit in 1. An answer there proposing variations or specifics would be welcome. Number 2 has been proposed in comments here and there, but comments don't really count as proposals, so that could make a fine feature request on its own if you were to edit this down.
    – jscs
    Commented Jul 28, 2016 at 17:30
  • 7
    @hatchet High rep in Q/A doesn't apply or even strongly correlate to competency. You can (and some people have) get to 100k answering only very easy question that a student can answer. Its all about when you started and how persistant you are.
    – Magisch
    Commented Jul 29, 2016 at 8:51
  • 2
    Ideas 3 and 4 are similar to Documentation shares the work: make the rewards shared, not multiplicative Commented Jul 30, 2016 at 14:20
  • 2
    I'd just like to share the insane scenario I'm in. In the past week, I've gotten 750 rep. Over the summer, I've been asking and answering to earn rep, and got to around 1k. I was pretty proud of that number, because it doesn't come as easily as it did a few years ago. Now, just by contributing to a couple of docs, I get 750 rep? I wasn't making illegitimate suggestions - they were, in my opinion, useful and needed. But they were so insignificant! This has to be fixed. The lights at the end of the tunnel are the headlamps of an oncoming train.
    – Athena
    Commented Aug 1, 2016 at 16:50

9 Answers 9

30

Update Sept. 14th - the repponing

I'm calling this status-completed simply because a lot of major changes to how rep works with Documentation have just rolled out... However, that does not mean this is entirely fixed, or that no further adjustments will be made - merely that new discussions will be needed.

For completeness, here are details on the 39 remaining examples that've generated more than 1000 reputation - contrast with the (> month old) totals below:

Example                                                           Example Docs Rep Editors Earning rep TotalEditors HtmlLength 
----------------------------------------------------------------- ---------------- ------------------- ------------ ---------- 
Creating and Initializing Arrays                                  36025            73                  155          18546      
String interpolation                                              15064            42                  85           12597      
Exception filters                                                 14158            31                  86           9147       
Introduction                                                      7273             30                  70           4775       
echo, print                                                       7230             29                  59           2802       
List Comprehensions                                               4526             23                  57           6686       
Using Streams                                                     4300             28                  64           8808       
Getting a result from another Activity                            4239             16                  43           5223       
Conditional List Comprehensions                                   3970             21                  58           3479       
Using console.log()                                               3853             36                  70           3064       
Integer division                                                  3730             19                  39           3729       
Auto-property initializers                                        3502             20                  37           7504       
Using an extension method                                         3317             24                  51           6216       
Null propagation                                                  3179             14                  28           7319       
Hello, World                                                      2929             35                  69           3964       
Creating a new Java program                                       2924             37                  87           5559       
Comparing Strings                                                 2836             27                  57           6556       
Creating a new console application in .NET using C#               2705             24                  56           4603       
Creating a List from an Array                                     2427             27                  52           8470       
Accessing elements                                                2157             17                  41           9224       
Attribute Selectors                                               2115             13                  24           8585       
Ignoring files and directories with a .gitignore file             1960             28                  50           5289       
Language support for Tuples                                       1652             9                   13           4027       
Passing data between activities                                   1585             14                  23           3602       
Introduction                                                      1436             16                  26           1262       
Create a UILabel                                                  1394             11                  28           3078       
What is a lambda expression?                                      1314             14                  20           8787       
Avoiding the repetition of expensive operations using conditions  1260             14                  24           4371       
A basic build.gradle file                                         1258             11                  18           5361       
Decoding a JSON string                                            1230             12                  22           7999       
Null checking                                                     1197             10                  23           1803       
String & Character Literals                                       1144             10                  17           4325       
Unpacking Iterables                                               1124             14                  25           4535       
Resource Routing (Basic)                                          1115             14                  22           7300       
Creating variables and assigning values                           1114             18                  27           2184       
Hello World                                                       1078             23                  41           4073       
Installing packages                                               1075             19                  27           7359       
Basic selectors                                                   1042             17                  32           1372       
Print statement vs. Print function                                1018             12                  17           3926       

Original answer follows

We're definitely going to fix this. Kevin and others will be discussing the particulars this week, though it may be a bit longer before anything gets rolled out (as the variety of suggested changes here demonstrates, there are a lot of factors that should be taken into consideration). Once changes are made, reputation will be recalculated for anyone affected and we'll post an announcement here on meta.

That said, let's try to keep some perspective here. I've laid out an overview of the reputation situation over on How much reputation is given per documentation post per day - convert to community wiki - the short version is, most participants are not earning very much reputation from Documentation, and most examples do not turn into rep-cornucopia.

There are currently 61 examples that've earned a total of 1000 points or more for their collective authors... That's few enough to just list, so... Here's a list:

Example                                                          Example Docs Rep Editors Earning rep TotalEditors HtmlLength 
---------------------------------------------------------------- ---------------- ------------------- ------------ ---------- 
Creating and Initializing Arrays                                  76585            103                 136          16124      
String interpolation                                              19853            54                  70           11122      
Exception filters                                                 18607            48                  66           6680       
Introduction                                                      13096            35                  53           2744       
Outputting the Value of a Variable                                12488            36                  52           12245      
Using console.log()                                               10552            47                  57           911        
Conditional List Comprehensions                                   8784             30                  52           3255       
Creating a new Java program                                       8356             56                  71           8074       
Getting a result from another Activity                            6914             25                  38           5447       
Using Streams                                                     6643             35                  55           7806       
Hello, World                                                      6305             38                  56           4623       
List Comprehensions                                               6281             26                  43           4644       
Creating a new console application in .NET using C#               5496             30                  49           4047       
Using an extension method                                         4785             27                  43           4799       
Integer division                                                  4724             25                  32           3940       
Creating a List from an Array                                     4088             29                  43           7513       
Accessing elements                                                3955             27                  36           8954       
Auto-property initializers                                        3907             18                  27           4779       
Comparing Strings                                                 3638             35                  48           6792       
Ignoring files and directories with a .gitignore file             3335             27                  34           3781       
Null propagation                                                  3077             18                  21           6797       
Abstract equality / inequality and type conversion                2921             21                  29           1940       
Create a UILabel                                                  2847             15                  23           3086       
Avoiding the repetition of expensive operations using conditions  2475             14                  21           3128       
Hello World                                                       2438             23                  30           4037       
Null checking                                                     2321             14                  19           1699       
Installing packages                                               1965             12                  17           5521       
Creating variables and assigning values                           1895             24                  26           1679       
Arrow Functions                                                   1845             17                  18           6857       
Promise chaining                                                  1755             13                  18           2003       
Resource Routing (Basic)                                          1740             17                  20           5084       
Strings vs Unicode strings                                        1717             15                  18           4203       
Installing Java Development Kit                                   1661             21                  28           7692       
Unpacking Iterables                                               1625             14                  20           2999       
Creating a hash                                                   1600             16                  19           2814       
Passing data between activities                                   1576             19                  19           2895       
Hello World                                                       1576             10                  21           1684       
Attribute Selectors                                               1531             15                  18           8293       
Using the wildcard character to select all columns in a query.    1491             18                  22           1365       
Using std::tuple                                                  1488             12                  16           3204       
Introduction to Java lambdas                                      1479             14                  22           2864       
String & Character Literals                                       1430             12                  17           5011       
Hello World in C#                                                 1425             7                   9            213        
Catching an exception with try-catch-finally                      1393             21                  27           8958       
Exiting Vim                                                       1370             10                  13           2392       
Basic use and chaining                                            1360             9                   14           2316       
Print statement vs. Print function                                1251             11                  14           2965       
Creating a new project in Visual Studio (console application)     1243             14                  24           3234       
Overview                                                          1228             17                  23           3915       
Introduction                                                      1195             11                  14           3540       
Hello World                                                       1155             17                  28           3642       
Creating an Array from a Collection                               1140             13                  17           2993       
6 Simple Performance Improvements                                 1105             9                   11           8072       
HTML output from web server                                       1090             12                  16           1850       
Getting Started                                                   1085             13                  14           3250       
Image loading                                                     1050             9                   15           1070       
Decoding a JSON string                                            1021             11                  20           7334       
What is a lambda expression?                                      1015             15                  16           8025       
A basic build.gradle file                                         1010             10                  14           3589       
Hello, World!                                                     1005             15                  21           4969       
Radio Buttons                                                     1000             10                  15           4698       

To be clear, I'm not saying these are all problematic. But they are anomalous; we should examine them carefully when deciding whether or not they embody behavior we should be encouraging here... And then adjust the system as needed.

7
  • 13
    There will be a post from me, soon, which more clearly lays out what we envisioned with the rep system for docs as it was launched. The goals are really good, balance is tricky. As Shog noted, we're going to have to do a recalc, so we've got to be really careful to not inadvertently make the balance a bigger issue than it is - which means this is going to be at least a few weeks. I'll be posting this week at some point, there's a lot of stones to turn over before we can say we've turned 'em all, and a breadth-first search comes in quite handy there.
    – user50049
    Commented Aug 1, 2016 at 18:02
  • 2
    @Tim will you please address one aspect which I haven't really seen discussed yet: will a rep system adjustment be retroactive, meaning undone reputation for the masses? Or only concerning future gains? From "recalc" I'd suspect (and hope for) the former, but it would be nice to have an explicit heads-up about this. Thank you. Commented Aug 1, 2016 at 22:58
  • 7
    At this point, changes to the rep system will be retroactive, @Andras. Still a beta, still in flux. That said, rep system changes are never trivial; we'll talk about this a lot more before anything happens.
    – Shog9
    Commented Aug 1, 2016 at 23:11
  • 3
    Thanks for taking the time to respond - we appreciate the communication. Commented Aug 2, 2016 at 0:55
  • @TimPost, What about, for each N up votes, the rep per up vote is reduced by 1, so after so many up votes additional up votes give no more rep to anyone. Commented Aug 2, 2016 at 14:27
  • 1
    @TimPost I think a big point to to take into account is that a lot of the upvotes happened within the first few days (even hours) of the public beta launch when documentation was new and exciting. Although the rep system still needs tweaks, it seems like the big rush has slowed and people aren't earning crazy amounts of points anymore, at least not any crazier than the people who earn 100s of points daily for old SO questions.
    – Tot Zam
    Commented Aug 3, 2016 at 2:28
  • 3
    @TotZam It reminded me of the earlier days on SO (and every public beta we've ever launched). There's just this initial sort of frenzy and then everything settles down. I actually think the main problem is that the system isn't directing people to move examples, or create new topics entirely, once there's more than [x] examples in a topic. Had 3 of the largest been spread over 4 - 5 each, I don't think the problem would have been nearly as pronounced. We're looking at tweaks there now.
    – user50049
    Commented Aug 3, 2016 at 3:18
173

Idea 5: Keep Q&A rep and Examples Documentation rep in different buckets

Asking good questions and writing great answers is the purpose of Stack Overflow. Seven years on (is it eight? nine? I don't even remember any more) the whole concept is well understood at this point.

Not that writing good examples documentation is not important, but it is a very different animal. Examples Documentation is highly speculative, in a way that traditional Q&A fundamentally is not. In Q&A, some real person asked a question, with our strict SO definition of "useful / researched / clear" -- a fairly high bar relative to the Internet -- and it got answered. This isn't someone guessing "hey someone might want to know X so I'll just type a bunch of stuff here just in case", it is a direct response to a direct question by an actual person.

Documentation, on the other hand, is answering a set of questions that we're not sure anyone is even asking! Now I am not arguing that this is right, wrong, good, bad, or guy with the gun, but that it is just ... different.

To the extent that the differences between examples documentation and Q&A is devaluing rep people earned by asking questions and answering them -- the primary activity in the system, the reason people come to Stack Overflow in the first place -- that is a concern. Wouldn't you be upset, too, if you earned 10k rep through a series of great answers, and then were directly compared to a user who earned 10k rep exclusively through writing examples documentation?

Perhaps merging these reputation systems for fundamentally different activities might be the source of a lot of the current friction?

39
  • 78
    "Perhaps merging <...> be the source of a lot of the current friction?" You bet. When I see "newbies" getting buckets of rep for trivial edits to documentation, it devalues the hard-earned Q/A rep.
    – Cerbrus
    Commented Jul 29, 2016 at 9:26
  • 15
    Thank you! (The author of this comment wishes to apologize for a complete lack of constructivity.)
    – TerraPass
    Commented Jul 29, 2016 at 9:27
  • 12
    Sad part is the disproportionate number of very low rep users doing the guessing of what is needed vs higher rep folks that have a better finger on the pulse for recurring issues
    – charlietfl
    Commented Jul 29, 2016 at 12:30
  • 8
    If being on a different site of the network which merely changes the topic is a good enough reason to have a separate bucket of reputation there, why should documentation and Q&A have a shared bucket? Split reputation already! Then figure out how to fix documentation rep before the database explodes from the overflowing reputation column in the users table.
    – null
    Commented Jul 29, 2016 at 16:14
  • 11
    I agree. I made an innocuous edit to an example in documentation the day it went live, when I didn't really know that there was rep involved. I got the +2 when the edit was approved and though it was weird, but well, it was little rep. But then I started getting rep from upvotes to the example, that's just not right, my edit was minimal, I want to give that rep back
    – Lamak
    Commented Jul 29, 2016 at 16:14
  • 7
    I truly truly hope that at least the creator of this dang thing has enough clout to finally put this in motion, as it appears from the lack of inaction that the community sure doesn't. Thank you @JeffAtwood for finally speaking up for the rest of us. Commented Jul 29, 2016 at 16:30
  • 17
    "Perhaps merging these reputation systems for fundamentally different activities might be the source of a lot of the current friction?" Certainly. But that's not the problem. Or rather, that's not the only problem. Divorcing Docs.SO rep from SO rep will help Q&A, but it still leaves Docs.SO with its own rather ludicrous rep system, where you get rep forever even if your changes have been obliterated. It still encourages making small, trivial changes to important examples rather than making new examples that will never bee seen. It just won't hurt Q&A with its terribleness. Commented Jul 29, 2016 at 16:38
  • 28
    @NicolBolas perhaps, but separating the rep systems makes it easier to revamp one or the other without breaking both. Commented Jul 29, 2016 at 18:14
  • 7
  • 5
    on the other hand if you make reputation system on the Docs more fair there will be no need to make the SO and docs reputations separate. Commented Jul 30, 2016 at 18:23
  • 6
    Why not just admit that the current reputation system for docs is poorly designed? The best way to earn lots of reputation on docs is making trivial edits to many high-profile examples and that's a problem that exists even if you put it in a different bucket. Not only because those people are rewarded for the behavior, but also because it floods the system with many pointless revisions.
    – Radiodef
    Commented Jul 30, 2016 at 22:54
  • 3
    @Cerbrus Here I disagree. The merging itself isn't the problem. The problem is in merging an unbalanced reputation system. Fix the docs reputation system, not the merge.
    – bwoebi
    Commented Jul 31, 2016 at 5:00
  • 6
    @uoɥʇʎPʎzɐɹC the satisfaction of having Rep in a Technical Writing bucket. So far, I haven't seen much Technical Writing talent. I am sure it is out there, but I haven't stumbled onto it yet.
    – Drew
    Commented Aug 1, 2016 at 0:03
  • 11
    @uoɥʇʎPʎzɐɹC considering that nearly all of your rep appears to have been gained from activities other than asking or answering questions (aka, documentation), I am not surprised at your objections.
    – David L
    Commented Aug 1, 2016 at 1:59
  • 5
    @uoɥʇʎPʎzɐɹC Most high-rep users get their rep from a few highly upvoted posts actually, no. Read the accepted answer to meta.stackexchange.com/questions/136059/…
    – Pekka
    Commented Aug 1, 2016 at 16:52
37

Well then I have not written on meta Stack Overflow for a long long time... but here I am ... so ... OK.

This is insane on about every level.

enter image description here

Why does this user have 272 rep?

  • Graph shows rep is moving up
  • 2 answers, both have zero votes
  • A lot of rep activity

Conclusion, rep-to-clause gave this user a bunch of rep.

It is very scary to discover all rep came from:

enter image description here

My recommendation:

  • Effective immediately stop granting rep to groups of users for documentation. An upvote of an example should not "magically" print money for 50 people, that is crazy

  • Always show why users are getting a bunch of rep in the activity screen, rep-to-clause is crazy.

  • Either go with @codinghorror's 2 buckets system, or @hlovdal new UI for voting up edits, which requires drilling in. (meaning upvotes that make money for the editor will be harder to upvote)

Granting Rep to "groups" of users in one go is a huge departure from the existing rep system. This is the core of the problem. Imagine if community wiki upvotes granted rep to every editor on upvote.

Also, I wonder, why not simply use the suggested edit rep system here (with slight tweaks), which already existed and had precedent?

2
  • 1
    Sam, I think you don't need to fear too much. The gents at Corporate are pondering the re-jiggering from what I hear.
    – Drew
    Commented Aug 1, 2016 at 0:17
  • 2
    On the other hand, maybe it has never been fair that significantly improving a CW post goes without any reward. Commented Aug 3, 2016 at 20:23
12

Docs should fill gaps in official documentation, rather than trying to replace it wholesale. Instead of being the place where anyone can contribute, and farm rep, it should allow intermediate and above users in a field to craft expertly-written content:

  1. Docs contributions are written by people with at least 1k rep (or an appropriate percentage of the rep range the tag has, whichever is lower) in the tag from answers
  2. Docs contributions are worth no rep by themselves
  3. Docs can be edited for technical inaccuracies/omissons, lack of clarity and lack of "flow" and references between sections of documentation
  4. Docs sections can be used directly as answers to SO Q&A questions, or embedded as subsections inside answers. Upvotes to these answers will provide the Docs contributor with Q&A rep
  5. Docs are versioned according to the tech version they refer to, so for example embedding PHP4 documentation in a PHP4 question will keep the same docs embedded, rather than have it change "underneath" the answer, when PHP5 content is added. Edits to that version are applied to the embedded text, of course
  6. Question and embedded Docs answer will be the preferred method of self-answering questions, where possible
3
  • Point 1 is too restrictive for small tags. Take ABAP for example, where no users are over 1k rep, and only four of which have over 100, two of which being relatively inactive.
    – gkubed
    Commented Aug 2, 2016 at 12:45
  • 1
    I think the key is that docs are there to answer questions and should only get rep when they do. Commented Aug 2, 2016 at 14:29
  • 1
    @gkubed fair point - perhaps it could scale with the range of rep available - e.g. top X% of rep or 1000 rep, whichever is lower?
    – Rob Grant
    Commented Aug 2, 2016 at 16:43
9

Idea 3: Scale rep awarded per-upvote, based on number of editors

If an example only has 5 or so editors, each person will be awarded the full 5 reputation for every upvote. But, once an example reaches 10 or so editors, each upvote may only be worth 3 reputation; and when there are 20 editors each upvote will only be worth 1 reputation.

This way, the amount of reputation received from upvotes is proportional to your contribution to the example. Alternative

Instead, reputation per upvote could be scaled by the amount you have contributed to an example: if you have only contributed a few words, you only get a few reputation; if you wrote most of the documentation, you will get a larger share of the reputation awarded. This would probably be a better better way to scale rep, but is more complex.

I'd take this even further.

In my opinion, if the documentation reputation were to stay in the same bucket, then a single upvote on an example shouldn't increase the sum of total reputation in the system by more than 5 (or constant N).

What I've seen is that the more editors there is to an example, the worse it usually becomes. It is just sawing between acceptable and unacceptable - invalid edits, and then deletions, yet all of them would continue (as far as I know) receive reputation from upvotes. An example is not made 50 times more reputable by 50 editors "fixing" it again and again than it was if the exact same content were thoughtfully written by one person in one revision.

I was thinking of a quite complicated algorithm that after each edit would simply "git blame" the current revision - count how many characters each editor would have contributed to the current state. Then for each upvote, basically each editor would get lottery tickets based on how much of the example was theirs. This can be made stable so that the reputation recalculation always produces the same result. Then the 5 reputation points from upvotes would be allocated by drawing lots, either all 5 points, or one reputation point a time.

As an example, in revision R, 68 % of the example text is provided by me, and 32 % by others. Upvotes to revision R would either

  • give me all 5 reputation points by 68 % probability, 32 % probability to the others.
  • draw each reputation point separately, most often giving me 3 or 4 reputation, sometimes 0, 1, 2, or 5 reputation points and the rest with others.

I am not sure how easy it is to make this scalable enough for Stack Overflow, but let's just ignore that for a moment.

3
  • 4
    Precedent: community wiki posts already display a contribution percentage for the original author, though of course it doesn't award rep (maybe that will change if Docs continues to do so?). On the other hand I recall a meta topic where someone complained someone else was getting partial credit by this metric despite their edit having negative value (and getting mostly reverted). Commented Aug 1, 2016 at 5:24
  • @JeffreyBosboom my algorithm would calculate each delta in the final result. If someone's edit was mostly reverted but then contains one retained fix, say a to an, then they'd get some fraction of percent of upvote rep, but not 5. Commented Aug 1, 2016 at 5:29
  • 1
    The lack of reputational enhancement for Community Wiki posts has always been one of the least comprehensible parts of SO - it's like contributions aren't wanted, or something
    – holdenweb
    Commented Aug 2, 2016 at 8:02
4

The problem is, any solutions which "fix" the reputation inflation motivate undesirable behavior or demotivate desirable behavior.

Only reward editors whose contributions still make up a substantial part of the example

Someone creates an awesome example, over the time it evolves somewhat: some sentences are reworded, typos are fixed, variables are renamed etc. Eventually SO's algorithm determines that "nothing is left", although for human readers the example remains essentially the same.

Result: less people want to create new examples and topics, it's more profitable to edit existing topics. This happens already, so the change will just make things worse.

Introduce a rep cap per-example

You'll gain more rep from creating new examples.

Result: less people will be interested in improving existing examples, topics will be flooded by new examples. Cheaters will remove existing topics and create copies of them. It already happens too.

The rep cap could only be introduced when a large number of people edit an example

Again, less motivation for editing examples. Or maybe even finding your enemy's examples and editing them to deny reputation gains. And again, deleting and recreating becomes a profitable strategy.

Scale rep awarded per-upvote, based on number of editors

The same. Though this looks like the most reasonable one. Mostly because it's gradual.

Instead, reputation per upvote could be scaled by the amount you have contributed to an example

Nobody will fix typos. You gain nothing, others lose a lot. Currently only the first part is true.

Also small edits may be rejected - if I notice that somebody tries to edit my topic I will want to avoid losing rep.

(From comments) Plenty of people want the reputation removed from Documentation - I would also approve of such action

Then Documentation will die. It's already flooded by low-quality examples which provide zero value, but even that content will be destroyed. Nobody will care.


The only fair way of giving reputation is manually grading every single edit to every topic by professionals, and this is impossible for obvious reasons.

We're doomed. Have a nice day.

10
  • 9
    "Nobody will care." Is it so much worse than everybody caring just because of cheap reputation? I mean, Documentation might die, but at least it won't take down the entire SO rep economy with it. And then, the death of Documentation without rep is not a given, either: there are people, who will contribute regardless of reputation rewards. They might be even more encouraged to do so once we remove the incentive for the swarm of me-too editors, ready to vandalize (I mean, improve) other people's hard work to get on the rep train.
    – TerraPass
    Commented Jul 29, 2016 at 9:14
  • 1
    @TerraPass I doubt SO's devs/owners want Documentation to die, most likely they want more content to be generated as fast as possible to get something to work with. Disabling rep isn't an option because they want vandals (I mean, editors) to generate text. Once that stage is over, maybe something will be improved...
    – Athari
    Commented Jul 29, 2016 at 9:27
  • 8
    "Then Documentation will die." Then Docs.SO deserves to die. If it can't live on the quality of people who do work for free, without reputation, then it deserves to die. Wikipedia manages to work even though it has no rep system. cppreference.org manages to work even though it has no rep system. And so forth. Why does Docs.SO need one? Commented Jul 29, 2016 at 16:40
  • 7
    To me, these are all further arguments for keeping docs/examples rep in a separate bucket, as I covered in my answer. Commented Jul 29, 2016 at 18:20
  • 2
    @JeffAtwood Yes. There's no way to fix all this mess without making it even worse (at least I can't come up with any), unless reputations are separated. I'm just afraid it's not an option SO dev/owners consider. :-( My previous post critisized the "solution" which SO devs chose (less rep for edits), and how it didn't actually improve anything. It's also not only about devaluing rep, it's also about losing motivation for answering, something which everyone seem to forget - after literally a couple of edits, you get Jon Skeet's level of passive rep.
    – Athari
    Commented Jul 30, 2016 at 6:40
  • @Squidward this is why my answer is the way it is: no Docs rep (or not directly, anyway).
    – Rob Grant
    Commented Aug 16, 2016 at 21:38
  • @RobertGrant I don't think Stack Overflow owners are ready to kill Documentation just yet. :)
    – Athari
    Commented Aug 17, 2016 at 7:26
  • @Squidward not saying that! Does it read that way?
    – Rob Grant
    Commented Aug 17, 2016 at 8:06
  • @RobertGrant J/k. :) Activity in Documentation severely declined about 2 weeks ago, after about 2 weeks of crazy activity (judging by my reputation graph :) and notifications). I suspect that removing reputation bonuses will hurt activity even more, so it's unlikely SO's owners will take that route. Judging by messages in the Documentation chat, they're going to tweak various thresholds again.
    – Athari
    Commented Aug 17, 2016 at 9:09
  • @Squidward Oh, right :-) That's what I think's best: occasional activity by (relatively) expert users who can plug useful gaps.
    – Rob Grant
    Commented Aug 17, 2016 at 13:40
4

"Let's try to keep some perspective here".
No further comment.

documentation rep

6
  • That looks like a user that wasn't active at all before documentation. Without context, this graph says nothing. Heck, for all we know, that rep could come from bounties.
    – Cerbrus
    Commented Aug 3, 2016 at 7:05
  • 8
    @Cerbrus Most of the rep comes from "creating and initializing arrays", "using console.log()", "using window.alert()" and "Hello World". None of the rep comes from bounties. The user has been active for over 2 years, posting questions and answers regularly, and gaining rep at a normal pace. In the last 10 days he has gained as much rep as he normally does in a year. I don't want to drag anyone through the mud here, this is not about him/her, it's about the rep system. Commented Aug 3, 2016 at 7:19
  • From the look of that graph, he wasn't active at all in the weeks prior to documentation. How much did he have before docs? Also, the fact that he's consistently passing the rep cap, means he's doing more than just getting upvoted, since example votes can't pass the rep cap.
    – Cerbrus
    Commented Aug 3, 2016 at 7:22
  • 1
    @Cerbrus 6150 before docs (after 2.5+ years ) 9400+ after 2 weeks of docs. Commented Aug 3, 2016 at 7:36
  • Users can earn just as much rep on Q/A... Even more if you're lucky with a bounty.
    – Cerbrus
    Commented Aug 3, 2016 at 8:49
  • 6
    @Cerbrus Well, from July 21st to 31st he gained 3039 rep points (mostly from doc), while Jon Skeet gained 2550 (from his 33k+ Q&A posts). Commented Aug 3, 2016 at 21:32
-5

The core problem is that the data model for giving out reputation is fundamentally flawed: Currently reputation is given to all contributors based on votes on the end result. That is not good because it does not differentiate between contributions. Thus

Idea 6:Each contribution/edit should be rated by its own and reputation given to its author based on that only

This could be implemented as when someone votes on an example, a "Please spend a few seconds rating one or more of the edit to this" question appears1, and also possibly for each X-th visit of the page.

The rating could be

  • Improves the example very much
  • Improves the example moderately
  • Improves the example a little
  • Improves the example insignificantly
  • Makes the example worse.
  • Don't know/unsure.

and then some rules for converting such ratings into reputation.


1 Along with rating hint: "Rate for the lowest part, e.g. if the added text is great but the added code allows for SQL injection, rate it as makes it worse".

2
  • 13
    This would absolutely be the fairest way to do it, but I'm sorry, neither I nor, I strongly suspect, most everyone else, wants to spend hours reading diffs to decide which particular edits deserve votes.
    – jscs
    Commented Jul 29, 2016 at 22:38
  • 3
    This was also proposed several days ago already: meta.stackoverflow.com/a/329336
    – jscs
    Commented Jul 29, 2016 at 22:39
-20

Whatever you do, don't separate Stack Overflow reputation and Documentation reputation.

If they were separate who would contribute - the major incentive for getting SO rep is bounties + privileges.

Just because the rep system is broken doesn't mean it can't be fixed. You don't remove it, you fix it. You don't fix a car with a broken engine by converting it to a electric car and buying a charger. Removing it doesn't really address the problem - we got a gas car for a reason - and we don't want to forfeit the benefits.*

I think with the changes suggested in the original question the rep system can be fixed.

If Jeff's answer were posted by someone else I would expect it to be down voted heavily.

A lot of people have gone "Oh, this user got most of his rep from documentation, he's pretty biased." I think the rep I got from editing popular posts is temporary and don't expect it to stay. But look at this: I contributed several original (IMHO - you can judge for yourself) high-quality examples. Because I know in the long term I'd get more reputation from those examples, and the reputation from the popular ones would vanish in a recalculation.

For example (link cluster)

9
  • *and unlike in real life, the electric car industry (separating the systems) is a mature technology and was considered as an alternative. Commented Aug 1, 2016 at 0:01
  • Not a DV from me, cuz I am not in the mood. You asked me to look at your contributions. One of your last Here on main. I am focusing on Q&A this split second. You did a nice job helping the op and clearly explained the situation. Very insightful, brief. I loved it. You allowed yourself to be opened up to scrutiny, field questions, and get married to the problem. Fortunately, no problem. You were willing to enter the arena, you left like a champion. Nice job. Docs is another issue and is no arena. I think the reason why has been explained well by others
    – Drew
    Commented Aug 1, 2016 at 0:34
  • @Drew then look at my docs contribs Commented Aug 1, 2016 at 1:14
  • 11
    Like this second to last one you did :p that should be worth probably 50 to 100 rep, no?
    – Drew
    Commented Aug 1, 2016 at 1:29
  • @Drew Yeah, I think the rep from that is worthless. Look at the link I gave you. 100% original example. Here's the link again: stackoverflow.com/documentation/django/1693/… Commented Aug 1, 2016 at 16:35
  • 1
    I agree boss. You know where to find me if you want any of your high-quality stuff quickly reviewed. The queue can be a hassle otherwise :p
    – Drew
    Commented Aug 1, 2016 at 16:37
  • @Drew mostly to the upvoters of your comment and not to you :p Commented Aug 1, 2016 at 16:39
  • 3
    It was a cheap shot. I apologize. Understand though the frustration to put it mildly that the new system represents to those that gave so much. We understand gaining rep on Main is difficult. We need some balanced system that allows for new contributors without the whole thing becoming a slot machine.
    – Drew
    Commented Aug 1, 2016 at 16:42
  • 1
    @Drew this... this should be an answer. brief and concise. Commented Aug 1, 2016 at 16:44

Not the answer you're looking for? Browse other questions tagged .