248

I think I can speak for many users to state that the current system of reputation gain from Documentation is implemented such that users can take advantage of it for easy reputation gain. I've seen many users make very minor edits, some do not help improve the quality, on many of the popular examples. As such some users have been gaining 200 reputation a day for doing practically nothing.

While there are several users who do improve the quality of examples and make important and helpful changes. I've seen many edits that where made "Just to edit", meaning it was clear that user made a meaningless edit purely for the sake to gain reputation from that example. In fact I've seen this happen so much on popular examples that the quality of the example has been decreasing.

I don't think it's possible to separate "bad" edits from "good" edits. I do believe if this where to go on longer without any changes it will completely remove the meaning that reputation has on Stack Overflow.


Since Documentation was released to public beta recently, the current damages are not too drastic. But I don't know how will it be two weeks or a month from now. I do think something needs to change and done so quickly. Some of the ideas for changes include:

  • Completely remove reputation gain from Documentation but add badges.

  • Separate reputation gain from the main site, having specific Documentation reputation.

  • Remove up-vote reputation gain, but keep reputation gain from edits.

  • Limit the amount of reputation gain a edit/user can get from up-votes for a specific example.

  • Provide a Reputation cap for Documentation. (ex: A user over 2,000 reputation will not gain reputation from Documentation). Similar to how edits work on the main Q&A site.

  • Only allow substantive edits (not minor ones) to benefit from up-vote reputation (Although I can see users getting around this).


I feel that the current system encourages "Just to edit" behavior in order to get reputation, rather than edits to improve the quality of documentation.

23
  • 31
    The substantive-edit requirement is already in place, although the bar is fairly arbitrary - need more data to put it in the right place, but expect that to happen. Caps of some sort will almost certainly happen eventually too. Still sez "beta" in the name, so expect whatever rep you earn will be subject to adjustment at some point.
    – Shog9
    Commented Jul 25, 2016 at 21:09
  • 98
    My current opinion is that Wiki-like collaborative efforts and rep-gamification don't go very well together because collaborative editing requires truly altruistic behavior (for the greater good) while rep-gamification promotes egoistic behavior (to some extent) and both don't work well together. So the simple way out would be to reduce rep gains so much that they do not really matter. Could be done right away. - The alternative - we vote on the edits instead of voting on the results, but that may result in a lot of voting. ;) Commented Jul 25, 2016 at 21:14
  • 7
    @Shog9 Thanks, I was a bit worried that the reputation would be locked. Good to know that it's subject to change. Commented Jul 25, 2016 at 21:14
  • 5
    @Trilarion, we have users abusing +2 rep gains from suggested edits already. Even a symbolic +1 rep would be widely abused given the current state of the documentation subsystem I believe. Substantive-edit requirements won't do much when you can have two users editing then reverting and both gaining rep from doing so, I'm afraid. But the devs may have a few cards left up their sleeves, I don't know. Commented Jul 25, 2016 at 21:29
  • 14
  • 2
    Related How much reputation is given per documentation post per day - convert to community wiki BTW, this comment counts as an upvote for Trilarion comment... I've hitting the limit 3 days straight!
    – Braiam
    Commented Jul 25, 2016 at 21:36
  • 7
    I made a small (albeit important) edit to one of the Python articles. I was surprised when I came back here the next day to see something like +120 rep just from that. I like my fake internet points as much as the next guy, but I don't believe I deserved that large of a reward for what I did. IMO, +5 rep/upvote for an edit is way too much... Commented Jul 26, 2016 at 13:59
  • 25
    @Hack-R No, I think folk are afraid that the means SO put in place to ensure rep had to be earned, & could (however loosely) be trusted as an indication that a user typically deserves said rep due to being skilled & making quality posts, - namely, the entire basis of the rep system & hence the site - ..are now under threat from a wholly unrelated & totally half-baked feature, which the Powers have decided - for some reason - must be inextricably interwoven w/ the (formerly?) main point of the site, despite being almost totally incompatible even if done right... while still in beta. Commented Jul 26, 2016 at 21:39
  • 12
    ...a rash decision that I've seen some people defend by saying 'it was just too complicated' to give Documentation its own rep system, so the second-best solution is of course to pollute the entire (again, formerly?) main system of the site with the effluent from a barely thought-through beta, because - I dunno - that's the kind of thing smart people do, I guess. Clearly I'm overlooking the true genius just below the surface here. Commented Jul 26, 2016 at 21:41
  • 5
    Please cap reputation gains to a flat amount. +5 rep/edit. Period. Commented Jul 27, 2016 at 1:16
  • 25
    I have made one single edit on StackOverflow Documetation (not very big either), and I gained two new privileges and nearly 700+ reputation. This is very shocking...my reputation graph has a near vertical rise...
    – user3117575
    Commented Jul 27, 2016 at 3:29
  • 4
    I agree that the incredible gains some users have seen are overkill. However I wanted to let you guys know that in some languages (tags) we see no such phenomenon and you're lucky if 3 hours spent on 3 high quality examples/topics will gain you even 5 - 10 rep. Since on SE we essentially provide free labor in exchange for points / fun I think that removing rep from Docs would doom it. Instead perhaps we could set a max rep amount per example, such as 100 or 200. It wouldn't be a bad idea for posts in general, though it would be too big of a change to implement after all these years for Q&A.
    – Hack-R
    Commented Jul 27, 2016 at 13:42
  • 2
    Someone made an upgrade to documentation I wrote, which actually invalidates the information that is meant to be shown - and actually will give you an error, so I feel as though in combination with some work on the reputation system being used, there needs to be better quality control - but I'm sure that it's very difficult to achieve that. Commented Jul 28, 2016 at 2:33
  • 16
    Please take away my reputation. I made a small handful of positive yet for the most part not incredibly substantial edits to documentation, and now I am hitting the rep cap of 200 every day without doing anything. Following the trend, it will be exactly 1 month until I hit 20k for a total of about 2 hours worth of improvement on documentation, and people who started from zero and did the same will be more than halfway to 10k. Looking for a Stack Overflow team member to mercilessly eviscerate my documentation reputation along with everyone else's. That is what I fantasize about. Thanks. Commented Jul 28, 2016 at 18:23
  • 2
    See examples of such suggestions 10 reputation for topic editing is too high , Should rep gains be turned off for Documentation's seeding period? and Reduce or don't award reputation for Docs upvotes. I hope SO Management will choose one of the ideas ASAP. Commented Jul 30, 2016 at 14:02

3 Answers 3

137

If you're going to use reputation as an incentive for documentation purposes, you have to make sure that it encourages useful forms of participation.

We don't give rep for asking questions; we give rep for asking good questions, as defined by peer review (ie: someone saw it and upvoted it). Same goes for answers and the like. This in theory encourages good questions and discourages bad ones, by allowing you to be penalized for them.

Such a system is viable, but only so long as a particular post really, truly is yours. And while we do allow people to edit questions and answers, these are intended to be used to correct errors, not to fundamentally change the material.

If you start providing rep for something that isn't really yours, then reputation is no longer an incentive. Or at least, it's not an incentive to do useful things.

For example, you post an example. But your example isn't that good. Then someone comes along and basically rewrites everything you did, making the example better, but it is also wholly original. And then someone upvotes it.

Who should get the rep? If we give it to both of you, then what behavior are we encouraging? We're not encouraging large-scale edit rewrites, no. We would be encouraging being first to make examples. Why?

Because someone is far more likely to take your crap example and make it better than to remake the exact same example under that topic. If you are the first person to post a "regex string replacement" example, then you will get rep when someone comes along and makes your example better.

Welcome to FGITW, the Docs.SO version.

At the same time, if you come along and make a spelling fix that really doesn't make the example significantly easier to read, you get rep if the post gets any more upvotes. Even though you really didn't exhibit any expertise.

And yet, you can't simply say that a person deserves rep based on how much of the example they changed. Why? Because if you discover a small-but-serious bug in an example, we want people to make such an edit. It may only be changing one word for another, but in terms of the meaning of the code, it goes from being broken to being correct. That was a really important change, and it's hard to argue that you don't deserve rep for it.

So you can't use some heuristic like number of characters changed to decide if someone should get rep from the change.

I don't really see an effective solution to this problem. It seems like there will always be plenty of opportunity for bad behavior. And really, an incentive for bad behavior is far worse than no incentive at all.

22
  • 37
    I fell that I have heard this same argument many times over the previous days and wonder "Why nobody hear these guys"...
    – Braiam
    Commented Jul 25, 2016 at 21:37
  • 28
    In addition, since nobody has made that argument AFAICT, I will do it. Remember all the requests for a Stack Overflow à deux vitesses? Like, one site for the experts and another for everyone else? Remember how we always decline these requests, and why we do so? Well, Documentation manages to implement exactly that in the worst way possible -- by segregating away experts due to disputable contributions from everyone else, and merging the two groups' reputation gain. Commented Jul 25, 2016 at 21:40
  • 67
    The solution obviously is to present the upvoter a menu with hundred slider bars - one for each contributor. The upvoter will then need to look at the edit history and decide how the rep from his/her upvote will be distributed among all of them.
    – Mysticial
    Commented Jul 25, 2016 at 22:06
  • 15
    This was proposed in all seriousness, @Mysticial: meta.stackoverflow.com/a/329336
    – jscs
    Commented Jul 25, 2016 at 22:08
  • 18
    @JoshCaswell AHAHAHAHAHAAHHHAHAHA
    – Mysticial
    Commented Jul 25, 2016 at 22:10
  • 16
    @Mysticial Heh. Dividing 10 rep 100 ways. I doubt anyone envisioned ever needing to store fractional rep in the db int field. :) Why don't we just give 1 rep for a doc upvote instead of 10, AND cap that to 25/day.
    – user4151918
    Commented Jul 25, 2016 at 22:16
  • 1
    "So you can't use some heuristic like number of characters changed to decide if someone should get rep from the change." I disagree. I think that's exactly what we should do. Sure it's not perfect, but I think making rep gain proportional to number of characters added is far better than the status quo of just assuming all "significant" contributions to an example are equally valuable. Let's not let the perfect be the enemy of the good.
    – Ajedi32
    Commented Jul 26, 2016 at 13:06
  • 3
    IMO there should be distinction between formatting/spelling and similar edits and edits that contribute knowledge and know-how. Only second type of edits should gain reputation and first ones should only get one time editing reputation.
    – Dalija Prasnikar Mod
    Commented Jul 26, 2016 at 13:11
  • 4
    Seems more closely analogous to cybersquatting that FGITW en.wikipedia.org/wiki/Cybersquatting
    – Frank
    Commented Jul 26, 2016 at 15:16
  • 53
    "an incentive for bad behavior is far worse than no incentive at all." <- THIS. Read it and weep, SO. And then remove reputation from this unless you prove you can do it properly. You have manifestly failed to do so thus far, and by putting live a beta scheme that contradicts your whole (former?) goal, you're making the place look like a laughing stock. Do you really want to encourage yet more valuable high-rep users to quit? And the resulting void to be filled by minor edit leeches? What a mess. Commented Jul 26, 2016 at 21:45
  • 4
    @DocSalvager: The ability to downvote does not cause examples to disappear. They simply move down in the sort order. Furthermore, a pop-up appears when you downvote an example which permits you to explain what the downvote is for, in the form of a request for improvement. So that's two ways what you said doesn't make sense. Commented Jul 27, 2016 at 4:10
  • 7
    @DocSalvager: "How does downvoting improve quality in a way that upvoting does not?" It solves the problem of not being able to lower something that doesn't deserve to be higher. Sure, you could upvote everything else, but that's ridiculous. You want to say "this is crap" not "everything else is better". Commented Jul 27, 2016 at 4:52
  • 2
    @DocSalvager because comments can be deleted for any reason, they aren't everlasting.
    – Braiam
    Commented Jul 27, 2016 at 13:11
  • 10
    I think it's important to mention the unlocking of SO moderation privileges via rep earned in Docs.
    – CubeJockey
    Commented Jul 27, 2016 at 13:32
  • 3
    Giving rep to every editor of an example also subverts the daily upvote limits. With a few upvotes of popular examples, an upvoter can distribute literally thousands of rep points.
    – samgak
    Commented Jul 27, 2016 at 16:31
35

This!

  • Separate reputation gain from the main site, having specific Documentation reputation

Then you at least contain the damage from the premature beta release - while you fix the bugs on documentation.

And drop the marketing bs about documentation and SO being "one site". That doesn't make the reputation systems any more compatible. At best they complement each other.

We have separate reputation on all the *.SO sites already. That should work just fine.

3
  • 3
    Best suggestion. Edits that don't require much of anything should not be awarded reputation of same value as that obtained by answering or asking useful questions. Commented Jul 29, 2016 at 9:51
  • 1
    I agree with this answer, but "marketing bs" seems unnecessarily pejorative. Commented Aug 9, 2016 at 21:38
  • @WayneConrad: "benevolent statement" doesn't seem awfully pejorative now, does it? Commented Aug 10, 2016 at 20:13
7

I totally agree with this answer. If they keep rep from Documentation linked to SO it will effectively mean rep has no correlation to programming ability and we have enough struggle trying to make a meaningful link with that as it is. Usually between juggling work and/or study. It takes an effort and commitment to the site for programmers to visit the site, assist in moderating the site and make contributions.

If this continues, soon the site will be run by people with less and less programming ability and enough time on their hands to be able to grab and regurgitate examples and documentation from other sources. For most people who are programming for a living or studying, it's a struggle and effort to make time to write decent programming question and answers.

I know that when I come to meta or take time out to answer a question, I am stopping my work, my study, my family time, to make that effort. I do so because I have believed in this site and the contributions is makes for the global programming community and I have respect for the overseers, the high rep users who have been here for years.

Without true programming experts rising to the ranks of the highest rep users, the site has not real credibility as a programming site and although there is a gap between these top users and the majority of users.

enter image description here This state of the art image demonstrates the increasing trends of gaining rep without posting questions and answers.

As with the current documentation gains and with suggested edit rewards, it is quite feasible for a user to gain the majority of a 5K rep without posting a question or answer, if the documentation rep remains a part of SO rep.

There will become an increasing doubt of skill level cast upon the users up to the 10K and then approaching the 25K with the ability to earn rep in so many ways other than by providing decent content in the form of questions and answers.

Caveat: This is not intended to slur the experts who have given their time and expertise to write good content in documentation, this is my take on the rep gathering (formerly known as rep whoring) of lower rep users and the associated activity on Documentation.

10
  • 2
    Exactly. It took me 2 years to get above 10K, and often my reputation tends to help me job-wise. Now people will be may be able to get that in a few months and not have to do anything. Commented Aug 4, 2016 at 17:02
  • @SpencerWieczorek I totally agree, my rep started a few years ago and I went with the CV idea here on SO, but it is only meaningful if the rep is reflective of our ability. Thanks for asking this question!
    – user3956566
    Commented Aug 4, 2016 at 17:05
  • @Spencer Docs rep is subject to the same daily cap as upvotes on Q&A. Maxing out, it takes 50 days to 10k via either kind of vote. In fact, though, you can get there faster on Q&A because the +15 answer acceptance bonus is not subject to the cap.
    – jscs
    Commented Aug 4, 2016 at 17:22
  • @Yvette: I think the Docs reward structure of "in perpetuity, for anything you ever touched" is totally out of whack, but I really don't like your characterization of people writing Docs as ipso facto not knowing what they're talking about. Plagiarism is a critical problem (note, it's also something of a problem on Q&A), yes. But to write a good entry on Docs takes as much time from one's life -- possibly more -- as writing a good answer on Q&A. I spent more than an hour -- actually more time than I meant to -- creating two examples the other day, and I know that there are others...
    – jscs
    Commented Aug 4, 2016 at 17:30
  • ...like me who are doing and will do similarly. Rubber Duck, Nicol Bolas and enderland have posted on Meta on this idea: that there are knowledgeable, good writers who would love to participate in Docs. There's no reason not to have a reward system for good contributors. We just need to make sure that bad contributions are not rewarded.
    – jscs
    Commented Aug 4, 2016 at 17:32
  • @JoshCaswell I have no issue with experts writing in docs and look at your rep? You have demonstrated your ability publicly as a programmer, as much as one can, using the rep on this site as a guidelines. As you note it's the people with less rep gaining rep from the docs participation, with perhaps less knowledge than many or the experts here, doing so in the hope of purely gaining rep, rather than contributing wisely and purposefully for the betterment of a global programming community. You have focused on the negative and taken that personally, when I mention the great respect I have for ..
    – user3956566
    Commented Aug 4, 2016 at 17:38
  • .. programmers of your ilk. Kudos and thanks for making that effort. (and the perhaps less - when referencing knowledge of lower rep users, was intended to mean a lot less)
    – user3956566
    Commented Aug 4, 2016 at 17:39
  • @JoshCaswell added a caveat to my post
    – user3956566
    Commented Aug 4, 2016 at 17:45
  • I'm sorry, I didn't mean to give the impression that I was taking your remarks personally, but thanks for your praise (your own contributions are appreciated too!) Let me try to clarify: I share your concern about the dilution of rep, and poor-quality material being rewarded. I disagree solely with the idea of Q&A being a better demonstration of know-how than Docs. I believe that there are programmers who know their stuff who can contribute to the global community more effectively in the Docs format.
    – jscs
    Commented Aug 4, 2016 at 17:48
  • @JoshCaswell yes I understand what you're saying re; the documentation. Ironically many of the people would lovingly write docs on their technology of choice would do it for zero rep,as programming can be a passion and a love. Also it's good to see how an argument's position can be taken, as my viewpoint could be considered offensive, without thought to the people who have put in a lot of effort. At the end of the day, the caretakers of the site are all striving for the same thing. So I guess we're all trying to nurture and protect it.
    – user3956566
    Commented Aug 4, 2016 at 17:53

Not the answer you're looking for? Browse other questions tagged .