92

In the blog post We’re Rewarding the Question Askers it is stated that:

Here’s the history: When Stack Overflow launched in 2007, we gave equal points to upvotes on answers and questions. Three years later, a decision was made to devalue upvote reputation on questions. The idea was that this change would encourage people to focus on providing good quality answers rather than asking questions.

What was the context of that decision? Why was it made? Why is it no longer the right call?

We can look back on this decision with the benefit of hindsight. This decision may have been the right call then with the information we had at the time, but we have seen the effects it has had on our community. We reward people who give answers at a higher rate than people that ask questions.

This begs the question as to why things are different now from the context of when the original change was made. We still want to focus on high quality answers in an attempt to optimize for pearls.

If it is at all possible, could the blog post be updated to point directly to the context around that decision for historical purposes?

2

1 Answer 1

102

The decision was made way back in 2010 by Jeff Atwood, one of the site's founders. (He has since moved on to other pursuits.) It was announced and explained in the blog post, "Important Reputation Rule Changes":

[…] question upvotes will now grant only +5 rep instead of +10.* There is no change to answer upvotes. This will apply retroactively to all users.

Why?

While we value good questions (and asking a great question is absolutely an art), we want to explicitly encourage people to provide the best possible answers. Without people interested in providing good answers, the questions are moot. We know that answers have more intrinsic value than questions, and the reputation balance should reflect that.

The question asker already enjoys a substantial benefit beyond reputation gain from upvotes on their question — namely, they get great answers to their question! Thus, the asker shouldn’t need as much reputation gain.

There are a few users who ask hundreds, sometimes even thousands of questions. Over time, these users generate a fairly sizable reputation entirely through the tiny trickle of upvotes gained by these questions. In a sense, we want to discourage question asking a little bit, and make sure that people who ask questions are doing it for the right reasons and not to generate reputation.

In other words, we’re rebalancing a bit to favor answers. Based on the existing data in the trilogy, I believe this will be a positive change for everyone. For more discussion see the meta topic.

* on Meta, the value of a question upvote will still be +10

The rationale was expanded further upon in a subsequent (rather famous) blog post, entitled "Optimizing For Pearls, Not Sand":

Perhaps you’ve noticed a theme here. Incoming questions are a universal constant, all around us in countless billions. But answers — truly brilliant, amazing, correct answers — are as rare as pearls. Thus, questions are merely the sand that produces the pearl. If we have learned anything in the last three years, it is that you optimize for pearls, not sand.

[…]

That’s why we’re determined to keep question quality high, even at the cost of refusing a little sand. It’s true that you can’t have Q&A; without questions, but having the wrong sorts of questions is far more dangerous. The fastest way to kill any Q&A; site is to flood it with low-quality questions. I think Mark Trapp summed it up best in this meta answer.

[…]

Fundamentally, answers can be filtered in ways that questions cannot. While there is a tension between having “enough” questions and a bunch of amazing, highly skilled answerers twiddling their thumbs waiting around for something to do, in the long run we’d much rather err on the side of having interesting and on-topic questions for these folks to sink their teeth into.

We feel that the world is awash in questions, but not answers. Answers are the real unit of work in any Q&A; system. Therefore, the only logical thing to do is to maximize the happiness and enjoyment of answerers. If this means aggressively downvoting or closing unworthy and uninteresting questions, so be it. Without a community of people willing to answer questions, it really doesn’t matter if there are questions at all, does it?

This is the history to which Sara refers in her most recent blog post. Stack Overflow is essentially reversing the change made back in 2010 to reduce the net reputation gain from upvotes on questions, and they're reversing it retroactively, so it will be as if it never happened.

This decision was made with the benefit of hindsight: Jeff's solution seemed logical enough at the time, but after nearly 10 years of it being the standing policy, there is now enough data that we can look back and judge whether it was really having its intended effect. Sara unfortunately doesn't go into very much detail about this data-led re-evaluation in her blog post, presumably because it is intended for a more general audience than the folks who read Meta.

The fact is that the changes made back in 2010 simply weren't having the intended effect of improving question quality. Reducing the amount of reputation gained from upvotes on questions did little or nothing to address the problem of users earning reputation from low-quality questions. Worse, it merely served to make it that much harder for users who were asking useful, high-quality questions to earn privileges.

Data tells us plainly that voting skews heavily towards answers. Even if the net reputation gains are made equal (as they will now be), folks who write good answers will still be earning reputation faster and more easily than folks who write good questions.

Furthermore, because the net reputation gained from upvotes has never been balanced with that which is lost from downvotes (in other words: upvote + downvote > 0), those users who asked low-quality questions were still coming out ahead, even after cutting the net reputation gain from upvotes in half (from +10 to +5).

In summary, Jeff Atwood's 2010 solution of lowering the reputation gained from question upvotes has not had the desired effect of reducing the volume of low-quality questions. Worse, it was throttling folks who were asking perfectly good questions just as much as it was throttling folks who were spamming the site with large numbers of terrible questions.

Fortunately, what did start to work was a variety of changes introduced gradually since 2010. These system-level tweaks include rate limits, quality filters, and question bans. All of these have sharply curtailed (perhaps not yet enough, but still measurably so) the influx of terrible questions, and they've done so without punishing users who were asking perfectly good questions. Recently, increased effort has been targeted specifically on addressing the issue of declining question quality, including some of the things that Sara calls out in her blog post: a wizard to guide users through the process of asking questions, improved post notices that do a better job of communicating why a question was closed, and better moderator tooling behind the scenes to deal with less-than-stellar contributions.

With these changes, and more in the pipeline, the Stack Exchange team felt it made sense to reassess the reputation system, and roll back an old policy that never managed to achieve its own lofty goals while unintentionally hurting users who participate effectively by asking useful questions.

23
  • 6
    Awesome, this is more or less what I was looking for. I knew that there had to have been discussions about it at the time, but wasn't sure where to look.
    – zero298
    Commented Nov 13, 2019 at 19:04
  • 37
    Good arguments, now this change makes sense, though I think we still need to increase the down-vote power overall, maybe to -3 or -4.
    – Krupip
    Commented Nov 13, 2019 at 20:46
  • 47
    Waah. We need Jeff back.
    – S.S. Anne
    Commented Nov 13, 2019 at 21:21
  • 5
    @CodyGray Thanks for providing those details. Note I asked a similar main meta question of Has there been any discussion regarding the original stated reasons for the question up votes being set at +5 from +10 earlier?. I've updated my question to provide links to this question and your answer. Commented Nov 14, 2019 at 1:08
  • 5
    "Reducing the amount of reputation gained from upvotes on questions did little or nothing to address the problem of users earning reputation from low-quality questions." Of course, if the question is not removed, sooner or later it will start gaining reputation regardless of its quality. Just the same increasing reputation will do even less for the question quality. (1/2)
    – Dalija Prasnikar Mod
    Commented Nov 14, 2019 at 11:02
  • 7
    Perceived problem of users asking high quality questions and not gaining much reputation is dubious. If the question is high quality and it is answered (has a solution) it will gain reputation over time. If it is about some obscure problem and in less popular tag, then it may not gain too much. But there is already huge discrepancy between popular and less popular technologies, so I don't see that as huge problem. At least not something that should be solved by giving ALL questions more reputation points. (2/2)
    – Dalija Prasnikar Mod
    Commented Nov 14, 2019 at 11:05
  • 3
    Introducing some bounty system for rewarding good questions would be better solution. If there is a need to give more reputation to great questions.
    – Dalija Prasnikar Mod
    Commented Nov 14, 2019 at 11:06
  • 2
    yeah yeah yeah, but not ReTrOaCtiVeLy!!!!!! make a change however you like it, don't change the very history that already happened. There was even a book written about this subject, relatively unobscure one at that. By a relatively not that unknown an author. Don't you understand how bad it is, to change history??? Not? Why????????
    – Will Ness
    Commented Nov 14, 2019 at 17:35
  • 5
    @Will The change was made retroactive in the interest of fairness. If we're going to reinvent the reputation system, it only makes sense to hold everyone to the same standard. Putting aside my feelings about the necessity of this change, I cannot for a moment imagine why it would ever make sense to do so only from a certain point forward. Across the entire history of this site, whenever the reputation system was tweaked, all of the changes were applied retroactively. Changing history is dangerous if you're denying what happened. No one's denying anything here.
    – Cody Gray Mod
    Commented Nov 14, 2019 at 17:52
  • 9
    There's also a very practical concern, @Will. Reputation recalculations are triggered dynamically all the time. This accounts for cases where you downvote an answer (thus reducing your rep by 1), then that answer is deleted (which refunds the -1 rep loss). If these changes to the scoring system weren't retroactive, you'd have all kinds of weird behavior. Users would start randomly gaining reputation whenever the system triggered a behind-the-scenes rep recalculation, and that would just be confusing.
    – Cody Gray Mod
    Commented Nov 14, 2019 at 17:55
  • 4
    It seems obviously to be the question itself, @Makoto. Why should questions retroactively gain value because they have been answered? The question is the seed that enables the answer to grow. You're buying Jeff Atwood's reasoning lock, stock, and barrel. Which I'm not blaming you for, because I did, too. But there's just as much hand-waving in his blog post as there is in Sara's, if not more. Question-askers are put at a systematic disadvantage in terms of rep, and the data shows the purported justification (discourage low-quality questions) didn't pan out. So...why keep doing it?
    – Cody Gray Mod
    Commented Nov 14, 2019 at 18:39
  • 4
    "lowering the reputation gained from question upvotes has not had the desired effect" - do you have any evidence to suggest the volume of low-quality questions would not have been much higher without that change?
    – NotThatGuy
    Commented Nov 14, 2019 at 18:45
  • 5
    @CodyGray: This is an interesting perspective. My belief on the reasoning was that the answerer's time was considered to be more valuable and thus more respected, thus preventing the circumstance in which "OP" could gain tons of rep from only asking questions. In another light, think of a lecture hall full of students. The obvious value in the lecture hall is not the students, but the lecturer, otherwise there would be no point to having the lecturer.
    – Makoto
    Commented Nov 14, 2019 at 18:49
  • 34
    @CodyGray: To suggest that question-askers are at a "disadvantage" is mischaracterizing it. They are the disadvantaged. They're the ones asking the question, and the whole value that they get from doing so is an answer. The answerers are the ones who are far rarer and their time is far more valuable, since they're the ones who are sharing their expertise and their time with others. This relationship cannot be shifted because of its very nature. If you're asking a question, you're subconsciously asking for expertise, time and some level of investment.
    – Makoto
    Commented Nov 14, 2019 at 18:51
  • 16
    @CodyGray: To then shift that relationship and suggest that there can be some kind of equivalence is fundamentally flawed. While this may be a silly discussion on some silly internet points, it definitely does impact how I view them from now on.
    – Makoto
    Commented Nov 14, 2019 at 18:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .