58

The documentation voting system reminds me somehow of "Hot Network Questions" and their answers:

You vote it up because others voted it.

The result can be seen for example in Javas Arrays topic:


The voting system meets the requirements perfectly for Q&A sits:

Good, interesting, well written and right answer are voted up and rise to the top. The best answers show up first so that they are always easy to find.

But the key is documentation examples are different:

Good, interesting, well written and right examples are only voted up and rise to the top because they are on top. This is because most of us tend to vote the top two examples (me too), even if others are equally helpful too.


Even if some of us start to up-vote other examples, they really have no chance to get under the top three. As @Frank pointed out, why bother editing other examples or adding new ones if no one is going to look at them: We – as an editor – gain much less voting for them. Instead if you really want to get reputation, you should try to edit one of the top examples …

On the other hand it’s the question why should we up vote an example? All examples are equal: the example I find interesting or useful could be uninteresting for you because you already know the described things. @samgak pointed out that the "sort by most popular" functionality should be removed from documentation since the only purpose this "stack overflow"-style serves for someone using the documentation is to find the best topics to edit for maximum reputation.

Another thing is as @ken2k mentioned in his comment, if you down-vote an unclear, incomplete, overly-broad example, from the day on it is vastly improved by someone else, it still has your down-vote. Although we can argue this problem applies to Q&A too, an example on the documentation is more likely and also supposed to be edited by many people to be eventfully drastically changed compared to ordinary answers or questions.

The question is now, do we need to adapt our voting system for documentation examples?

4
  • 4
    That's my impression as well, and I think it distorts the pattern of edits, too. Why bother editing other examples or adding new ones if no one's going to bother looking at, much less voting for, them? This seems related: meta.stackoverflow.com/a/329457
    – Frank
    Commented Aug 2, 2016 at 22:14
  • 1
    The question of the "expected goal of the voting system" seems to be more support than discussion. Are you looking for a discussion about the voting system (i.e. reputation in documentation), or are you looking for an explanation of what the expectation was?
    – Travis J
    Commented Aug 2, 2016 at 22:16
  • 7
    The "sort by most popular" functionality should be removed from Documentation. What actual useful purpose does it serve for someone using docs other than to find the best topics to edit for maximum rep?
    – samgak
    Commented Aug 3, 2016 at 1:19
  • Now that SO citations of SOD examples have had some time to accumulate, it seems feasible that we could introduce an alternative sorting scheme based on how many times a documentation example is cited helpfully in an answer. It would not be panacea, but it would encourage the creation and exposure of measurably useful examples (instead of just "that looks nice" examples).
    – Thriggle
    Commented Aug 29, 2016 at 17:50

3 Answers 3

35

It's not voting system that needs to be changed, it's sorting that is inadequate.

There're all sorts of websites with this exact problem: people vote for things others have voted for. Sorting by votes is always the worst solution because several posts receive majority of upvotes and stay on top forever. Competing with these posts becomes virtually impossible. What once becomes popular, stays popular forever, not matter how relevant it is, or what awesome content was added later.

See YouTube, Reddit, Hacker News and numerous other websites — they all have views, votes and other metrics, but they never sort by them, unless you explicitly ask for it. Other factors like how recently something was added, how many views and upvotes it received recently etc. are taken into account.

The longer it goes on like this, the harder it'll be to fix. You can't just reset votes, people will be annoyed. So it's important to fix sorting as soon as possble.

And no, removing sorting by populariy is not an option. Stack Overflow needs to show new visitors the good side of Documentation, not the usual side.

P. S. The same feature was suggested for the Q&A part too, to solve the same problem (it takes a lot of time for new answers to rise above old answers), but it wasn't implemented, unfortunately.

2
  • 1
    You're implying that Reddit has a good (or suitable) algorithm for Stack Overflow, which I disagree with. Reddit has "vote fuzzing", SO does not. Reddit has rampant racism and misogyny, SO does not. Reddit is designed to foster discussion. SO is not. And so on and so on. Commented Aug 5, 2016 at 11:48
  • 11
    @uhohsomebodyneedsapupper I'm not implying blindly copying someone's algorithm makes sense, I'm just saying that they're putting much more effort into sorting than "ORDER BY Post.Score" and so should Stack Overflow's devs.
    – Athari
    Commented Aug 5, 2016 at 13:48
1

When we designed documentation, and had gotten most of the way through the private beta phase, all information that we had was pretty clearly telling us that it's really hard to get people to write examples. I know that sounds silly now, but that's what we had.

Because of that, it didn't occur to us that people would be writing so many examples in a single topic, and, thus, it never occurred to us that we might want to place soft and hard limits on how many examples could be posted. Soft limits would be more gentle prods, e.g.:

This topic currently has 15 examples, are you sure that you want to add another? Did you know that you can create new topics, and move examples, when something requires a greater breadth of coverage?

And then hard limits, which basically say "this many examples in a single topic is never a good idea. Some things within this topic really need to be in their own topics."

Had that been in place, many of these really big topics would have been spread out over a bit more than several more, and I think the effect would not have been as amplified.

To be clear, we are looking at ways to tweak balance even more, but I want to make sure that we have a pretty firm grasp of how much balance is needed, which also entails eliminating design oversights like the one described above. How stuff is presented has a pretty big influence over how folks vote, we're talking about it now.

8
  • 20
    I think this discussion is missing the forests for the trees. The problem of topics with too many examples is due to a bad topic. And that tends to be due to people not really understanding what a "topic" is intended to be. Does everything about, for example, "strings" or "file IO" need to be its own topic? And yet, people constantly create these broad topics. There needs to be a better understanding of what should and should not be a topic. Commented Aug 3, 2016 at 4:37
  • 19
    This does not answer my question actually: Placing soft or hard limits on how many examples someone could post does not change the way how we vote on them!
    – elegent
    Commented Aug 3, 2016 at 12:14
  • @elegent I think that limiting the number of example in a single topic will drive users to delete low quality examples - This will increase the average quality of examples in each topic, making more of them upvote worthy too
    – Alon Eitan
    Commented Aug 3, 2016 at 12:15
  • @AlonEitan: That’s a good point :) On the other hand in my opinion up-voting content you consider useful and down-voting content you don't consider useful (and displaying it "stack overflow"-style like) is pointless in some way for doc examples because an example does not answer a specific problem, but rather provides information for something which could be helpful for you, but uninteresting for me … Hope you get what I mean ;)
    – elegent
    Commented Aug 3, 2016 at 12:28
  • @elegent I totally got you, the votings on the Documentation are, as I see it, only reflect the amount of reputation awarded to the author and the editors, which is important in itself, BUT - I too don't think of it as a factor to evaluate the real quality of the example.
    – Alon Eitan
    Commented Aug 3, 2016 at 12:45
  • 5
    @Alon People will still be reluctant to delete examples, though, since they may include some valuable contributions, and it's rude/plagiarism to consolidate examples by merging content and deleting one of them (depriving its authors of rep): meta.stackoverflow.com/q/329002 Because of this, example limits might just lead to duplication of topics, like "More ways to do x", "Even more ways to do x".
    – Frank
    Commented Aug 3, 2016 at 14:19
  • @Frank Yup, haven't thought it through... I can't understand how the SE team are handling with the amount of RFC, suggestion, reports about plagiarism (And complains)
    – Alon Eitan
    Commented Aug 3, 2016 at 14:54
  • 1
    Consider that an ideal set of examples covers a variety of use cases. Often a particular feature can be leveraged in very different ways to accomplish different goals. You don't want 20 ways of doing the same thing that have minor stylistic differences, and then 1 really unique example that is buried among them. I'd think about how you could nest/collapse examples that have been flagged as essentially very similar. So that at the top level you have a set of fairly distinct examples where each accomplishes something different.
    – AaronLS
    Commented Aug 4, 2016 at 18:12
1

There is Something off currently about voting and topics. Many topics the voting system is completely irrelevant (for example the C# keyword topic). Now we could agree that this is a bad topic, too broad sure. But fact remains that this pattern is pervasive in SO-Documentation.

Perhaps a form of sub-topic hiearchy could be done where a broad topic could be broken down in multiple but related sub-topics (each keywords in the list get it's own sub-topic, each with their own examples).

This is true for list like topics but also for many other topics for which there are many corect ways to do something, the choice of which depends mostly on how the technology is used. Take JAXB for example. It can be used to generate code from XSD but can also be used to generate XSD from code. Each of these can be intergrated in the project through calling the standard tooling or wrapped in an ant task or maven plugin's configuration.

So for a topic "Code Generation from XSD" one could see valid examples on the many possible ways this can be accomplished, but better still, one can see sub-topics (with Maven plugin xyz, with Ant task abc, etc) each with their own pertinent examples. At which point the voting system is again meaningfull, that is it brings to the top the most voted examples as being, arguably, the best example.

Point is that it should be possible to reduce the scope of a topic to a point where examples are in competition to one-another and not complimentary to each-other. The basic rule here is one (most atomic) topic should be expressible with one example.

Tooling should reflect this situation allowing editors to also create these sub-topics and move examples to them. The hierachy here is important in that it retains the context for the new topics. That said, it should perhaps be limited in depth (soft and/or hard).

Topics are probably as, if not more, important than the examples themselves.

Not the answer you're looking for? Browse other questions tagged .