-8

It appears to me that answers listed higher in the list of answers get upvoted disproportionately more than the quality of the answer would necessarily justify.

My gut feeling is that this happens simply because answers get read top to bottom and not everyone reads all the way to the bottom.

The result of this would be that every vote could be theoretically broken down into two components - a portion that actually rates the value of the answer and a portion that reflects its position in the list

This latter portion that reflects its position in the list seems to me to be (a) significant in amount and (b) completely irrelevant to helping the answers reach an ordering, final score that accurately reflects their value to readers

In other words, answering a question early seems to heavily bias the score of the answer away from it's otherwise natural score which would be based entirely on quality. I can understand the benefit of thereby encouraging fast answers but this also detracts from the quality of those earliest answers.

What could/should be done about this?


I have since thought of a possible solution, which would be to use a mathematical model to estimate the portion of the vote that has been awarded solely by virtue of its position in the list, and only attribute the portion of the vote that remains. That way the answers could still be listed in order of highest vote first, but voting on an answer higher in the list would count less than voting on an answer lower in the list. Not quite sure how this could be simply implemented yet though. Seems in might require counting some votes and not others, or scaling the voting all up e.g. by a factor of 100.

14
  • 2
    You've got it a little backwards meaning they are randomized until the scores change. So everything is pretty much even until they are voted on so the "best" move up which is how it should be.
    – codeMagic
    Commented Oct 31, 2016 at 19:08
  • 1
    @codeMagic No, he's just not talking about situations where answers have equal votes.
    – Servy
    Commented Oct 31, 2016 at 19:11
  • 1
    @Servy maybe I took it wrong but that's how I read it. Sounded like he didn't know they were in a random order. If I am wrong, then I'm not sure the point of the question. You certainly don't want them random with highly upvoted answers showing at the bottom and we can't do anything about people not wanting to scroll and read a little.
    – codeMagic
    Commented Oct 31, 2016 at 19:15
  • I will edit my answer to suggest a solution Commented Oct 31, 2016 at 19:17
  • 1
    @codeMagic It's mostly just another FGITW complaint.
    – Servy
    Commented Oct 31, 2016 at 19:21
  • 1
    So how will this formula be able to tell if, for example, all the upvotes come from the answer simply being good? Or how many are just because it's first or second? What about upvotes because the voter likes the user's avatar? Or upvotes because the voter accidentally hit the button? If this mathematical model can tell a voter's intent and reasoning, I'll be much impressed.
    – Kendra
    Commented Oct 31, 2016 at 19:22
  • 2
    @Kendra. Not designed to determine intent. It's intended to be an approximation, probably based on something like a power law Commented Oct 31, 2016 at 19:25
  • 1
    @BradThomas My point is that your model can be, and likely will be, completely inaccurate in a lot of cases. What will then come of it will be that terrible, low quality answers start to rise in "power" just because people haven't upvoted them as much while they've instead voted on a good, high quality answer above it. You can't be sure people are voting solely because an answer is first.
    – Kendra
    Commented Oct 31, 2016 at 19:28
  • 1
    @kendra I believe it could be made more accurate than now by design, and I don't think that would be hard since current system is way off, in my view Commented Oct 31, 2016 at 19:30
  • @Servy no, I disagree. I'm saying the votes at present doesn't reflect the relative value of the answers accurately. It's even questionable whether the voting gets more accurate over time or not. I'm happy if the system encourages fast answers. It's not really a complaint about speed of answers. It's a complaint that people are essentially rewarded for speed, when that's not what counts to readers Commented Oct 31, 2016 at 19:33
  • @BradThomas I do think that you can do better, but it's certainly not easy. Changes like this are very hard to do well. If you think it'd be easy, you probably haven't spent enough time researching the problem domain.
    – Servy
    Commented Oct 31, 2016 at 19:33
  • I can't see the math itself being too hard to estimate from existing Q&A. But maybe that's because I have that kind of modeling background. I can see changing the UX and managing user expectations to be tricky. Commented Oct 31, 2016 at 19:35
  • That is how it goes for me : read the question, read the first answer. If i think the first answer is very good, i upvote it without looking at the others. If i don't think the first answer is THAT good, i read all the others and upvote the one i juged the most appropriate. So the position in the list have an effect, but the answer on top still has to be a VERY good one. This said, i don't know who could complain... Commented Oct 31, 2016 at 19:46
  • 1
    sorry for answering and bringing more attention to your question, getting it new downvotes, and closed, even. wasn't my intention obv. that'll teach me to stay away from meta for another 10 years. (wasn't good at staying away the first 10 years either). :( cheers.
    – Will Ness
    Commented Nov 29, 2020 at 22:14

2 Answers 2

16

Higher voted answers are more likely to be read than lower voted answers. This is by design. The whole purpose of the voting system is so that the best answers are the ones that users first see when viewing the question, preventing them from needing to sift through lots of bad answers to find the best one.

A consequence of showing an answer to more people is that it will be voted on more, and if it's a good answer, those votes will tend to be upvotes, not downvotes. So yes, a good answer posted early will tend to snowball. It tends to be hard (but not impossible) for this to happen with a bad answer, it's also hard, but again, not impossible, for better later answers to gather the votes to surpass earlier answers of moderate quality. But yes, because of this system there will be times that moderate quality answers posted early will staying ahead of better answers posted later.

This effect has been seen pretty much since SO came about, but there aren't many particularly good ways of combating this effect while still actually showing the answers in descending order of quality (as best as the SO engine can estimate quality). If you have a proposal for a particular mechanism for ensuring that posts are sorted in descending order of quality that is better than just using votes, by all means propose it. It's not a particularly easy problem to solve though.

29
  • 1
    Good work, easy to point a problem, not so easy to solve it ! Commented Oct 31, 2016 at 19:54
  • they are not "best". they are most voted up. the two qualities are almost completely independent of one another. fastest gun in the west still wins the SO day, and if that's by design, that's just too bad -- for those of us who care for the content.
    – Will Ness
    Commented Nov 29, 2020 at 16:38
  • @WillNess Fastest Gun in the West only really applies for low quality questions that can easily be answered by lots of people very quickly, usually for questions that are already duplicates and easily answered with the most trivial of research anyway. It's not important for their answers to be ordered particularly carefully. For quality questions, it's not that votes are independent of quality, rather it's an imperfect approximation of quality. It's certainly better than a random order, but it is indeed not perfect. If you have an idea for a better way of determining quality, propose it.
    – Servy
    Commented Nov 29, 2020 at 21:05
  • @Servy I just did, below. got minus 7 votes so far. and counting. :) as for low quality questions, that's the absolute majority on SO nowadays, and no, it's not the correct criterion, the correct one is "easy to answer". this includes well asked questions as well. and before you say it's not a problem then, yes it is, as there are often more than one good answer to a question, and late ones get almost no eyeballs, and that's unfair, and makes people want not to contribute anymore.
    – Will Ness
    Commented Nov 29, 2020 at 22:11
  • @WillNess See, the problem with your answer is that you assume SO is a place for you to have fun playing a game getting points, and not a place to create a high quality repository of knowledge where people want to go to get an answer to their question. Obscuring the quality answers as you propose is very against the whole point of the site. You feeling that things are fair is not the end goal of the site. If this was a game where entertainment of answerers was the goal, then your proposal would at least be worth considering, but it's not.
    – Servy
    Commented Nov 30, 2020 at 1:12
  • @WillNess Oh, and for the record, your assertion that SO should optimize itself for low quality, poorly research, trivially answered questions, just because lots of people post them, is just false. Again, the site exists to be a repository of knowledge creating useful lasting resources that will help everyone with that problem in the future, not to optimize for people asking a FAQ for the 1000th time that is never read by anyone an hour after the question is posted.
    – Servy
    Commented Nov 30, 2020 at 1:16
  • "your assertion that SO should optimize itself for low quality" I have never said such thing nor suggested it in the slightest and thus your accusation is low, immoral, unseemly false smear. as is the preceding comment of yours about the playground etc. the problem with SO is precisely that it is that playground and does nothing to support it being real knowledge creation and curation platform. which really is what I've said my concern is, and what it indeed really is. blocked.
    – Will Ness
    Commented Nov 30, 2020 at 5:03
  • @WillNess But you did. I said we sho9uldn't optimize for low quality questions, and your response was, "but that's the majority of the questions on the site". You were opposed to my assertion that we shouldn't optimize for low quality questions. Personally insulting me by calling me low, immoral, accusing me of smearing you is itself highly inappropriate, given that you did in fact do as I said. That you think SO isn't actually a good knowledge creation or curation site, and that it should not even attempt to be, is an opinion you can hold, but don't expect people to support it.
    – Servy
    Commented Nov 30, 2020 at 13:54
  • wow. that's some warped thinking. turning my words "the problem with SO is that it ... does nothing to support it being real knowledge creation and curation platform" inside out into "you think SO ... should not even attempt to be [that]". such perversion of truth in the space of few lines, wow. just wow. another is presenting my words "your accusation is low, immoral, unseemly false smear" as if said about the person.
    – Will Ness
    Commented Nov 30, 2020 at 16:36
  • @WillNess Considering you're literally proposing removing one of the most important content curation features the site and claiming that no attempt should be made to present quality content over low quality content no, that's not a stretch or an assumption. Making blatantly false statements and inflammatory accusations about my actions that are trivially proven false is a personal attack. If you argued the merits, for example explaining how showing answers randomly actually helps people looking for answers, that wouldn't just be a personal attack.
    – Servy
    Commented Nov 30, 2020 at 16:47
  • again with totally false misrepresentations. never said what you falsely accuse me of. your perception is really warped, looks like it. cheerio.
    – Will Ness
    Commented Nov 30, 2020 at 17:13
  • @WillNess Yeah, it's almost like I think you posted an answer saying that answers should be ordered randomly. You would never say anything like that, and asserting that you would is such a wild misrepresentation of your idea to, if I quote your answer directly, "always show the answers in random order". Clearly your have not voiced that opinion at all. I apologize for accusing you of suggesting answers should be ordered randomly, that was out of line. I should have instead realize that what you were suggested simply to order answers randomly instead. Novice mistake.
    – Servy
    Commented Nov 30, 2020 at 17:17
  • the root of your misunderstanding is that apparently you think that votes count is an indication of quality. which sometimes it is, but for late posted answers it is most often not. my suggestion aims to combat false voting of low quality content.
    – Will Ness
    Commented Nov 30, 2020 at 17:20
  • @WillNess You'll need a basis for your assertion that votes are not an indication of quality. You're asserting that votes are worse than deciding randomly when it comes to determining quality, meaning you think votes correlate negatively with quality, given that you think pure randomness is more likely to choose the best answer. (Interestingly if your assertion were true then showing posts in reverse score order would be best) That's an assertion that requires enormous evidence to support. Lots more than just a few questions where you think the best answer isn't the highest voted.
    – Servy
    Commented Nov 30, 2020 at 17:24
  • I've updated the answer. turns out I've omitted one crucial detail that changes everything (I think). sorry for the confusion. :)
    – Will Ness
    Commented Dec 2, 2020 at 14:20
-9

The obvious simple solution is to always show the answers in random order on page load (with slight preference to lower voted answers, perhaps, to give them more chance to be seen).

(edit: what I meant here, and failed to specify explicitly in the first version of the answer, is that this is to be done only during the first short period of time, like first few hours to few days after the question was posted. then the normal mechanisms would be in play. my motivation here is searching for the ways to contravene the Fastest Gun in the West effect skewing the vote counts, so that the counts would reflect the quality of the answers more accurately. Reading back I realize that was not at all clear.)

The user can always click on "by votes/newest/oldest" if she so chooses (and BTW where is that "newest" option???)

A slightly more radical option is to delay displaying (and taking into account for ordering purposes) the vote counts on answers by some not inconsiderable amount of time, so as to not unduly influence the readers. The answerer would still see the upvotes by their answer's side / on their profile's reputation tab, immediately, of course.

2
  • 1
    my motivations are explained e.g. here. :)
    – Will Ness
    Commented Dec 2, 2020 at 15:17
  • 1
    A user can upvote every single answer to a question. If that isn’t happening then that should be taken as the user didn’t find value in one of the answers. How does changing the order of the answer change that fact? Commented Dec 2, 2020 at 15:21

Not the answer you're looking for? Browse other questions tagged .