-65

I strongly dislike suggestions about changing our system or "way of doing things" for the benefit of a sub-group of users who will abuse the site regardless of such innovations. So recent discussion around the topic hasn't been enthralling.

But, in reading a recent suggestion about imposing a 24 hour "vote freeze" on meta-linked questions, I saw what I think is an excellent idea from Paul Draper that deserved a bit more attention: voting with stars, rather than arrows.

Rather than displaying an up arrow (heaven/the sky/life) and a down arrow (hell/caves/death) and positive numbers (awards/money/power) and negative numbers (punishment/poverty/weakness), what if we just displayed a 0-5 star rating that averaged the votes?

This would take care of several possible problems with "one stone":

Absolute vote counts aren't a good indicator of quality.

  • As I've said before, an answer with a score of "10" doesn't tell me much about the quality of that answer. If the vote is 10/0, it's probably good. If the vote is 100/-90, then the quality is probably suspect. Why should an old, out of date answer with a score of 100/-90 be ranked above a newer, better answer with a score of 8/0?

    A star rating that averaged all the votes would provide a much clearer picture of just how "good" the question, or especially answer, actually is without the expense of hitting the server to get the complete voting history.

Binary voting is tricky with "so-so" questions and answers.

  • Sometimes an answer is pretty good, but not great. It might be correct overall, but have some bad practices within it. I'd like to recommend it to others, but with some caution. A downvote doesn't acknowledge the good content but an upvote doesn't admit the poorer-quality content. Edits in such cases are discouraged as "best practice" is subjective. Comments are temporary things that should not be relied on. A scaled vote would provide more nuance and better represent how I and other professionals gauge the content.

People don't get as upset over stars.

  • This is the least important aspect in my opinion, but may speak to the current discussion about freezing posts and transforming Meta into a mentoring station.

    The noisiest and most annoying users are those that complain about a single downvote on a question with 10 upvotes. If that downvote was a two star rating (yay! +2!) instead of a downvote (boo! -1!) then maybe we could achieve the same ranking effect without all of the moaning, groaning and angst.

Posts with no votes would show with N/A (or something similar). The ranking could be something like:

enter image description here - 0 stars.
This question/answer has some serious issues that can't be easily resolved. It may need to be closed/deleted.

OP Rep -= 2 (Answer/Question)

enter image description here - 1 star.
This question/answer has some glaring issues that might be resolved with heavy editing.

OP Rep -= 1 (Answer/Question)

enter image description here - 2 stars.
This question/answer has some big issues but also some valuable content. It should be edited.

OP Rep += 0 (Answer/Question)

enter image description here - 3 stars.
This question/answer has valuable content but also some minor issues. Editing is optional.

OP Rep += 5 (Answer) 3 (Question)

enter image description here - 4 stars.
This question/answer has very valuable content but may also have some minor issues. Editing is probably not necessary.

OP Rep += 10 (Answer) 5 (Question)

enter image description here - 5 stars.
This question/answer is excellent in every regard and can be referred to/trusted without reservation.

OP Rep += 10 (Answer) 5 (Question)

Voters would also get a -1 reputation hit for rating a question/answer as 0 or 1. Maybe just 0. Not sure about that.

Pros

Such a system could easily address some of the biggest (and some smaller) concerns without making the act of voting any more difficult. Those who still prefer a binary system could just vote with 1/5 or 0/5 and go on their merry way. Those who want to provide a more measured or nuanced response would have the ability to do so rather than just refrain from voting, which is what I most often do. And the averaged score would provide much more valuable information then an absolute count.

Cons

It's a change to our voting system, which is more or less enshrined in SE culture. Such fundamental changes should not be taken lightly.

Such a core change would not be trivial to implement from a development standpoint.

There's also the question of what to do with all of the existing votes.

9
  • 31
    It's hard enough to get people to vote correctly already, with only the up/down arrows. Getting people to vote correctly with a rating scale that has six choices would be... an interesting experience.
    – user
    Commented Aug 21, 2014 at 20:51
  • 1
    @MichaelKjörling - Imagining for a moment that there's such a thing as "correct voting", I don't see that implementing a sliding scale would have much of any effect. Such voters would simply vote 0/5 as they currently do. No change, in that regard.
    – JDB
    Commented Aug 21, 2014 at 20:53
  • 8
    For the last few years we have, almost universally, gotten rid of gradients in user rating because it is ineffective. People tend to vote for either extreme. Not to mention the usual "y u gief 1 star?!" complaints that are even more prevalent than our downvote complaints.
    – user98085
    Commented Aug 21, 2014 at 20:57
  • @FEichinger - As a whole, that's true. But I like to think that users on SE tend to be much more engaged with the content then users of sites like YouTube. In fact, you must be a registered user just to cast your first vote. As for griefers - they exist already. The hope would be to reduce their numbers.
    – JDB
    Commented Aug 21, 2014 at 21:03
  • 6
    Under the proposed system if a post got one 0-star rating and a second got post got 10 0-star ratings, what users looking at either post would see is the exact same rating: 0 stars. Arguably, each 0-star rating under the proposed system translates to a downvote under the current system, so one post would have -1 and the other -10 under the current system. Is there something I missed or is it in fact the case that the proposed system would just obscure how the community rated a post?
    – Louis
    Commented Aug 21, 2014 at 21:07
  • 1
    @Louis - Easily remedied with "1 User Voting" or "10 Users Voting" underneath the rating.
    – JDB
    Commented Aug 21, 2014 at 21:08
  • 4
    @Louis evanmiller.org/how-not-to-sort-by-average-rating.html . You can even display 0-5 star ratings and allow only +1 or -1 votes, which is what YouTube actually does as far as I remember, and correspond to the extreme votes that smart people are only likely to cast anyway. Commented Aug 21, 2014 at 22:11
  • 1
    Relevant: xkcd.com/937 xkcd.com/1098
    – jkd
    Commented May 20, 2015 at 23:57
  • I like the idea of displaying votes as a gradient scale somehow, but voting should definitely be binary.
    – Alex
    Commented Feb 4, 2018 at 12:03

1 Answer 1

42

People don't get as upset over stars.

Sure they do. CodeProject has used a 5-star rating on articles for years, and that never stopped anyone from complaining when they got a low-star rating. Heck, they finally implemented a system wherein votes of 3 or fewer stars required adding a comment:

Votes of 3 or less require a comment (this works slightly better for long-form articles than it does for posts)

So yeah.

The bigger problem though is that it doesn't map 1-1 to up/down votes; you have 3 other options, and what those mean varies wildly. Some folks never give out 5-star votes save for the most brilliant works, others stick exclusively to 1- and 5-star votes, still others default to 3 for anything they can't really be bothered to form an opinion on - thus diluting everyone else's opinions.

There are countless examples of star ratings falling apart over the years. One of the most ridiculous is probably eBay - if you've ever used eBay, you know how well that works. "4.5 stars? RUNAWAAAAAY!"

8
  • See for instance: youtube-global.blogspot.co.uk/2009/09/…
    – nico
    Commented Aug 21, 2014 at 21:01
  • Yeah, but SE users with voting privileges are far more engaged in rating the quality of the content on the site then EBay or YouTube users. I've never seen a Meta EBay or Meta YouTube where such issues are debated endlessly. I don't think that's a fair comparison. At the very least, it's a better idea then freezing voting on questions linked from Meta.
    – JDB
    Commented Aug 21, 2014 at 21:06
  • 12
    People are people, @JDB - if you're arguing that the engagement here would reduce complaints, I think you're misunderstanding why folks complain - it's not that they care too little. And OMG, just google "eBay star rating"; the folks there depend on their rating for their livelihoods in many cases, and the drama is unimaginable.
    – Shog9 Mod
    Commented Aug 21, 2014 at 21:10
  • Again, Shog, you highlight the difference between eBay and SE. No one here is making a living off of the star rating. Perhaps, also, you missed the "as"... the hope would be a 2-star rating would not cause as much angst as a downvote. In the end, though, I don't care if people get upset and I made that pretty clear. Even if people don't use the stars and still go only for 0/5 voting, the end result would, in my opinion, be a more accurate ranking. Please tell me how popular this post was without looking at the vote breakdown.
    – JDB
    Commented Aug 21, 2014 at 21:26
  • 1
    Not particularly popular; under 400 views, under 40 votes, on Meta. BTW, have you seen CP's vote breakdown? Folks get very invested in this stuff...
    – Shog9 Mod
    Commented Aug 21, 2014 at 21:29
  • So true. Even from my occasional browsing of articles on CodeProject, I see tons of ranting in the comments section about star ratings. Y U NO GIVE ME 5 STARZ? This would not be an improvement, it would just make voting more difficult.
    – Cody Gray Mod
    Commented Aug 22, 2014 at 5:26
  • 1
    Sorry but it had to be linked xkcd.com/325
    – kleineg
    Commented Aug 22, 2014 at 19:49
  • 2
    @kleineg If we're linking to XKCD here, then given the context 937 and 1098 are a must
    – Arc676
    Commented Jan 3, 2016 at 9:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .