-25

I've been thinking about the reputation system, and I believe it is a bit flawed. That's probably not a surprise for anyone that a given system is not perfect, and even though I don't have a proper solution, I believe I have a kind of direction to follow.

What I feel is flawed

The reasoning behind this is the quality of questions and answers of the overall Stack Overflow... database (not sure that's the word I'm looking for, feel free to edit). I often see great answers with a few votes, and bad answers marked as the accepted answer. That's most certainly a problem many of you have witnessed countless times. And to be honest I don't even think there should be an "Accepted" option, but that's another issue for another time (or is it?).

The good thing is the reputation system allows you to judge the quality of an answer if you are completely unable to do it any other way. I remember the days I was really bad at programming, and out of two working answers, I would blindly take the one from the highest reputation poster. But that's not the case for everyone, and even worse, you never know who votes. That's the core of the problem.

A bad answer (only talking about answers here, questions can be bad but should be edited or deleted altogether, in a way that there should never be any bad questions in the end) can be good for a novice user; he sees code, copy pastes, and it ends up working. It's working hence it's good. And then some other novice with the same level of knowledge votes it up, which makes it even more reliable. But it's still a bad answer, the expert knows it and votes it down. We could even argue there are fewer experts than novices on Stack Overflow, and we're left with a potentially bad answer with higher votes than the good answer. Again, I'm sure we've all seen "this should be the accepted answer" comments.

This is where the problem lies, I believe, is in the quality of the vote. We're not all equal on Stack Overflow, and a higher reputation should give a stronger balance in the score of an answer.

The idea

(which you most probably have guessed by now)

Simply put, a complete revamp of the scoring system could maybe fix this. Someone with a reputation of 1 has very little strength in the score of a question, while Jon Skeet pretty much crushes everyone with his opinion.

Question scores wouldn't be like it is now, +1s & -1s calculated together; it'd still be the same idea, "agree" vs "disagree" or "good" vs "bad", but the score shown would be very different.

What I have in mind is bad on reputation per tag, and apply that value to the answer score. There is need for improvement, as someone with 20k in a domain is definitely very good and most probably above average. There is less reason for that person to be less valuable than someone with 50k, even though there is a lot of reasons to value someone that has 100 vs 20K. So it's not only about the difference, but maybe about some average levels or caps you can hit. Maybe base it on the average of the Stack Overflow user base, or maybe base it on arbitrary numbers, like 1k - 2k users have the same strength, but once you reach 3k you've more influence? I don't know.

All I feel is, a bad answer upvoted by 100 novices should be able to be downvoted by a single experienced developer in the domain. But five experienced developers could still have the chance to argue vs one very experienced developer. Again, arbitrary numbers for the sake of the example.

There are some problems that rise using a system like this one

  • Are previous votes influenced by my reputation gain?

I don't think they should; when you voted you had that experience, maybe now you'd even disagree with yourself (we all do!)

  • [Adding problems here as people participate]

Conclusion

I'm sure you've all got the idea; I'm doing my best to explain it, and I agree, the solution is clearly not set in stone in this post. But I strongly feel that the current system is a bit flawed and that something oriented around the above idea fixes those flaws.

28
  • 11
    EveryBody's Vote is equal. No Discrimination. I like this about SO.
    – Sandeep
    Jun 15, 2016 at 11:37
  • Nobody is equal and there is no discrimination in that. Everyone lives it every single day. Some people drive better than you, some don't. Some people code better than you, some don't. And a knowledge based community should be all in favor of people mastering that knowledge. Favouring those users can't be bad for the community. While it could be bad for your own anwser and personal reputation, you'd still be able to provide feedback and constructive criticism like anyone else
    – Gil Sand
    Jun 15, 2016 at 11:40
  • 7
    rep has nothing to do with how good you are in something, heck, it can even be a question giving you rep.
    – Mafii
    Jun 15, 2016 at 11:43
  • 5
    How will you judge who is better coder and who is not? Experienced users are favoured in SO by giving different level of Moderation rights.
    – Sandeep
    Jun 15, 2016 at 11:44
  • @Mafii I'm not sure I understand the second part of your statement. But rep sure is an indicator of how implicated and rewarded you are in something. Spending time and energy on something while being rewarded for it is usually a display of quality. It could be false, but most of the time, someone spending a lot of time doing something gets good. And if people reward him (with reputation here), then it's even more reliable
    – Gil Sand
    Jun 15, 2016 at 11:46
  • @Sandeep the moderation rights are irrelevant here. Yes it's a reward for most implicated users, but they're not the subject here.
    – Gil Sand
    Jun 15, 2016 at 11:47
  • Okay, lets go math on this one. The chance is probably higher, that when everyone is voting equaly, that there will be less mistakes made. Whats unclear? You can be a douchebag and still ask many question. Look at these javascript questions that gave a user 1000 rep or so, and it was a simple question about operators or something. Its really saying nothing
    – Mafii
    Jun 15, 2016 at 11:48
  • 1
    rep doesnt show how knowledgable someone is, it shows how much time/effort they put into SO. (often atleast.)
    – Mafii
    Jun 15, 2016 at 11:49
  • There can be more than reputation. Badges could be taken into account. Overall quality of answers. Sure "that answer has a lot of votes", thanks to good timing. But you don't have to take rep alone. It could be the average score/answer (for a given tag). That'd be more accurate. I'm not saying my system is perfect and good to go. I'm saying the current one is flawed.
    – Gil Sand
    Jun 15, 2016 at 11:54
  • 2
    Badges can be "farmed". How would you measure "Overall quality of answers"? What if questions have 2 tags that are equally relevant, but the user only has a crapload of rep in one of the tags? I'm not saying the current system is perfect, but it's certainly not as flawed as you make it out to be.
    – Cerbrus
    Jun 15, 2016 at 11:56
  • 1
    The irony is that, based on your suggestion, I should be able to single-handedly downvote this suggestion into oblivion.
    – Cerbrus
    Jun 15, 2016 at 11:58
  • @Cerbrus Just imagining what John Skeet would be able to do.
    – Sandeep
    Jun 15, 2016 at 11:59
  • Indeed you would, but others woul be able to either support you or not, and that's what nice about it. Maybe my idea is bad. I'm here to defend it until proven wrong (though this debate is less technical than the actual SO content which is more often black or white rather than grey). If someone as valued as you are tells me I'm wrong and why, I'm more enclined to believe it coming from someone who just got here without any real baggage. Of course I would LOVE something more accurate, but I can't think of anything else
    – Gil Sand
    Jun 15, 2016 at 12:00
  • 5
    "And Jon Skeet would probably be right anyway." We're all only human.
    – Cerbrus
    Jun 15, 2016 at 12:04
  • 1
    Oh my god.... Let's NOT do that please.... Reminds me of tenurr in academia.... And it's about as broken as tenure is... "You got good stats on this code, so your opinion now is better than everyone else's".... Just.... No
    – Patrice
    Jun 15, 2016 at 20:31

2 Answers 2

24

Heck no.

My vote is my own. It isn't more or less valuable just because someone else has less or more imaginary points.

Someone that knows nothing about a certain tag could undo votes cast by users that have gotten all their rep in said tag.

Rep is a poor indication of knowledge. The right answer at the right time can earn users thousands of reputation, even if the answer is just a lucky guess.
Same for a decently written question.

12
  • Maybe a combination of rep per tag, and badges? Indeed, your vote is your own, but in general, if multiple users with more experience than you are saying something, they're a higher chance that they're right
    – Gil Sand
    Jun 15, 2016 at 11:38
  • 6
    That's simply not true. I've had users with 10 times as much rep as I have tell me my answer was wrong, only for them to realize it wasn't, after explaining. Experience can't be measured in rep. If you post enough mediocre answers, you'll eventually get a load of rep.
    – Cerbrus
    Jun 15, 2016 at 11:40
  • I agree with you, and discussing / arguing over content can still be done. Votes can still be removed, and experts can't always be proven wrong. And it's their job to remove/add their vote at that point to the correct answer, wether it's your answer or theirs.
    – Gil Sand
    Jun 15, 2016 at 11:42
  • 2
    We shouldn't have to argue about the content. A vote should be enough. I'm not interested in engaging in a discussion with everyone that disagrees with me, that has less rep.
    – Cerbrus
    Jun 15, 2016 at 11:48
  • You don't have to ; others can defend your answer. Not wanting to discuss content makes you less implicated, but doesn't make you more right or wrong. Though I believe discussing answers (non trivial answers I mean) is what makes good content. You get the answer AND different opinions about it.
    – Gil Sand
    Jun 15, 2016 at 11:50
  • 1
    @Zil a higher rep user shouldnt be valued higher. This wont work.
    – Mafii
    Jun 15, 2016 at 11:52
  • They already are.
    – Gil Sand
    Jun 15, 2016 at 11:55
  • Only in the available tools to moderate the site's content.
    – Cerbrus
    Jun 15, 2016 at 11:58
  • And I would assume that people look at higher rep users more highly than lower rep user. But that's not measurable, and that's certainly not implementend in any way. But that's the principle of reputation in the first place, being seen differently. Otherwise it wouldnt' be visible at all, you'd just have more features without others knowing it. Displaying it like that is to make you look more reliable than others. Too bad it doesn't show only your total reputation for the given tags. Here I see you at 33k. Maybe you know nothing about "reputation" or "discussions", maybe you do.
    – Gil Sand
    Jun 15, 2016 at 12:08
  • 1
    People should judge an answer based on the answer's contents. Not based on who posted it.
    – Cerbrus
    Jun 15, 2016 at 12:28
  • @Cerbrus to be fair, there is a good amount of people who do upvote high rep users everytime they see an answer, because it "must be good". Not saying this is good tho. Ofc, high rep gives you high creditibility.
    – Mafii
    Jun 15, 2016 at 12:36
  • On the other hand, I'm more inclined to downvote an slightly incorrect answer if the user has a load of rep (experience). I'm less tolerant of mistakes then.
    – Cerbrus
    Jun 15, 2016 at 12:37
13

I do not support this feature request.

If I am running into an issue and I need to go to SO for a solution, I'm going to look at the answers that appear to solve an issue similar to my own and evaluate which seems most promising. Then I will try using the answer in code and test it. If it works, then I will upvote the answer. If I do try multiple answers and they all work fine, and are equivalent, they'll all get votes. However, I won't additionally upvote everything that looks like it might work. Basically, I don't upvote any answer unless I can reasonably vouch for its correctness, and you've seen where I've set the bar: I have to have used the proposed solution to vouch for it. (Sometimes I can reasonably vouch for correctness just by eyeballing an answer but this entails already knowing the solution and having used it, which is not the case if I'm looking for a solution in the first place.)

This has been among my principles for day one. The votes I cast at rep 100 were not generally less accurate than those I cast at rep 10k or those that I cast now.

11
  • I don't agree with your last statement. When you gave those votes you were 100% agreeing with them. But maybe if you look at them now you'd realize the voted content wasn't the one that should've. Or could've been better. When it comes to programming you could say that putting everything in main works, so is upvotable. But the guy showing you to use classes is clearly showing you the correct way. Yet his "more complicated answer" that people might not even have bothered trying will have less impact, and decrease the overall learning of the original person asking.
    – Gil Sand
    Jun 15, 2016 at 12:12
  • 3
    I already had years of experience in the business when I was at rep 100, but by all mean do assume whatever story will confirm your opinion.
    – Louis
    Jun 15, 2016 at 12:15
  • Oh come on man why do you have to talk like that? I'm just defending my opinion with the information I'm given. There is no need to be rude. The fact that you were experienced made your answers of higher quality and therefore more likely to be voted up. My "story" is still valid translated into your context, where one day, you had less experience and were looking up to others, and valued their opinion higher than your less-experienced-at-the-time opinion. But I've got the hint, if you're not ready to discuss I, don't want to either.
    – Gil Sand
    Jun 15, 2016 at 12:23
  • 16
    @Zil: A 46.9k rep user and a 33.2k rep user disagreed with your suggestion now. Yet you remain stubborn, basically saying we're wrong. That alone proves this suggestion doesn't work, since you're not trusting our judgement based on rep alone. The whole premise of your suggestion is to trust experience, while you refuse to do so.
    – Cerbrus
    Jun 15, 2016 at 12:27
  • You're right. I'll stop arguing, but I still feel that the current situation could be better. I trust you when you say my suggestion won't work. But I don't see anyone trying to improve it. Maybe I'm being stubborn, but when I read everyone's answer I'm not sure people took the time to think about it. They straight up said "it's bad". All I've seen is (experienced) people disagree with bits of an idea, but those can be corrected and eventually, maybe, shape better idea. Instead of saying "it's bad", why not "try this" ? Being told to stop discussing in the discussion tag is ironic though.
    – Gil Sand
    Jun 15, 2016 at 12:42
  • And for what its' worth, thanks for at least taking the time to participate. You probably didn't here, but I do enjoy discussing ideas.
    – Gil Sand
    Jun 15, 2016 at 12:43
  • 4
    It's not up to us to fix issues with your suggestion. If that were the case, someone could just say "Fix the voting system!", and it'd be a valid meta thread.
    – Cerbrus
    Jun 15, 2016 at 12:47
  • 3
    @Zil Many high-rep users have seen this suggestion many times, it's been discussed to death. The insurmountable problem you continue to ignore is that reputation doesn't matter. It does not reflect your expertise in a subject. Reputation reflects the volume of content you post, mixed with luck, mixed with time. I've gained 10,000 reputation in the JavaScript tag for one answer, am I an expert in JS because I could Google the array docs?
    – user229044 Mod
    Jun 15, 2016 at 19:29
  • 2
    Should my votes in the C++ tag count for more than Bjarne Stroustrup's because I gained 3000 rep for this insanely basic answer? There are a million different ways of trying to gauge a user's proficiency in a technology based on the data SO produces, but none of them are any good. SO's gamification mechanism is simply not suitable for this purpose.
    – user229044 Mod
    Jun 15, 2016 at 19:32
  • Sure, and it wouldn't have to be based on reputation. Wouldn't it be nice to give réal experts more power over their vote? That's what I'm suggesting ; what it is based on is a problem that comes afterwards. And indeed, rep is a bad idea.
    – Gil Sand
    Jun 15, 2016 at 19:47
  • 1
    @Zil so...your suggestion is flawed and people point out the flaws. You say "they are bits that can be changed" .. Then change them? You come with a proposal. My answer is "yes", or "no". I dont have to answer "no,but if you change this and that,yes". I dont see a way to make your system work. The fact i dont see it means i cant try to improve it. I agree voting isnt necessarily working as it should. But as cerbrus points out: as per your own post, you should give deference to the high reps. The fact you dont just proves the damage this would do...
    – Patrice
    Jun 15, 2016 at 20:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .