73

So I was wondering something about how reputation level may affect our behavior when interacting with Stack Overflow and each other: what actions does one need to conduct in order to be the most objective regardless of the reputation points level of the contributor?

Now, here is how I behave:

  1. I tend to answer questions asked by lower-reputation points users as I'm sure I'm more likely to understand their problem and able to help them out than questions asked by higher-reputation points users (I take for granted that higher-reputation points users have more knowledge than me and therefore, that I would be unhelpful to them).
  2. I tend to less downvote answers (adressed to others' questions) that I have a doubt on when they are provided by higher-reputation points users: the question "Am I sure I have correctly understood the problem?" tend to arise more on those higher-reputation points users' answers (and I cannot guarantee that I downvote less because answers are of better quality).
  3. For the same reasons as 2., I'm less likely to flag and downvote questions asked by higher-reputation points users (slightly related to 1. too).
  4. As for accepting answers (to my questions), I have to admit that the contributor's reputation points level would guide me in whether accepting his answer or not. That is to say: the highest reputation level, the more tendency I have to accept the answer over others posted by lower-reputation points users (and I think it gives more "credibility" to the answer).
  5. When I comment, I'm less likely to correct an higher-reputation points user if I think this user is wrong (related to 1.) because I'm not that confident in myself so I would double check before commenting (when I don't for a lower-reputation points user). When a higher-reputation points user corrects me, I will take for granted that I'm most likely to be wrong. I won't if the correction comes from a lower-reputation points user.

Long story short, a user's reputation points level tends to affect how I perceive his/her contributions where I know it shouldn't.

Therefore, what does one need to do in order to not be impacted by what I call the ?

32
  • 41
    As soon as I see a k under someone's name, I do tend to assume what they're doing is ok and I'm less inclined to challenge it. Commented Jul 21, 2015 at 10:10
  • 23
    @SuperBiasedMan: That doesn't mean you should just accept whatever a higher rep user says, though. Always verify the answers you get.
    – Cerbrus
    Commented Jul 21, 2015 at 10:15
  • 3
    @Cerbrus 100% agree. Especially because that might mean they have high rep for posting lots of answers on not necessarily good questions. Commented Jul 21, 2015 at 10:27
  • 21
    I suspect that everybody will be ahead when you base your voting on facts instead of appearance. If you are not sure whether a post is helpful then just don't vote. Commented Jul 21, 2015 at 12:52
  • 9
    the user's reputation has no affect on my habits. Heck, half the time i'm done voting/responding to a post before i even look to see who posted it.
    – Kevin B
    Commented Jul 21, 2015 at 15:18
  • 1
    @KevinB I guess this attitude is more likely to be adopted if you have an higher reputation level. Would you be able to guarantee you'd act the same way even if you had, say, ~500 points (instead of 63k)?
    – D4V1D
    Commented Jul 21, 2015 at 15:21
  • 27
    I was about to explain to @HansPassant and Kevin B that they are poor students of human behavior, elevating a wishful theory over real-world biases. Then I saw their reps and thought better of it. Well said, you two!
    – danh
    Commented Jul 21, 2015 at 15:23
  • 3
    Personally, reputation isn't something I normally notice when I first read a post. After I see a user's reputation I might be influenced in ways similar to what you described, but usually when I read a post I don't even notice who wrote it. (Seriously, there have been times where I didn't even realise an answer was written by Jon Skeet until reading the comments below the answer.)
    – Ajedi32
    Commented Jul 21, 2015 at 15:35
  • 13
    Doesn't calling it "reputation" sort of imply that it's supposed to affect your perception of/behavior to that user? Commented Jul 21, 2015 at 15:40
  • 1
    @Frank I agree although I might tend to accept the answer that addresses all my points. Perhaps I won't accept any, either (dunno yet).
    – D4V1D
    Commented Jul 21, 2015 at 15:49
  • 12
    "I take for granted that higher-rep users have more knowledge than me" or that they just have a lot more time available to spend on the site picking off easy answers. Doesn't apply to everyone, but there are a few that I think fall under this banner. Commented Jul 21, 2015 at 16:07
  • 2
    @Zizouz212: if the answer is good, upvote it. How hard can it be? Rep shouldn't matter when voting on an answer.
    – Cerbrus
    Commented Jul 21, 2015 at 19:28
  • 3
    Perhaps I should have said that the original version was fine and it shouldn't have been closed. I voted to re-open your original version. Primarily Opinion Based is a really stupid reason to close a Meta discussion.
    – apaul
    Commented Jul 21, 2015 at 22:38
  • 1
    Write a user script or custom browser style that hides users' reputation. Or that replaces it with a generic number (1 rep = 1, under 100 rep = 2, under 1000 rep = 3, under 10000 rep = 4, etc. that way you know generally whether they can be trusted but you aren't relying necessarily on "oh he has 30,000 rep and the other person only has 17,000, so I should trust the 30k user more).
    – TylerH
    Commented Jul 22, 2015 at 3:49
  • 4
    One could rename "reputation" to "internet points". That said I have the same bias, am aware of it and overcompensate. A new user with a decent answer gets an upvote from me, a high rep user with a good but not great answer does not. Maybe it is envy and not overcompensation. Also I don't know if my behavior is a problem.
    – nwp
    Commented Jul 22, 2015 at 13:36

10 Answers 10

81

One important thing to note here that is relevant to your first two points is that a high rep user might have acquired their reputation in tags that have nothing to do with the current question at hand.

I have most of my reputation from jQuery, Facebook and PHP questions however if I were to visit the C++ or Objective-C tags my reputation wouldn't at all reflect my capabilities or knowledge in those fields.


In order to not be affected by we need to be able to focus exclusively on the content as opposed to the user who is contributing the content. This is easier said than done simply because content is usually directly attributed to a user - without any additional actions the reputation of a user is displayed along side their questions/answers and even on their username in a comment (when hovering).

A good point to consider (related to my previous points about gaining reputation for tags not relevant to the post at hand) would be that there are many experts in a certain field that have not yet opened a Stack Overflow account. When a brand new user joins they are given 1 reputation point regardless of their actual expertise. These new users are very capable of posting great, quality content but if one was to glance at their profile it wouldn't indicate that this user is a professional. Only by judging their contributions would one be able to get an idea as to the level of their experience/expertise.

Remembering that reputation and any other Stack Exchange based gamifiation metrics are solely based on the users actions within the context of the site will help us to put the reputation metric in proportion and focus more on the content.

8
  • 11
    To that end, is there any value to highlighting someone's top tag/s somehow? (I know their profile page does but that's not visible on a question or answer) Maybe in the expanded out view of their profile when you hover over them? Commented Jul 21, 2015 at 10:41
  • One important thing to note here that is relevant to your first two points is that a high rep user might have acquired their reputation in the same language tag, prior to the language changing, and as such has little to do with the questions at hand. Commented Jul 21, 2015 at 15:25
  • 4
    Or they could have got a lot of rep from 'low hanging fruit' questions. Reckon you could farm rep off the 'gimme regex' questions that come past.
    – Sobrique
    Commented Jul 21, 2015 at 15:53
  • I'm not sure 'top tag' highlight is necessarily helpful - it promotes an 'appeal to authority' which is undesirable. To a certain extent - so does rep though, so ... shrug. Makes little odds in my opinion
    – Sobrique
    Commented Jul 21, 2015 at 16:02
  • @SuperBiasedMan Wow, I didn't even know that something appears when you hover over a user's card like that. Neat. | Edit: Apparently it only happens for some card only?!
    – SeinopSys
    Commented Jul 21, 2015 at 16:34
  • @DJDavid98 It's a privilege, at 1000 rep I think? And it can be disabled if the person doesn't want it. Commented Jul 21, 2015 at 16:51
  • While that is 100% true, I would guess that the "K" behind your repuation influences the perception of your expertise when answering a C++ question. The reader is highly unlikely to analyze where your rep came from before forming a judgement influenced by your rep.
    – Eric J.
    Commented Jul 21, 2015 at 19:03
  • Just make the display of the reputation of other users optional. Thereby we can focus solely on the content (if we want to). This would actually be a feature request. Commented Jul 30, 2015 at 15:30
55

This is an interesting topic and, as somebody who has been referencing SO for years, but just opened an account last year, I've thought about this before.

Every question should be evaluated solely on the merits.

Reputation should be used optionally as a filtering device.

We don't know much about the person asking their first question (they could be an expert in their field) and a high rep user could be asking a question in a subject tag where they have no experience (which would make sense).

UPDATE
Thanks @MattP (comment below) for illustrating my point (in world-class fashion, I might add) that a person asking their first question on SO might nonetheless be an expert in their field, and how a user's SO rep may not always reflect their skills and experience.
How can I see the internal compile commands which fail in a "go get" installation?

4
  • 5
    You just proved my point with your apologies. You are welcome to jump into whatever topic you want to participate in, regardless of your reputation level so thanks for that.
    – D4V1D
    Commented Jul 21, 2015 at 14:43
  • 2
    I definitely respect reputation. I just don't think anybody should be intimidated by it or change their normal behavior. Like I said, when answering a question the focus should be on the substance alone. Commented Jul 21, 2015 at 14:48
  • 1
    [I believe] I almost always avoid being influenced by someone's reputation by almost never looking at it.
    – TripeHound
    Commented Jul 22, 2015 at 13:47
  • 3
    Proof this week that SO points != Reputation. stackoverflow.com/questions/31345230/…
    – MattP
    Commented Jul 23, 2015 at 18:57
41

Rep is more a measure of activity than skill. It pays to remember this. Usually there is a correlation between 'active' and 'knows how it works here (and is a bit more invested in the community)' which usually means better questions/answers.

But that really all rep means. Even tag score - might well be more a measure of activity than skill.

So for all that - higher rep means I will look twice at an answer that I think is "rubbish" at first glance, just to ensure I'm not missing something profound. But if it genuinely doesn't look 'right' (and assuming I have at least enough supporting knowledge to tell) I'll do what I do with any user - comment requesting clarification/expansion, downvote or vote to close as appropriate.

Now, higher rep might mean I look twice, but it also means I think you should "know better" what makes a good answer/question.

To answer your broader question though - the only way you can avoid bias is to hide the source of it. E.g. obfuscate our magic unicorn points either partially or fully.

Now personally, I don't like 'badge count' anyway - some of the badges are considerably less meaningful than others. But to an extent, the same is true of rep score - you will look at it because it's there, and any conclusion you draw will generate bias.

So perhaps that's a case to say - hide both. Drop the rep score and badge count from the question/answer page. (Still allow it to be viewed if someone looks at a profile though).

The one thing I would still find useful for answering questions is metadata about their behaviour once they get an answer. I'm thinking primarily in terms of distinguishing the low rep users for whom this is their first question, and the low rep users who have a lot of negative scoring questions/answers and can't be bothered to accept.

So maybe something like:

  • Illuminator - more answers than questions.
  • Well received - Questions score / number of questions asks > 1
  • Acceptor - >50% of asked questions have accepted answers.
8

The whole point of reputation is a standard indication of how skilled and how long someone has participated in a community. In this case of SO, every tag is almost it's own community. (refer to the textbook definition of what reputation is considered to be)

I feel that it isn't wrong to have a reputation bias, because a higher reputation can (and usually is the case) but does not directly/always mean:

  • The user has been here for a period of time.
  • The user has participated on the website as an answerer and their answers have helped others and is deemed useful.
  • If the user has a higher rep than you, it appears that the user is more skilled and knowledgeable in a certain area than you are.
  • The user posted quality answers in the past, contributing to the high rep.

All this is fine and dandy, and makes a good yardstick to judge someone's competency, it is however not always representative of their answer posted.

It is common to see really high rep users leaving short answers that appear to answer the question, but not provide enough help to the asker, leaving the asker unsure of the case. I've even commonly see answers posted as comments, also ever since being allowed access to the review queue, time and time I've seen answers by users with their rep in the denominations of tens of thousands show up due to their one lined answer with a link.

That being said, I've also seen users with lower rep provide a much more in-depth and detailed answer that might have the same concept/implementation, but gives more explanation and even points of reference for further reading. These are definitely more helpful than some answers I've seen posted by 10k++ users that give a very short explanation and leave a link to do most of the heavylifting in explanation.

(I feel the need to provide an analogy that is people with a clean and good record in public does not mean that they are saints all the time, and people with a felony or criminal record does not mean they are bad people, it just means they have did something bad in the past, and tarnished their reputation)

In short, it is okay for reputation to be used as a yardstick, and contribute to a slight bias, but it should not be taken as a factor in considering what a 'good answer' should be to a question, because every question is different and has specific requirements and every answer is also the case. Users with lower rep should definitely be given a chance that they can provide good answers, (even the higher rep users started from the bottom), and everyone's bad answer should be improved as required.

Note: This is what reputation is defined as on Google.

define rep

1
  • 5
    Actually we have reputation points here, not reputation (which would be very difficult to measure). Reputation points may or may not be a proxy for reputation. More likely they are a proxy for participation. Commented Jul 22, 2015 at 21:55
5

The main way I know that it affects my behavior is that I don't leave comments when I down vote anything by a user who also has enough rep to down vote. I do find myself checking their profile and recent activity if a high rep user does something like post a blatant recommendation request.

The only thing (I'm aware of) that makes me less likely to flag is unfamiliarity with the technology/language and task. It's harder to be sure the provided code is a proper MCVE, for example.


To address the main question, "what one needs to do in order to not be impacted by what I call the reputation-bias?", you could use that script to hide that information if it's a serious problem.

You could try doing a focused read of the question instead of scanning the page first but seeing someone's rep before you've formed an opinion of the question is a habit, that'll probably be hard to break.

But I think the, "imagine you're talking to a busy co-worker" advice the site gives to Askers can help Answerers here too. If you were busy and this question came from a co-worker with decent/good skill level, how would you react? Would you have to put them off until you had more time to answer it? It's probably too broad. Would your answer be different if you worked at a different company? Probably opinion based. Did someone else ask you this yesterday? It's a duplicate.

You can also try flipping your internal script. Would you hesitate to flag a post if the user had 5 instead of 5K rep? Would you really think this question were OK if the user had 5K instead of just 5 rep?

In trying to be fair to new users, remember that the new user whose question you're looking at is not the same new user who yelled at you when you tried to help them. They are not affiliated in any way. They just happen to also be new.

In trying not to be too generous to high rep users, remember that they've already got a ton of rep. A -2 isn't going to sink them and if anyone should have been able to tell that their question was off topic, it's them.

It'll probably slow you down at first, because you have to deliberately interrupt and interrogate the bias, but hopefully it'll just become a habit.


Seeing a user with more rep than you be a jerk also helps take the shine off, but I don't recommend it.

3
  • A shame you don't comment on 125+-rep users posts. Or does that change once it's considerably higher, like maybe 3/5/10/20K? Commented Jul 21, 2015 at 18:25
  • @Deduplicator - I only avoid commenting if I'm down voting (although leaving a comment when someone else down votes is also risky). If I'm not leaving a down vote I have no problem leaving a comment.
    – BSMP
    Commented Jul 21, 2015 at 18:37
  • And once you hit 1k, you'll be able to see the actual number of up/down votes instead of just an aggregate score.
    – ryanyuyu
    Commented Jul 21, 2015 at 18:57
4

(Answering your first question "how their reputation level affects your behavior")

I feel that I can be much more straight with high rep users (just write facts or opinions without beating around the bush) whereas for low rep users I assume they are relatively new, and I will often reread my texts to avoid having them feel offended. And impatience/repetition sometimes counterbalances that ;-)

(Answering your second question "how your own reputation level affects your behavior")

Another effect comes from hunting 'the next' badges, so that depends on your current reputation level: if you are going after certain badges, your behavior will be geared towards achieving that goal.

As an example: I try to reach the 1000 level for most queues - when I have reached that level for a certain queue, I hardly ever visit it.

You could say these are changes on a 'quantative' level, not 'qualitive' - I still try to contribute with the same intention, and the quality of my actions will only change gradually through experience.

1
  • Please also answer the main premise of the question: what one needs to do in order to not be impacted by what I call the reputation-bias?
    – Martijn Pieters Mod
    Commented Jul 22, 2015 at 17:51
4

When judging a question (is it clear, answerable) or an answer (is it correct, relevant for the question) I generally read the text before looking for the writer and its rep. The only exception is the list of interisting/recent questions where I tend to skip questions asked by 1 rep user when I cannot understand the title.

I try hard to stick to that rule, because I've already found nice posts written by low rep users. But honestly, I would not like the rep nor the user name to be hidden. I mainly use SO for researching information, and when I see Martijn Pieters name, I know that it will be high quality information, same for BalusC when it comes to JSF, and many other names...

For the problems of votes, I seldom downvote, mainly incorrect answers (in my understanding) or unintelligible questions. I try hard not to look at the rep for that use case, but when I see the name of a user from whom I have already seen good posts, I read it twice and ask for clarifications in a comment instead. And often it was my misunderstanding of a point, or simply a lack or precision. In order to try to be fair to low rep (more recent) users, I generaly point out in a comment the reason for a downvote (even if I know that it is a disputable usage) and remove it after a fix - or after they convinced me that their answer is correct.

For upvotes, I assume that they are far more important to low rep users (I was one not so long), so I only look at the quality of the post, and do not find it hard because I really think that Hans Passant, Paxdiablo, or Jonathan Leffler (not speaking of Jon Skeet) are not really running after one more upvote.

For accepting answers, I always accept one, the one that best answered my own problem or leaded me to the right path. Here, rep should be ignored IMHO, because it is not the point even if it can help to be confident in an unexpected way to solve a problem. But the rule should be simply accept the answer you used.

The case where I most use rep is in fact in comments. As they are short and poorly formatted text, it if hard to judge per their real content. As an example, when a low rep user asks me Are your sure? on one of my answers, I read it again, test it and I try to explain why I am. But when one of our highest rep user asks the same, I take for sure that I missed an important point and immediately dig into the manuals! The nice part is that comments have no influence on rep.

So the rules to limit the bias are simple :

  • when voting for a post, only look at its content - except that if you know the poster to be an expert, you should better ask in a comment what you do not understand first ;-) - anyway a high rep user should not care too much for one single vote
  • when accepting an answer, just accept the one you used, or that lead you towards the solution you finally used - votes should be for the intrinsic worth of the answer for the community

My conclusion is that I think that the rep bias most certainly exists, but we must also note that there are really experts posting in SO. And of course their posts are much more upvoted than the ones from ordinary users ... but they really deserve it! So while the current system certainly has enormous faults, it's as Churchill said about democracy(*), I still think it is among the best possible ones. Because as I have already said, I would not like that poster name and its rep are hidden when I use SO for what it is: a reference for questions and their answers.


(*) democ­racy is the worst form of Gov­ern­ment except for all those other forms that have been tried from time to time (Churchill, House of Com­mons, 11 Novem­ber 1947)

1
  • I was about to upvote without even reading: just by looking at the length of the answer and at your rep. Then I refrained myself and read it. Upvoted anyway :)
    – D4V1D
    Commented Jul 23, 2015 at 14:43
1

For the reasons you listed, I am in favor of not exposing the credentials of a poster until you have committed to vote. That will give users a chance to assess a question or answer objectively without being swayed by the reputation of the poster. Once you have voted, up or down, you should be able to see the credentials of the poster. I realize that you can retract your vote but hopefully your real vote becomes more objective.

13
  • Is there a feature request for this anywhere? Commented Jul 21, 2015 at 16:12
  • @JamesThorpe, not that I know of.
    – R Sahu
    Commented Jul 21, 2015 at 16:17
  • 1
    Ok - just how you phrased your answer, sounded like there might be :) Perhaps you should propose it - see how it fairs! Commented Jul 21, 2015 at 16:18
  • I agree. Could worth a feature-request MSO question.
    – D4V1D
    Commented Jul 21, 2015 at 17:09
  • @D4V1D, this answer has two up votes and two down votes. That's a quick glimpse of what people think of this idea :)
    – R Sahu
    Commented Jul 21, 2015 at 17:16
  • So, if we never vote on the post, we never get to see who the poster is?
    – Kevin B
    Commented Jul 21, 2015 at 17:22
  • @KevinB, that's my thought.
    – R Sahu
    Commented Jul 21, 2015 at 17:23
  • 1
    That could help in reducing the reputation-bias users (at least I) tend to have when interacting with SO.
    – D4V1D
    Commented Jul 21, 2015 at 17:23
  • 1
    Just added: meta.stackoverflow.com/questions/299805/…
    – R Sahu
    Commented Jul 21, 2015 at 17:40
  • I haven't seen the duplicate coming.
    – D4V1D
    Commented Jul 21, 2015 at 18:00
  • I like the idea, but the final duplicate link makes a lot of sense. Too bad for the dupe hammer.
    – Cayce K
    Commented Jul 21, 2015 at 19:05
  • 1
    @CayceK, it was worth a shot :)
    – R Sahu
    Commented Jul 21, 2015 at 19:14
  • Providing an option for this and not making this a default would be nice. Commented Jul 22, 2015 at 13:53
0

I currently have more than five accounts on Stack Overflow covering upwards of 800 answered questions. When I start a new account I can go for weeks without getting any reputation points as users mostly gravitate to awarding high-reputation points users' the answer even if it's not the optimum solution. There are so many questions with the best answer unrewarded throughout Stack Overflow.

I answered a similar question on two different accounts using the same answer, one with over 4k rep and another with <200 rep and the awarded points is truly astounding.

Give it a go creating a new account and view for yourself the bias.

Edit, to give some examples of where SO hurts new users here are 3 examples from only 14 answers on this account [ps this I'm not complaining :)]

Dell Service Tag Request using VBA - Spent some time on this Q. OP can't give feedback, experienced user attempts to answer and gives no feedback either.

Fully reference a worksheet by codename - Accepted answer here was totally wrong originally and had to be corrected in comments (which were since deleted)

Excel extract from brackets - Accepted answer is based on edit which copies an existing answer.

6
  • 2
    Some has already done it, actually.
    – D4V1D
    Commented Jul 22, 2015 at 21:09
  • I didn't find that when I tried stackoverflow.com/users/1494478/ms Commented Jul 22, 2015 at 21:13
  • @MartinSmith - surely you cannot deny there are many excellent answers not appreciated. For every example you dig out I could do the opposite.
    – user3904868
    Commented Jul 22, 2015 at 21:18
  • 1
    If I see a good answer by a low rep user in the tags I frequent I'd probably be more likely to upvote - to encourage them to stick around - than for the same answer by a higher rep user tbh. Commented Jul 22, 2015 at 21:20
  • I agree - many of the excellent answers are by low rep users but their effort is not being rewarded.Some of the most difficult questions I have asked are solved by <1k rep users. However, most Q's are asked by people that have no idea what is the best answer so pick the highest rep.
    – user3904868
    Commented Jul 22, 2015 at 21:22
  • 1
    ... why do you turn over a new leaf so often??? Commented Jul 22, 2015 at 23:54
0

Reputation SHOULD influence decision making - that is the whole point of it. I find your concerns valid but they are only the down side of a good thing.

7
  • 3
    I disagree. A contribution should, and only, be judged as per its quality, not by the reputation of the author.
    – D4V1D
    Commented Jul 23, 2015 at 12:43
  • 2
    That would make the entire concept of expert moot. And since you disagree and downvote my answer - I assume you have not been really asking a question. Commented Jul 23, 2015 at 12:47
  • I asked this very question: "what does one need to do in order to not be impacted by what I call the reputation-bias?". How downvoting and disagreeing with your answer is even related to whether I have been really asking a question?
    – D4V1D
    Commented Jul 23, 2015 at 12:52
  • Because in your first comment to my answer you gave an answer yourself - that means you are not expecting opinions but merely fishing for rep. Commented Jul 23, 2015 at 12:55
  • 1
    My opinion has been polished since I asked this question thanks to the answers this topic has gotten. Also, how can I be fishing for rep on MSO?
    – D4V1D
    Commented Jul 23, 2015 at 12:57
  • 3
    @D4V1D I do believe reputation should impact your interactions on stackoverflow. however, i also think posts should be judged based on the merit of the post, not the person who posted it. Those two things may look contradictory on the outside, however, they are not. Interaction on stackoverflow is far more than giving out votes, it also includes commenting, and sticking around after the fact. I may comment differently when responding to someone with high rep vs someone with low rep. If someone with high rep tells me i'm wrong, i'm far more likely to research to see why they think that.
    – Kevin B
    Commented Jul 23, 2015 at 22:19
  • Although it is a bias. Some new users write better answers than me.
    – m4n0
    Commented Jun 18, 2018 at 13:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .