-51

I found an analysis of Stack Overflow on Quora. I'm not sure if its premises are accurate, but it has merit according to my experiences here.

It alleges that a high reputation on Stack Overflow can be obtained without having a high level of programming experience, simply by answering easy questions and other steps not requiring high expertise.

A user who answers a difficult question gets the same rep as one who answered a hard one.

This leads to the idea that there should be a survey (in whatever fashion convenient) done of programming experience levels in Stack Overflow, to attempt to get hard data about whether there are moderators and high rep users who don't really have such high experience in their fields. I don't necessarily think that it is this way (I always thought their attitude came from their high experience), but I think there should be data collected to attempt to tell if a problem like this is occurring - if there are "noobs" acquiring high reputation on this site by gaming the system.

Quote:

It is because the system is flawed.

I have over 15000 points on SO.

People hate it because some users there are NUTS, and when you'll start asking questions, you'll inevitably run into those users.

Large portion of moderators don't care about knowledge, learning, programming, they care only about rules, and proving themselves to be right.

Current reputation system does not reward quality of questions, but their quantity. Newbie question will grant you more upvotes than something that require 5 years of experience and arcane knowledge.

As a result, if you grab book for beginners and type very fast, you can farm reputiton on simple questions. Over time you can rise to the godhood, err, moderatorhood and then you'll be able to reign supreme over people that ask questions. Another consequence of that is that when you get DIFFICULT question, you won't be getting answer on SO. Instead you'll run in some clueless "know-it-all" that 'll argue with you to death about unrelated issue.

There are quite a lot of non-knowledgeable and toxic people with over 50k reputation because they had too much free time. Some of the knowledgeable people are still around, but they're pretty much a minority.

Community is not professional by any means. Asking questions would result in your question being closed in under 5 minutes, then there's high probabilty that someone will mock or argue with you in the comments, answers that you'll get are likely to overlook important parts of your question or address unrelated, but similar problem, because they didn't bother to read your entire questions. In case your questions resembles anything else, you'll waste half of a day arguing in comments and trying to explain why it is not a duplicate (although you already explained that in questions text which nobody bothered to read).

Take a look at the SO developer survey: https://stackoverflow.com/research/developer-survey-2015 and the 12,000 out of 26,000 users listed as "something else". Thats not really useful information.

I suggest to correlate years of programming experience, with reputation and to use this to ascertain whether the site is being taken advantage of reputation wise, as alleges this user.

EDIT: it appears this data has already been collected, so now I just suggest that the analysis be done and the graph(s) be created.

15
  • 21
    So a lot of rep points isn't an exact predictor of technical skill, and anyone looking to make a hiring decision based on a SO profile better look at users' actual track records. That's hardly news. If you want a thorough analysis done, go ahead, here's the data - but programmatically telling apart a cheapo user from a legitimate one is a very hard problem.
    – Pekka
    Oct 17, 2015 at 23:07
  • 18
    No one is required to comment on the reason for their down votes. Putting that statement at the top is just going to get you more.
    – theB
    Oct 17, 2015 at 23:09
  • Good point. What's your personal level of programming experience? I am recommending to study whether there is a large problem with this via statistics. Oct 17, 2015 at 23:09
  • I didn't ask the reason, I asked what is your experience with programming and who am I talking to and dealing with? Oct 17, 2015 at 23:10
  • 16
    So suggest some actual statistics to detect "bad" users. Show us a query to start with. If it's good, or a good start, it's bound to impress people here. (I know absolutely nothing about programming. I hang out here only for the drama)
    – Pekka
    Oct 17, 2015 at 23:10
  • 4
    @Pekka웃 - Good week to hang out here last week then.
    – theB
    Oct 17, 2015 at 23:15
  • 2
    'Same when up-voting.' - a waste of 20 keystrokes. Oct 17, 2015 at 23:27
  • 4
    If I was of a suspicious frame of mind, I would say that this question is an attempt to farm CV information for sale to agencies. Oct 17, 2015 at 23:29
  • @Pekka웃 Can I query the "developer-survey-2015" somewhere? The info I need (ie. real world experience) isn't in the general stack overflow data. Oct 17, 2015 at 23:38
  • I don't think you can, @user1122069. But given that the info is unverifiable and there wasn't even any checking of rep levels against the user's actual user profile, it's unlikely it would have helped much.
    – Pekka
    Oct 17, 2015 at 23:45
  • The page you linked to, if you scroll all the way to the bottom (past where the results are all broken down) the Unicorn has the information you want. Seriously. He's got it.. This comment provided by [unicorn-tears]
    – theB
    Oct 18, 2015 at 0:52
  • 14
    "When down-voting this post, please state your level of programming experience in the comments. Same when up-voting." I'm a moderator, and who do you think you are to tell us what to do based on our votes?
    – BoltClock
    Oct 18, 2015 at 4:31
  • 2
    I don't think answering "easy" questions is necessarily bad, undeserving of rep, or an indication that someone is "cheap", "farming rep" or lacks technical knowledge. Quite often I see answers to newbie questions from both high and low rep users that manage to provide useful insights into the language illustrated by the bug or problem the newbie user is happening. Reading through these worked examples of simple debugging and logical problem solving can be useful even to an experienced programmer. All this obsessing about what is "legit" rep gets old. If someone upvoted it, it's legit imo.
    – samgak
    Oct 18, 2015 at 6:50
  • 2
    This post isn't so much research into our experience levels as research into how far downvoted an MSO post can get.
    – ArtOfCode
    Oct 18, 2015 at 22:05
  • 1
    @ArtOfCode - The answer is -162.
    – theB
    Oct 19, 2015 at 18:21

4 Answers 4

26

A lot of rep points isn't an exact predictor of technical skill, and anyone looking to make a hiring decision based on a SO profile better look at users' actual track records. That's hardly news.

The level of expertise users have provided in the survey is unlikely to help this issue. For one, you can lie about your expertise (and your rep level). For another, expertise on a topic isn't always a requisite for writing a decent or even good answer on SO.

I'm not sure whether numbers are what you are really looking for, though. The observations in your article pretty much amount to "Stack Overflow sucks and isn't working anymore." You'll find people who share that view, and people who will vocally disagree. It's an inherently subjective question, and it is unlikely that any amount of analysis will settle it.

In a similar vein, while we all can tell a bad-faith, easy-upvote user from a dedicated expert serving their community when we see them, it's super difficult to define a set of rules, or write an algorithm, to find them - also because defining "frivolous" SO activity is so subjective, too.

Perhaps the real question is something else?

9
  • 2
    Yes, the real question is to see if any good users on SO. So much hostility here. I think from the stats, people are qualified. Just I'm not sure why so many would be against finding out these stats, to prove or disprove. I did find one relevant in the survey which I found after writing my post. Not exact, but a good estimate. "Problem" is elsewhere then. I definitely did more want to repost this user's comment as my own. Oct 18, 2015 at 0:03
  • 3
    The site is framed narrowly and with much recourse to user opinion and feeling in judging that, so it is bound to alienate many people. It makes a good encyclopedia though, I just won't be here for the "community". Fair enough. Oct 18, 2015 at 0:06
  • 4
    @user the real question is to see if any good users on SO. I think the only way to really find out is to hang out here, or analyze manually users' track records. It's usually pretty easy to spot the good users. I personally think they still outnumber the bad ones. The site is framed narrowly and with much recourse to user opinion and feeling ... so it is bound to alienate many people. Agreed. I too think we may have become too narrow and bureaucratic in what questions we accept - but then SO is getting 10k+ questions every day, so maybe it's the only way to stay sane. I'm not sure.
    – Pekka
    Oct 18, 2015 at 0:19
  • 1
    Definitely a lot of real questions, intended and unintended. I think the partial result is that it is definitely traits related to programmers. There is definitely a shortage of super-experts, but then why should such work for free... I definitely get angry with the non-technical staff where I work and answer them shortly... "how many times do I need to tell you to submit a bug report through the bug report system? I can't do anything from your email." Oct 19, 2015 at 0:16
  • My personal offense here is that some questions will only be understood correctly by someone of a high expertise. A short message is sufficient for such users, but then come along others who say "I don't understand your question." They want the question adapted to them and threaten to close the post and down-vote it. If the problem asks too much of the responder, it is not well accepted here, even if it is technically a real question. It definitely just boils down to the problem of free labor. It would be cool if thats what this site was, with some reciprocity, but its not. @Pekka Oct 20, 2015 at 16:19
  • I've started using Upwork when I need help on private projects, but in a work environment, I would view them as competitors. Oct 20, 2015 at 16:22
  • For rep there + $10 to $20 they will solve some good problems. Oct 20, 2015 at 16:24
  • @user - it does happen that questions get treated badly because they're too advanced for most people to grasp, but in my experience, that is very, very rare. I think it's fair to say that most questions are indeed downvoted for lack of quality... if you have examples to the contrary, I'd be interested to see them. Haven't heard of Upwork, will check it out
    – Pekka
    Oct 20, 2015 at 18:03
  • For example, this question made it past the moderators, but nobody answered it or up-voted it. One person made a stupid but not offensive comment. So out of 2 posts in the last week, I got one answer. Still pretty good, but a bit weird. As a rule, my Java/Android questions never get answered, but I got one about AndroidStudio. stackoverflow.com/questions/33299002/… Oct 24, 2015 at 16:25
28

To make it clear: whenever I use the second person ('you') below I mean it in the general sense, not you as in user1122069. I do this because to my ear saying 'one' seems pretentious.

So there are a couple issues with the analysis. First:

Take a look at the SO developer survey: https://stackoverflow.com/research/developer-survey-2015 and the 12,000 out of 26,000 users listed as "something else".

The something else, includes Enterprise level services developers, CIO/CTOs, System administrators, DBAs and a few more. Scrolling down the survey a little bit, we can see that 71% of respondents are a developer of some sort, 3% are VP level executives or System administrators, and 14% are students.

Second: expertise != reputation

Reputation means that you've contributed to the site in a meaningful way. Wrote an answer that helped people? Awesome. Edited some posts to make them better? Awesome. Reputation does not imply expertise, any more than correlation implies causation.

Now on to the 'commentary'.


It is because the system is flawed. I have over 15000 points on SO.

Good for him? Jon Skeet has 800000 and I have 1900. Good for us? Irrelevant to the discussion.

People hate it because some users there are NUTS, and when you'll start asking questions, you'll inevitably run into those users.

Hi, welcome to the world. There are crazy people in it. And when you collect a couple million of them in one place, it's statistically impossible that there are no crazy people. We certainly do our best to keep the worst offenders away, but sometimes people slip through the cracks. Some tags are also more likely to have angry people. It happens. If it's offensive or not constructive flag it for moderator attention and move on.

Large portion of moderators don't care about knowledge, learning, programming, they care only about rules, and proving themselves to be right.

Moderation by it's very nature is often going to make someone mad. If I had a nickel for every time someone complained about a moderator, I'd quit my job and move to the Bahamas.

In the general case I've found the moderators here far better than anywhere else I've been out there on the web. Yep, they are human and make mistakes sometimes, but with the amount of crap they're expected to sort through, can you really blame them?

Current reputation system does not reward quality of questions, but their quantity. Newbie question will grant you more upvotes than something that require 5 years of experience and arcane knowledge.

Sort of. FGITW has been discussed many times. You can even make a convincing argument that it is a problem. (Not that the original writer did that.)

But if you post an answer to an arcane question that requires years of experience in some obscure topic to answer, then only the people with years of experience in that subject will be able to evaluate its quality. If you choose to answer MUMPS questions, no one is stopping you, but the small group of people worldwide that use it are not going to grant you a million up votes. Such is the life.

Additionally, badly worded new user questions may get you a small handful of upvotes right away, but the well crafted answers are useful for a long time, and continue to collect votes on into the future. (Jon Skeet could quit posting entirely, and would still be collecting 200 reputation a day, for a long time. Quality matters.)

As a result, if you grab book for beginners and type very fast, you can farm reputiton on simple questions. Over time you can rise to the godhood, err, moderatorhood and then you'll be able to reign supreme over people that ask questions.

Hyperbole is an art that this person has apparently mastered. First off, high reputation != moderator. Moderators have diamonds next to their names, and can generally act unilaterally, whereas even 100k reputation users generally cannot. With the exception of employees, the moderators are elected by the community as a whole.

The only exception to the unilateral action is that gold badge holders can close a question using their dupehammer. Look at the requirements for the gold tag badge. It's hard to get, even if you're spamming answers at the newbie questions. Are there instances of conflict? Of course there are, but most of the time it's handled quickly and efficiently by the community. (Just look through the past few days here on meta.)

Another consequence of that is that when you get DIFFICULT question, you won't be getting answer on SO. Instead you'll run in some clueless "know-it-all" that 'll argue with you to death about unrelated issue.

If you know better than the people who you've come to ask a question from, then why the heck would you come here to ask? Do you go to your doctor when you're sick, and then tell the doctor that she's wrong, and you know better? Of course not. If it's really an unrelated issue, say so and if the person doesn't get it, then let it go.

There are quite a lot of non-knowledgeable and toxic people with over 50k reputation because they had too much free time. Some of the knowledgeable people are still around, but they're pretty much a minority.

The old "things were better in my day" argument. Frankly, no they weren't. Five years ago I wouldn't have found SO as useful as I do today. There were a lot of questions and answers that wouldn't make it through the quality controls enforced now. The second part of the statement is completely speculative and I'm not going to even address it.

Community is not professional by any means. Asking questions would result in your question being closed in under 5 minutes, then there's high probabilty that someone will mock or argue with you in the comments, answers that you'll get are likely to overlook important parts of your question or address unrelated, but similar problem, because they didn't bother to read your entire questions. In case your questions resembles anything else, you'll waste half of a day arguing in comments and trying to explain why it is not a duplicate (although you already explained that in questions text which nobody bothered to read).

If you write a turd, then it's a turd, and should be closed quickly. I've seen plenty of questions here that I can't for the life of me figure out. If your question isn't clear, it isn't clear. It doesn't matter if it makes sense to you, but rather whether it makes sense to the person who will answer it. If you have a question with a lot of little intricate details, break it up into smaller parts, simplify, or clarify. If you're arguing with people in comments about how horrible they are that they forgot to answer point 21a buried in paragraph 16, sub-section 21, then don't let the door hit you on the way out. Bye Felicia.

I've seen people take hours out of their day working with someone who doesn't speak English, to help them write their question and help explain the answers to them. Where else can you ask a question about some obscure Windows API behavior and have Raymond Chen answer? Where can you go ask a Java question and get an answer from Jon Skeet or Eric Lippert? Unless you're counting books, then the answer to those hypothetical questions is: nowhere.

But totally right. We're all a bunch of unprofessional jerks, that know nothing, and somehow have built the best single repository of information about programming available on the web.

And no one is going to end up reading all this way so if you did, then +5 for you :)

4
  • 1
    "Some tags are also more likely to have angry people." *cough* php *cough*
    – Bob
    Oct 18, 2015 at 0:32
  • 5
    Where's my +5?​
    – user4639281
    Oct 18, 2015 at 0:37
  • @Bob - I didn't say that... ;)
    – theB
    Oct 18, 2015 at 0:39
  • 4
    @TinyGiant - You should get it in 6-8 weeks.
    – theB
    Oct 18, 2015 at 0:39
10

If I follow your logic, you're saying one might be able to farm rep by answering easy questions only, and that such users might be able to get tons of reputation without being much of an expert. You claim this is "a problem" and amounts to "gaming the system." You propose correlating programming experience with reputation to determine if there are users with very little experience and high reputation.

First, even if "noobs" were racking up tons of reputation by answering easy questions, I don't see why this is a big problem. As stated in the first sentence of the help center page about reputation, reputation exists as a rough measure to determine how much the community trusts you; note that it doesn't say reputation exists to certify that you're an expert in some area. The logic is that if you have high reputation then you have been active on the site; it then stands to reason that you might be more trustworthy in carrying out moderation and other privileges.

Second, I don't see what we would learn by studying programming experience. I went to high school with an IOI gold medalist who at age 15 doubtlessly could have answered programming questions that few other people in the world could answer despite not having programmed for too many years. Alternately, a mediocre career software developer might not be able to answer challenging questions.

I don't see a way to study the phenomenon you're referring to short of hand labeling questions as "easy" or "hard" and determining if some high-rep users only answer easy questions. Even that sort of analysis would be fraught with the obvious subjectivity in the labeling.

9

That post seems to be yet another butthurt post of someone who got their question closed on the Stack Exchange network. The web is full of such posts. You can safely ignore them.

It is not our problem that those people can't write questions. Writing a good question is hard and takes a lot of work. If anything, I would want someone to have to prove their programming skill level before being able to ask a question.

In the years that I've visited SO (both before and after I made an account) I never had to ask a question to get my project moving again. I did start typing one more than often though, usually offline, in some text editor. Properly explaining my problem in textual form, i.e. "rubberducking" with myself, usually already points out a misconception or a lack of knowledge from which I could work forward. If not, then creating a Minimal, Complete, and Verifiable example so that I could post relevant code and relevant code only, would point out a trivial implementation mistake I made. If those two actions didn't already solve my problem (which honestly, in 90% of the cases it did), then another peek at the documentation, or a more targeted web search based on the things I just learned generally pointed out yet another misconception or mistake - problem solved, no need to post a question.

So then to your question: the post is based on the invalid premise that users with no professional experience are the one closing and downvoting such posts, "because they don't understand it".

No, the users asking such questions don't want to do any work to make their question clear, reproducible and understandable to a large audience. They want to post an unpolished "brain dump", so a wall of irrelevant text, either too much code or code that's simplified beyond the point of compilation and in their mind their target audience is the great, knowledgeable professional programmers that happen to have experience with exactly the thing they're trying to do, because they must have had the exact same problem before, will be present at the time of posting and thus must be able to answer it, right? So they think they don't have to put any effort in posting. They want to outsource the process I lined out above, and that is not what a professional developer does - and funnily, it's what gets your question closed on Stack Overflow.

Yet anyone who dares to have any critisism of their post must have a degree in the subject they're having trouble with, otherwise all their points are moot. Right. Like I said, ignore such posts. Don't let them fool you.

3
  • I agree with all this, except the bit about having a question and not needing to post it. That sounds to me like you've got a stack of self-answered questions waiting to be posted!
    – halfer
    Oct 18, 2015 at 10:39
  • 3
    @halfer "Q: I have this perfectly functioning method according to its unit tests, but the results are weird when I call it from my application" - "A: I called it with the wrong arguments" isn't really helpful for anyone else. Or more specifically, "I wanted to ask how you could do X in way Y, but found out in [this Q&A] you cannot do it the way I thought you could, so I used the approach from [other Q&A]". I'll be documenting my research rather than asking an answerable question. I may try to do so in the future though, thanks for the suggestion.
    – CodeCaster
    Oct 18, 2015 at 10:43
  • Agreed, not all blobs of research work well as questions. I like the idea though that a good question can be bashed out of a formless collection of things, and the disadvantage is that it takes a good deal of time to write well!
    – halfer
    Oct 18, 2015 at 10:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .