47

After downloading the raw data from the developer's survey, the salary field has a surprising number of NA's and a complete absence of any salaries over $200k. Is there a reason why the salary caps at 200k? The input appears to be a numeric field, and the values in the 190k range would indicate that the distribution is still fairly robust at that point. Was the survey intentionally truncated at <= 200k, is there a bug, or does no one who responded truly make more than 200k?

15
  • 3
    It must have been truncated, or there is a bug.
    – Travis J
    Commented Oct 13, 2017 at 18:34
  • 14
    Maybe 200k a year is just enough for people.
    – LinusG.
    Commented Oct 13, 2017 at 18:38
  • 3
    Or maybe the effects of wage suppression by Google, and the other BigTech players is still being felt.
    – user9903
    Commented Oct 13, 2017 at 18:45
  • 30
    Most developers here that make over $200k are locked up in a secure government facility and are never allowed to see the real internet.. or daylight. Commented Oct 13, 2017 at 18:45
  • 3
    @SterlingArcher Not true at all. Including stock, anyone senior who works for a major Valley firm is making over 200K. Some well over. Commented Oct 13, 2017 at 18:47
  • 35
    @GabeSechan that was mostly a joke Commented Oct 13, 2017 at 18:51
  • 2
    More probably because high-income people are less likely to share their salary, for many reasons, one of them being more at risk in case of data loss/leak ?
    – Pac0
    Commented Oct 13, 2017 at 19:17
  • 73
    Maybe devs making over $200k have better things to do than take surveys.
    – kjhughes
    Commented Oct 13, 2017 at 19:18
  • 3
    In any case, it raises the question on the statistical "corectness" of the distribution of the survey results. A correct comparison analysis with some public census data would be welcome.
    – Pac0
    Commented Oct 13, 2017 at 19:21
  • 5
    Perhaps they removed the values over $200k as being outliers for some reason. Afterall, highly paid dev's are allowed out of their secure facilities for major government holidays, and surveys.
    – Travis J
    Commented Oct 13, 2017 at 19:23
  • we don't show off through surveys Commented Oct 13, 2017 at 23:57
  • 2
    I know some brilliant people earning well in excess of 200k - they don't participate on SO, they're too busy 8-[
    – user3956566
    Commented Oct 14, 2017 at 10:20
  • The maximum limit is 200K huh? now I know why Sathya Nadella, Sundar Pichai, Tim Cook, etc didn't participated in SO at all.
    – Sagar V
    Commented Oct 14, 2017 at 13:37
  • 1
    Any decent survey has check questions. The kind that helps the surveyor to detect that the answers are probably not truthful. They go on the "maybe" pile. Commented Oct 14, 2017 at 22:22
  • 1
    "200k ought to be enough for anybody" - not Bill Gates (he almost certainly didn't say the 640K quote either) Commented Oct 15, 2017 at 21:50

2 Answers 2

19

Extreme outliers were trimmed from the current salary data by removing any response greater than $200,000, corresponding to the top half percent of responses. (source)

So not a bug. Apparently the SO data scientists felt that the results would be more useful without these outlier values.

4
  • 2
    Topcoding extreme values in public versions of data is usually not done to make results "more useful", but rather for reasons of privacy - to prevent identification of responders.
    – FooBar
    Commented Oct 15, 2017 at 19:33
  • 6
    However, what you usually do is not "trim" the data at the top, but replace all values >200 with the conditional average >200, to preserve correct unconditional averages.
    – FooBar
    Commented Oct 15, 2017 at 19:34
  • 1
    The main issue I take here is that the conclusions from this trimmed data were used elsewhere. Most notably for my complaint it was used in the salary calculator and THEN BLASTED ALL OVER STACKOVERFLOW WITH A NICE SHINY NEW BUTTON IN CAPS THAT WOULDN'T GO AWAY. Trimming off salaries above $200k, which in a field where the top salary is nearly $200M, makes any conclusions baseless. It would be nice to see a retraction of the calculator, the blog post, and any other set in stone statistical analysis which used this set of trimmed data.
    – Travis J
    Commented Oct 17, 2017 at 20:16
  • Thanks. I actually looked for this in the methodology, but apparently did not search in quite the right places.
    – Namey
    Commented Oct 23, 2017 at 21:24
10

We trimmed salaries at the top of the distribution for a few reasons:

  1. Concern that some of the high salaries could, when compared against other factors (e.g. location, role, languages used), be used to identify a specific individual.
  2. Suspicion that some of the high-end answers might be typos, deliberate trolling, or mistakenly including non-salary compensation in the figure.*
  3. Concern that outliers at the top would have extreme influence on means, standard deviations, and any regression models we built (such as the one used for the salary calculator).

In retrospect, setting a global cap was the wrong approach -- we trimmed .5% of all salary data globally, but close to 1.5% of responses in the USA. We're evaluating whether next year we should set the caps on a country-by-country (or perhaps region-by-region) basis, or perhaps simply set the cap based on the data from the USA.

That said, the excessive trimming in the USA didn't affect the medians we've reported. And, thanks to the specific type of regression model we used for the salary calculator, it didn't have much effect on the validity or reliability of the predictions from that model.

*For those who have pointed out that stock-based compensation in Silicon Valley often takes one's salary well over $200k...yes, that's true, but the question specifically asked about salary, before bonuses, grants, and other compensation.

4
  • 1
    Why trim salaries at the top and not at the bottom? Do we really think there's a USA based full time developer with 10 years of experiencing making only $135 a year (not $135,000 but $135. He probably assumed it would be interpreted as thousands). Commented Jan 23, 2018 at 1:19
  • @SamuelNeff well for starters, we asked whether they were employed full-time at the time of filling out the survey as opposed to salary over the past year. So someone who had been unemployed for 364 days prior to taking the survey might have given an unusually low answer. And from a statistical perspective, the floor of zero presents less of a problem than a ceiling of 10^10 (or whatever number of digits we had to use to accomodate users who answered in Indonesian rupiah).
    – Kevin Troy
    Commented Feb 1, 2018 at 5:29
  • When would you compare salaries in dollars and ruplah as if they're the same thing? How does that make any sense? You'd need to convert them to equivalent values to compare, and even then it really isn't that useful to compare salaries across such disparate countries. Commented Feb 2, 2018 at 4:03
  • @samuelneff yes, we converted the values, based on the currency the respondent chose to answer in
    – Kevin Troy
    Commented Feb 2, 2018 at 4:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .