After downloading the raw data from the developer's survey, the salary field has a surprising number of NA's and a complete absence of any salaries over $200k. Is there a reason why the salary caps at 200k? The input appears to be a numeric field, and the values in the 190k range would indicate that the distribution is still fairly robust at that point. Was the survey intentionally truncated at <= 200k, is there a bug, or does no one who responded truly make more than 200k?
-
3It must have been truncated, or there is a bug.– Travis JCommented Oct 13, 2017 at 18:34
-
14Maybe 200k a year is just enough for people.– LinusG.Commented Oct 13, 2017 at 18:38
-
3Or maybe the effects of wage suppression by Google, and the other BigTech players is still being felt.– user9903Commented Oct 13, 2017 at 18:45
-
30Most developers here that make over $200k are locked up in a secure government facility and are never allowed to see the real internet.. or daylight.– Sterling ArcherCommented Oct 13, 2017 at 18:45
-
3@SterlingArcher Not true at all. Including stock, anyone senior who works for a major Valley firm is making over 200K. Some well over.– Gabe SechanCommented Oct 13, 2017 at 18:47
-
35@GabeSechan that was mostly a joke– Sterling ArcherCommented Oct 13, 2017 at 18:51
-
2More probably because high-income people are less likely to share their salary, for many reasons, one of them being more at risk in case of data loss/leak ?– Pac0Commented Oct 13, 2017 at 19:17
-
73Maybe devs making over $200k have better things to do than take surveys.– kjhughesCommented Oct 13, 2017 at 19:18
-
3In any case, it raises the question on the statistical "corectness" of the distribution of the survey results. A correct comparison analysis with some public census data would be welcome.– Pac0Commented Oct 13, 2017 at 19:21
-
5Perhaps they removed the values over $200k as being outliers for some reason. Afterall, highly paid dev's are allowed out of their secure facilities for major government holidays, and surveys.– Travis JCommented Oct 13, 2017 at 19:23
-
we don't show off through surveys– Felipe PereiraCommented Oct 13, 2017 at 23:57
-
2I know some brilliant people earning well in excess of 200k - they don't participate on SO, they're too busy 8-[– user3956566Commented Oct 14, 2017 at 10:20
-
The maximum limit is 200K huh? now I know why Sathya Nadella, Sundar Pichai, Tim Cook, etc didn't participated in SO at all.– Sagar VCommented Oct 14, 2017 at 13:37
-
1Any decent survey has check questions. The kind that helps the surveyor to detect that the answers are probably not truthful. They go on the "maybe" pile.– Hans PassantCommented Oct 14, 2017 at 22:22
-
1"200k ought to be enough for anybody" - not Bill Gates (he almost certainly didn't say the 640K quote either)– Andrew GrimmCommented Oct 15, 2017 at 21:50
2 Answers
Extreme outliers were trimmed from the current salary data by removing any response greater than $200,000, corresponding to the top half percent of responses. (source)
So not a bug. Apparently the SO data scientists felt that the results would be more useful without these outlier values.
-
2Topcoding extreme values in public versions of data is usually not done to make results "more useful", but rather for reasons of privacy - to prevent identification of responders.– FooBarCommented Oct 15, 2017 at 19:33
-
6However, what you usually do is not "trim" the data at the top, but replace all values
>200
with the conditional average>200
, to preserve correct unconditional averages.– FooBarCommented Oct 15, 2017 at 19:34 -
1The main issue I take here is that the conclusions from this trimmed data were used elsewhere. Most notably for my complaint it was used in the salary calculator and THEN BLASTED ALL OVER STACKOVERFLOW WITH A NICE SHINY NEW BUTTON IN CAPS THAT WOULDN'T GO AWAY. Trimming off salaries above $200k, which in a field where the top salary is nearly $200M, makes any conclusions baseless. It would be nice to see a retraction of the calculator, the blog post, and any other set in stone statistical analysis which used this set of trimmed data.– Travis JCommented Oct 17, 2017 at 20:16
-
Thanks. I actually looked for this in the methodology, but apparently did not search in quite the right places.– NameyCommented Oct 23, 2017 at 21:24
We trimmed salaries at the top of the distribution for a few reasons:
- Concern that some of the high salaries could, when compared against other factors (e.g. location, role, languages used), be used to identify a specific individual.
- Suspicion that some of the high-end answers might be typos, deliberate trolling, or mistakenly including non-salary compensation in the figure.*
- Concern that outliers at the top would have extreme influence on means, standard deviations, and any regression models we built (such as the one used for the salary calculator).
In retrospect, setting a global cap was the wrong approach -- we trimmed .5% of all salary data globally, but close to 1.5% of responses in the USA. We're evaluating whether next year we should set the caps on a country-by-country (or perhaps region-by-region) basis, or perhaps simply set the cap based on the data from the USA.
That said, the excessive trimming in the USA didn't affect the medians we've reported. And, thanks to the specific type of regression model we used for the salary calculator, it didn't have much effect on the validity or reliability of the predictions from that model.
*For those who have pointed out that stock-based compensation in Silicon Valley often takes one's salary well over $200k...yes, that's true, but the question specifically asked about salary, before bonuses, grants, and other compensation.
-
1Why trim salaries at the top and not at the bottom? Do we really think there's a USA based full time developer with 10 years of experiencing making only $135 a year (not $135,000 but $135. He probably assumed it would be interpreted as thousands). Commented Jan 23, 2018 at 1:19
-
@SamuelNeff well for starters, we asked whether they were employed full-time at the time of filling out the survey as opposed to salary over the past year. So someone who had been unemployed for 364 days prior to taking the survey might have given an unusually low answer. And from a statistical perspective, the floor of zero presents less of a problem than a ceiling of 10^10 (or whatever number of digits we had to use to accomodate users who answered in Indonesian rupiah). Commented Feb 1, 2018 at 5:29
-
When would you compare salaries in dollars and ruplah as if they're the same thing? How does that make any sense? You'd need to convert them to equivalent values to compare, and even then it really isn't that useful to compare salaries across such disparate countries. Commented Feb 2, 2018 at 4:03
-
@samuelneff yes, we converted the values, based on the currency the respondent chose to answer in Commented Feb 2, 2018 at 4:51