In this years Developer survey, several of the graphs have the option Weighted by Gender. What is this weight and how is it applied? Obviously the results between the two are similar. but there are some changes in percent. I could not find anywhere in methodology about how this was calculated, but I feel it is worthwhile knowing.
-
insights.stackoverflow.com/survey/…– πάντα ῥεῖCommented Apr 9, 2019 at 17:44
-
11This section explains it a bit, but it doesn't have the exact numbers for the weights they used– user5664615Commented Apr 9, 2019 at 18:33
-
1@osdavison That explains it pretty well. I saw the title and when straight to methodology to see if it was explained. I somehow missed the explanation right under the graph!– Ben MohorcCommented Apr 9, 2019 at 18:50
1 Answer
I'm the data scientist who worked on this survey, and I can add a little more context. I saw that after you asked the question, you saw the more detailed explanation we have about survey weighting, and what it is:
Survey weighting is an approach used to analyze survey data when the survey sample doesn't match the underlying population well. For example, in our survey this year, 11% of US respondents identify as women, but data from the US Bureau of Labor Statistics estimates that women's participation in the software developer workforce is about twice that, more like 20%. We can use survey weighting to adjust for the mismatch between our survey sample and the population of developers.
[....]
When we use weighting, we see small increases in the representation of developer roles that have the most representation from women, like designers, and decreases in others with low representation from women, like DevOps.
Given the proportions involved (just about 11% of US respondents identified as gender minorities, and the US Bureau of Labor Statistics estimates that more like 20% of software developers are women), the actual weight is close to 2. So that means that a response from a woman or non-binary respondent (we assumed we undersampled all gender minorities about the same) was weighted about twice that of a man, for those weighted US results.
If you want to look more at the (quite simple) math involved, I like this explanation, with some examples.
-
22Shouldn't this result in men's result being weighted down. That is, imagining that we have 90% of men and 10% of women answering for a population of 80%/20%, I would expect the following weights to be applied: 8/9 for men and 2 for women. This way, if we had 9 votes from men and 1 vote from women (total 10) afterwards we have 8 virtual votes from men and 2 virtual votes from men, for the same total. Commented Apr 10, 2019 at 12:45
-
5Thank you for the good explanation. Since anything gender related in the survey has the potential to become controversial, I appreciate the straight forward answer. Commented Apr 10, 2019 at 13:41
-
11Controversial indeed. Can we separate on religion and/or level of education too? Let's get some real controversy! Separate on hair colour? Self-identified hair colour? In all honesty though, I think calling women a "gender minority" is the most controversial thing said here, there are actually more women in the US than men. I suppose this is concerning SO users specifically though, still. I also don't see where "Software Developer" is listed on your survey - So maybe the "bureau" have different boundaries for their classification of what a software developer is? That may skew results. Commented Apr 10, 2019 at 13:59
-
@DanRayson As there is both result unweighted and weighted, I don't see the problem, MORE STATS. However, I don't care of these stats I don't judge people by sexe, but stats can't hurt ! Commented Apr 10, 2019 at 15:30
-
11
gender minorities
is a thing now? Making everyone a victim it really not the way of improving \ fixing anything (written by a "racial minority" according to your definition.)– gdoronCommented Apr 11, 2019 at 7:46 -
9@Stargateur As there isn't a set of primary categories, it's only gender that's been singled out, it's obvious that this is a politically motivated view on the data. I'd rather that was left out of SO. Commented Apr 11, 2019 at 11:39
-
12Women are a gender minority in the tech industry. You people. Commented Apr 11, 2019 at 16:53
-
4Really? You all think that classifying 20% of a sample as a minority is controversial? You must be kidding. Here's a definition of the term "minority" for you: (Noun) The smaller in number of two groups forming a whole. Commented Apr 11, 2019 at 17:06
-
10All this obsessing over gender. Honestly. Commented Apr 11, 2019 at 17:15
-
There's a simple litmus test. Do research on "how many women became active SO/SE users influenced by new SE initiatives aimed at upping female participation" (as opposed to, "because SE added emails" as the Dev survey results ironically hint as :). While at it, "how many women became active software developers explicitly influenced by new * initiatives aimed at upping female participation". Then, honestly decide if you're doing this to feel self-virtuous, or to actually achieve a real measurable positive result.– DVKCommented Jun 1, 2020 at 14:45