From the methodology section itself, the survey included "101,592 software developers from 183 countries". Of these, only 67,441 people completed the entire survey. Some estimates put the number of software developers in the world at over 18 million. The number of people surveyed is just a small, small fraction of the number of global software developers.
Also consider that the users who took the survey were all self-selected and tended to be users of Stack Overflow, I'm not convinced that you can take any useful data out, if you want to learn about the broader software development community. It's probably useful to people who want to learn about the Stack Overflow community, however. It's very unlikely that the results are truly representative of the broader software development community because of the lack of rigorous sampling applied.
My biggest concern is that various outlets are picking up this annual survey and presenting it as a representation of the state of the software development community. This is absolutely not true, and I don't see any clear warnings on the survey results page itself about how to properly interpret the results. I don't want to say that this survey is bad, because it does offer interesting insights and can launch some deeper thinking and investigation. But from what I can see, the results are often being used or presented improperly by third parties.
I don't think you can stop everyone from cherry-picking results or twisting data around, and I don't think it's even fair to ask that. However, I do think that there should be more information at the beginning of the survey results page that talks about the limitations of the survey.
To be very clear and have a concrete proposal, I'm asking for some minor changes to be made to the presentation of the results. One way to do this would be to make minor changes to the Overview section:
Each year, we ask the Stack Overflow community about everything from their favorite technologies to their job preferences. This year marks the eighth year we’ve published our Annual Developer Survey results—with the largest number of respondents yet. Over 100,000 developers opted to take the 30-minute survey this past January.
{{ With some data analysis on visitor demographics vs survey results, perhaps say how representative this is of the whole SO community. If there are any recent enough other surveys, talk about how representative this survey is of the global population. Be clear: Does this survey accurately represent the SO user base? How well does it represent the global population of software developers? Be sure to cite the estimates 18-23 million global software developers (or more recent surveys, if any are available) as well as the number of active Stack Overflow users during the survey timeframe. }}
This year, we covered a few new topics ranging from artificial intelligence to ethics in coding. Here are a few of the top takeaways from this year’s results:
DevOps and machine learning are important trends in the software industry today. Languages and frameworks associated with these kinds of works are on the rise, and developers working in these areas command the highest salaries.
Only tiny fractions of developers say that they would write unethical code or that they have no obligation to consider the ethical implications of code, but beyond that, respondents see a lot of ethical gray. Developers are not sure how they would report ethical problems, and have differing ideas about who ultimately is responsible for unethical code.
Developers are overall optimistic about the possibilities that artificial intelligence offers, but are not in agreement about what the dangers of AI are.
Python has risen in the ranks of programming languages on our survey, surpassing C# in popularity this year, much like it surpassed PHP last year.
When assessing a prospective job, different kinds of developers apply different sets of priorities. Women say their highest priorities are company culture and opportunities for professional development, while men say their highest priorities are compensation and working with specific technologies.
Want to dive into the results yourself? In a few weeks, we’ll make the anonymized results of the survey available for download under the Open Database License (ODbL). We look forward to seeing what you find!