256
votes

[Edit: The survey is now closed]

It’s that time of year again—the annual developer survey is now open!

The survey will be open for three weeks starting today, and will tentatively close on January 26th.

As in previous years, anonymized results of the survey will be made publicly available under the Open Database License. We encourage you to download and analyze the dataset yourself when it becomes available.

On that note, throughout the survey, certain answers you and your peers give will be treated as personally identifiable information, and therefore kept out of the anonymized results file. We'll call out each of those in the survey with a note saying "This information will be kept private."

We've made this year's survey short (and by short we mean somewhat-freaking-long), but we've done our absolute best to be respectful of the time you're giving by taking the survey. If you have any questions or feedback in general, we'd love to hear it, just post an answer or comment as you deem appropriate. Thanks again to everyone, and happy new year!

Take the Survey Now

76
  • 35
    I'm looking forward to seeing the results again this year. P.S I liked the section on asking us if we're worried about Skynet.
    – George
    Commented Jan 8, 2018 at 14:10
  • 140
    You lost me at "should take about 30 minutes"... is this survey aimed at the unemployed?
    – musefan
    Commented Jan 8, 2018 at 14:23
  • 139
    There really should be more frequently a "I have no idea how to answer that" especially when the question is US culture oriented or any other reason the question feels very alien to your activity. Commented Jan 8, 2018 at 14:39
  • 30
    “How many monitors are set up at your workstation?” – At home, I have a very large monitor that is bigger than my multi-monitor setup at work. So this question does not really tell much about the work environment. It should rather ask for the total pixel space or something.
    – poke
    Commented Jan 8, 2018 at 14:49
  • 37
    Have you considered doing testing with external users before going live, instead of only doing in-house testing? Every year multiple issues are reported shortly after the survey going live, it's unfortunate this mistake is made every time.
    – user247702
    Commented Jan 8, 2018 at 15:21
  • 193
    Why were there so many questions about advertisement? Is this whole survey thing just a cover for you to evaluate how to improve your advertising revenue? Because that's exactly how it felt after 3 pages of asking about my advertising preferences and habits...
    – musefan
    Commented Jan 8, 2018 at 15:26
  • 72
    Next year, please do some QA including non-US testers before releasing the survey. I was struggling with a lot of questions. Unfortunately, I can no longer view the survey, because I have already finished it. So I can't provide more detailed feedback :(
    – honk
    Commented Jan 8, 2018 at 15:38
  • 53
    No opportunity to get swag by guessing coins, M&M's or whatnot? Terrible tradition-breaker :-( Commented Jan 8, 2018 at 16:31
  • 45
    The primary purpose of the survey has always been about improving our ability to serve relevant advertising, @musefan. We've expanded it considerably over the past 7 years to try to make the results more generally useful and interesting, and last year's results seem to have been widely successful in achieving that secondary goal, but that core need remains in place - IMHO, that's a far better bargain than the sorts of intrusive analytics (with secret results) that certain other sites use for this purpose.
    – Shog9
    Commented Jan 8, 2018 at 16:50
  • 55
    Why do you want to know my sexual preference?
    – Tim
    Commented Jan 8, 2018 at 17:51
  • 89
    You mean you're not on Stack Overflow Dating yet @TimCastelijns? Commented Jan 8, 2018 at 18:28
  • 83
    Missing Option: I use an adblocker for security. The reason I use an adblocker has nothing to do with how I feel about ads, and 100% because ad networks are malicious code vectors. Commented Jan 8, 2018 at 20:21
  • 49
    While this doesn't really apply to SO, the whole section on advertising is hopelessly skewed by leaving users unable to report what is most probably the #1 most hated thing about online advertising: it's frequently intrusive and extremely annoying. Does that mean it's not "trustworthy" or "honest about its goals"? I don't know, but if you make my speakers blare at 90dB uninvited I don't really care if I like your company or you avoid fluffy language. Commented Jan 8, 2018 at 22:03
  • 41
    Also to reiterate @JoelAnderson's point, adblockers are about security, period. There is no whitelisting or trusted sites or "oh that site's mainstream so they won't have malicious code". Companies have shown again and again they don't police their ad networks: twitter.com/SwiftOnSecurity/status/950088799014572034 Commented Jan 8, 2018 at 22:08
  • 44
    Gave up at the "rank these 10/11 items in order" questions: most I don't care about enough to rank, so the results would be random/meaningless. The old format, where you gave a ranking to only as many as you wanted was much better. If you must show off your "look, we can do drag-and-drop JS/CSS" skills, add an "I don't care below this entry" option that can be dragged to the appropriate place.
    – TripeHound
    Commented Jan 9, 2018 at 8:25

46 Answers 46

187
votes

All these rank questions don't really work. On every one there are a bunch I care about and a bunch I don't. Forcing me to give a full ranking will give you largely random data. One person's rank-2 may mean "very important" while another person's may mean "completely irrelevant".

You should have used the standard 5-point scale rating for each one. If you wanted more relative detail, then e.g. a 100-point slider (with the five labels in appropriate positions) should work.

14
  • Interesting point. I wonder if there is literature on this topic proving what you intuitively assert. Commented Jan 8, 2018 at 16:52
  • 50
    Also, I there was no option like "I don't want to answer". I guess the default order will appear in the result set most often...
    – honk
    Commented Jan 8, 2018 at 17:10
  • 3
    @honk the numbers are invisible until you start moving around, so I guess it at least knows if you've skipped it entirely. Related
    – OrangeDog
    Commented Jan 8, 2018 at 17:17
  • 2
    @OrangeDog: And what if for a user the default order is fine and the user wants it to be considered and continues without moving any item?
    – honk
    Commented Jan 8, 2018 at 17:20
  • 11
    @honk as I say, the methodology for these questions is completely broken anyway, so it's a moot point
    – OrangeDog
    Commented Jan 8, 2018 at 17:22
  • @OrangeDog: OK, I agree :)
    – honk
    Commented Jan 8, 2018 at 17:26
  • 16
    I'd give this 100 upvotes. For most questions, whatever after the 6th option, for me, was "I don't care". Commented Jan 8, 2018 at 18:14
  • 1
    @BlackVegetable Yes, in fact there is. :) When I was taking survey methods we read a lot by Dillman and a couple papers that were critical of ranking based questions.
    – rjzii
    Commented Jan 8, 2018 at 21:24
  • 9
    Completely agree with this. I certainly felt like I was over-inflating some of the lower choices when I truly couldn't care less about any of them. This one could have been fixed by assigning each one a score, or if you really wanted to get clever, give 100 points to distribute across the options and let the user decide how many to assign each one. Too late now I guess.
    – DavidG
    Commented Jan 9, 2018 at 1:07
  • 2
    I gave up the survey at this point for the same reason OrangeDog says: there are probably two or three options I'd care enough about to rank but the rest would be random. If you must keep the drag-and-drop format (has someone found a shiny new JS library?), add a "I don't care below this entry" option that can be dragged to the appropriate place.
    – TripeHound
    Commented Jan 9, 2018 at 8:22
  • 2
    You need to vary the way of information retrieval because as much as it might be confusing a sequence of 30 agree/disagree batteries might scare away more people (and you run into more inter-item correlations like tendencies to the middle, left or right - you have to admit that you had to think a bit for this one - and that's what it's for and why it's necessary. An "I don't know" and "I don't want to answer option has to be added to every question in professional survey. This requirement is unrelated to the form of the item. Commented Jan 10, 2018 at 0:50
  • I left that window with default ranking. I can't spend 10 min to figure out how to arrange 10 items which may or may not be related to me Commented Jan 10, 2018 at 13:20
  • Those were most annoying for me too, but all noise can (and should) be eliminated by statistical analysis. You can tell, with a large enough sample, if there is or there isn't a definitive tendency towards something, and of course responses won't be 100% correct, but again a large sample should be consistent enough (if you feel two are equal, you'll put them 1 after 2, and somebody else 2 after 1). Also, I think it's sane to assume default order is randomized so to prevent any bias (actually, it'd be pretty insane not to randomize default order).
    – Luke
    Commented Jan 10, 2018 at 17:50
  • 1
    These rankings made me upset too. 1/ Weird default look w/o visible weights of individual answers; 2/ pain to fill - no sane way to input these using a keyboard 3/ missing option to throw an answer out from the ranking
    – helvete
    Commented Jan 22, 2018 at 13:08
184
votes

We added both Oracle & sqlite to the question.

Please add Oracle to the list of "databases you have worked with in the last year".

Oracle is far more prevalently used than most of the databases listed.


As per @Sklivvz's comment, please add sqlite too.

28
  • 71
    Also: sqlite is very used in desktop apps
    – Sklivvz
    Commented Jan 8, 2018 at 16:22
  • 13
    @Masiama But that could be said of all options. Why not just have an entry field? No. Oracle is very widely used, especially in older organizations, where it is possibly the most commonly used database. Either remove 80% of the options, or add in Oracle... nothing else makes sense.
    – Bohemian Mod
    Commented Jan 8, 2018 at 19:05
  • 4
    I think the reason is they didn't want an equivalent of "Tick here if you've ever been a member of a terrorist organisation." What if you select it by accident - who knows what'd happen?
    – Rob Grant
    Commented Jan 8, 2018 at 19:58
  • 2
    @Bruno "Oracle" is close enough. There's no value is breaking down flavours. In case you live in an Oracle induced superiority bubble, most databases offer cloud, clustered, distributed and similar flavours.
    – Bohemian Mod
    Commented Jan 8, 2018 at 22:34
  • 2
    The proper name to be added to the database section is Oracle Database. However, there was a section that included AWS and Azure (I think) but omitted Oracle Cloud.
    – Dan McGhan
    Commented Jan 8, 2018 at 22:44
  • 21
    (full disclosure, I work for Oracle) The fact that you managed to build an IT survey without managing to mention Oracle even once suggests more than just accidental omission but malicious bias. That's pretty poor form for StackOverflow. Commented Jan 9, 2018 at 0:20
  • 3
    Most of the internetz eg this site or this site says that Oracle is #1 used in the world. WTF did stack overflow not include it? Commented Jan 9, 2018 at 0:41
  • 3
    @JumpingJezza "#1 used in the world" depends on how you measure it. I guarantee that there are more instances of mysql than oracle (due to oracle's draconian per-CPU fee structure). Google for example extensively uses the LAMP stack and they run gazzilions of servers.
    – Bohemian Mod
    Commented Jan 9, 2018 at 7:25
  • 1
    @Bohemian, "Oracle" is a company. "Oracle Database" is a technology, and so is "Oracle Cloud". That's why you see in the list "AWS", "Azure", "Amazon Echo", "Firebase", "Google Cloud", "Windows Desktop"... And not just "Amazon", or "Microsoft", or "Google". So yes, "Oracle Cloud" and "Oracle Database" should have been included (each into its related section). Commented Jan 9, 2018 at 7:31
  • 1
    @dev.mi Except it calls into play the competence of the people that made the survey when they miss obvious things like Oracle. If you want accurate survey data you want to minimize respondent burden as much as possible.
    – rjzii
    Commented Jan 9, 2018 at 15:05
  • 13
    So as of this morning, Oracle is ON THE SURVEY. There was an error on our part and we apologize. It was absolutely not purposeful or malicious, especially since Oracle is a client of ours. I am going to be handling the analysis of this question carefully (looking at the responses from the 1st day separately from the rest) to be as accurate as possible.
    – Julia Silge Staff
    Commented Jan 9, 2018 at 17:15
  • 11
    @Julia Rather than let errors like this through to production, next time why not have the community review such lists before releasing survey? There is an army of SO users who would donate their time to review questions - you should use this resource.
    – Bohemian Mod
    Commented Jan 9, 2018 at 18:06
  • 13
    This is apparently news worthy. See theregister.co.uk/2018/01/09/…
    – Dijkgraaf
    Commented Jan 9, 2018 at 20:43
  • 2
    @Bohemian Pretesting of a survey instrument should always take place before it is sent, see "6 Ways to Pretest Your Survey Before You Send It"
    – rjzii
    Commented Jan 9, 2018 at 20:44
  • 2
    @Bohemian The problem is that the the results get picked up by the broader community and potentially by the lay media as well. Stack Exchange presents the findings in an authoritative way (see., insights.stackoverflow.com/survey/2017) and exposes the data set to allow other people to try and draw conclusions from it. If you are going to try and do something in an authoritative way (i.e., have data scientists analyze the data and present findings on it) it behooves you do make sure that what you are doing can stand up to the scrutiny of other scientists.
    – rjzii
    Commented Jan 18, 2018 at 18:42
140
votes

On the assumption of the American job market:

I'm currently a Student in Germany. I already have completed a vocational education and worked for a bit before hitting University.

The following items in the "benefits package" are mandatory in Germany:

  • Parental Leave
  • Health insurance
  • Equipment Allowance, given a medical need for it
  • Retirement savings matching (for the enforced minimum retirement savings)

It's nearly impossible to rank benefits that are mandated by law... Results from Germany (and quite probably a lot of European countries) will accordingly be skewed.

If you have a way to fix the question, that's just fine as well.

Ambiguous Framework name(s?):

In the section "frameworks, libraries and tools have you done extensive development work in [..]", I assume that "Spark" refers to "Apache Spark" and not "Spark-Java", the first being a big-data framework and the latter being a simple REST-Server framework

Overall I like that there's more options to amend missing answer options.

40
  • 24
    Given the benefits, you can still rank them by personal preference even if they're mandatory.
    – George
    Commented Jan 8, 2018 at 14:40
  • 15
    They're still either important to you or not, aren't they?
    – ivarni
    Commented Jan 8, 2018 at 14:46
  • 27
    What about if for example you do not have children, so you put "Parental Leave" as least important as it's "not applicable". Surely that would skew results?
    – musefan
    Commented Jan 8, 2018 at 14:53
  • 3
    RE: the frameworks; I find the list to be strangely small anyway. I guess it is based on trending tags, but what is hoped to be learned from the question I do not know...
    – Gimby
    Commented Jan 8, 2018 at 15:55
  • 6
    @ivarni try ranking whats more important: grocery stores or gas stations. Sure they are important, but I am not able to usefully prioritize things that are 'just there' since I before I could think
    – Vogel612
    Commented Jan 8, 2018 at 15:57
  • 22
    @ivarni You cannot rank them according to how it would be without them though since the perception is very different. To me, health insurance is a guaranteed thing. I actually have a hard time imagining being without it, and I would probably never take a job without having health insurance (assuming I take work outside of Germany). So I would personally rank this #1 but this is mostly because I am used to this as being a default and because being in a situation without it is unthinkable to me. But that thinking does not apply to other countries where health insurance is not a default.
    – poke
    Commented Jan 8, 2018 at 16:14
  • 2
    I'm not familiar with Germany, but do some companies go above and beyond the minimum needed by law? If so, the question still makes sense. If going above and beyond the law is important to you, then it should be ranked higher. If you're OK with the minimum required, then it would be ranked lower. Commented Jan 8, 2018 at 16:18
  • 28
    s/Germany/EU -- also to the point: if they are required by law they are not benefits. Your right to strike is protected by law, it's not a benefit.
    – Sklivvz
    Commented Jan 8, 2018 at 16:21
  • 4
    Thanks for the thoughts here. International differences in workplace culture are something we always try to think through when writing survey questions for our worldwide audience of developers. Our goal is to not to have many questions that only get shown to developers in certain countries, so we try to write these to apply broadly but in ways that hopefully most/many can interpret, as some of these other commenters did.
    – Julia Silge Staff
    Commented Jan 8, 2018 at 16:30
  • 14
    You have a good point. I'm also EU and took the survey just now. Almost without thinking I've put the "guaranteed by law" items at the end of my priorities. It's not that they are not important, it's just that they are, well... guaranteed. I do feel, by intuition, and by quite recent experience there will be some bias.
    – armatita
    Commented Jan 8, 2018 at 16:30
  • 4
    @Mixxiphoid Some things may not change. But for other things, like parental leave, if the minimum is 6 months, there's nothing stopping a company from offering 9 or 12 months or some other perks related to parental leave (the minimum time off plus additional time of flex work or something). If you were planning on having a child, a company that is extremely flexible with parental leave could be a game changing decision. But again - I don't know how it works in Europe. Maybe very few if any companies offer more than the minimum. Commented Jan 8, 2018 at 18:41
  • 2
    Australia also has a minimum percentage of the salary that all (but the smallest) employers must contribute to a retirement fund.
    – Bohemian Mod
    Commented Jan 8, 2018 at 19:10
  • 29
    This annoyed me too, especially since I have pointed out the same issue previously, on earlier surveys. In the majority of the counties in the western world, "health insurance" etc are covered by the government, not by private companies. USA stands out as an exception here, lagging behind the rest of the western world by 50 years or so.
    – Lundin
    Commented Jan 9, 2018 at 7:32
  • 15
    @armatita The root of all problems in these surveys have always been "lets assume that every company everywhere in the world works just like the SO office in USA".
    – Lundin
    Commented Jan 9, 2018 at 9:22
  • 6
    @ThomasOwens maybe you don't understand ( not being pedant) that the countries that enforce it by law mostly if not all already provide full coverage. in my country there is nothing beyond that, except maybe "quicker response times" when you have a nonimportant illness and you need quick medication, else, you're all covered. got cancer? treated. got back pain? treated. cut a leg? treated. have insomnia? treated. private & public severe operations are performed on the public centers, you get full treatment, medicines are either free or almost free. there's no way to rank that comparatively.
    – CptEric
    Commented Jan 10, 2018 at 7:15
114
votes

The styling of the radio buttons is needlessly unconventional, and it's confusing:

enter image description here

It's a pretty universal convention that a radio button is marked as selected by putting a dot inside it. This styling makes it look like the radio button is merely focused, rather than selected. I actually aborted and restarted the survey after rapidly clicking through the first page because I thought I hadn't entered an answer on the last question.

Can we have a dot? There was no reason to mess with this.

4
  • 26
    Yes, the "focussed and checked" style confused me too.
    – DavidG
    Commented Jan 9, 2018 at 1:08
  • 4
    Adding to the fire is the fact the lit up effect dies down upon clicking another button
    – Passer By
    Commented Jan 9, 2018 at 2:35
  • 1
    From a developer point-of-view: The background of the focus style is overriding the background of the selected style. That’s why the filling feels so terrible. If you remove the background from the focus, everything makes a lot more sense.
    – poke
    Commented Jan 9, 2018 at 8:06
  • 6
    Re: "There was no reason to mess with this." I think it's clear there is always a reason to mess with older paradigms, which is: there's something newer. Sometimes the newer thing is better from a functional standpoint, other times it is not. Some organizations may feel like they have to use the newest thing, for no other reason than to avoid looking out of date.
    – John Y
    Commented Jan 9, 2018 at 22:31
101
votes

Not trying to be rude here, but I think answering 'I identify as an autistic person' is too much over the top. This actually implies people choose to be autistic, or just identify as those. Autism and or a disorder in the related spectrum is a mental condition, not a social construct.

Autism Speaks website:

Autism, or autism spectrum disorder, refers to a range of conditions characterized by challenges with social skills, repetitive behaviors, speech and nonverbal communication, as well as by unique strengths and differences. We now know that there is not one autism but many types, caused by different combinations of genetic and environmental influences.

You might assume you have some sort of, or symptoms implying autism, but identifying sounds just plain wrong. I'd like to suggest one of the following:

  • I’ve been diagnosed with a disorder in the autism spectrum
  • I believe to have a disorder or some symptoms of a disorder in the autism spectrum

Not forfeiting the rest of the options.

19
  • 51
    I found that option quite odd. I thought it was a diagnosed disorder, not some trendy thing to "identify" as.
    – ohmu
    Commented Jan 8, 2018 at 17:35
  • 13
    Yeah, "I am autistic", or perhaps "I have autism", to fit with the "I have..." format of the other answers, would flow much more naturally. The attempt at political correctness here - which, as you say, carries with it the implication that what is in fact a very real (and sometimes life-ruining) mental disorder is instead some kind of subjective personal label - ends up being, as it often does, significantly more distasteful than plain English would have been.
    – Mark Amery
    Commented Jan 9, 2018 at 0:36
  • 11
    "I'm on the autism spectrum" might be the best choice.
    – Joe Friend StaffMod
    Commented Jan 9, 2018 at 2:56
  • 2
    I would propose either ‘I’ve been diagnosed with a disorder in the autism spectrum’ and ‘I believe to have a disorder or some symptoms of a disorder in the autism spectrum’
    – roberrrt-s
    Commented Jan 9, 2018 at 7:18
  • 20
    Another question about gender had 5 choices. The survey is suffering heavily from "moral panic", trying to be politically correct to the point of silliness. That's what we see in this question too.
    – Lundin
    Commented Jan 9, 2018 at 7:47
  • 10
    Alas, it's the decade of diversity -- so you can identify as whatever you wish, disregarding any reality or biology-based criteria.
    – ACEG
    Commented Jan 9, 2018 at 10:49
  • 3
    @Lundin I disagree that the two are equivalent. The non-binary gender options - unlike this weird wording - actually allow the survey to gather information that it otherwise wouldn't. If I remember correctly, looking at the results last year I found that among survey respondents there was 1 male-to-female trans person per 10 non-trans females, compared to a ratio of maybe around 1 to 200 in the general population (based on Googling for US stats). That result, IMO, is interesting in its own right and a useful piece of evidence in the ongoing debate over why more women don't become programmers.
    – Mark Amery
    Commented Jan 9, 2018 at 11:47
  • 4
    @MarkAmery That may be because the survey of the general population was made in scientific ways, while the SO survey is more of "lets ask some fun questions to people who were not randomly chosen at all".
    – Lundin
    Commented Jan 9, 2018 at 11:52
  • 5
    @Lundin I'd be more concerned about reptilian Muslim climatologists from Mars sabotaging the data. If the explanation is selection bias, as you say, then that's still an interesting result in its own right, it's just that the explanation is nothing to do with programming and is instead "transsexuals innately drawn to surveys: compulsively fill them in at rate 20x the rest of the population!". I kinda doubt it, though.
    – Mark Amery
    Commented Jan 9, 2018 at 11:56
  • 8
    @MarkAmery Rather it would be something like that users of SO are more likely to belong to a generation where you would openly admit being transsexual. Or much more likely: there isn't enough survey participants to draw any form of conclusions from the survey what-so-ever. One perfect example is Stack Overflow Trends, where some would-be scientist made the conclusion that certain technologies were becoming less popular just because the percentage of questions on SO of those technologies had been in decline. Even though the amount of questions had actually increased. Dumb things like that.
    – Lundin
    Commented Jan 9, 2018 at 12:05
  • 4
    @Lundin Indeed, just because of that I was very tempted to write "I identify as a red fox" instead of choosing male/female/etc..
    – Izkata
    Commented Jan 10, 2018 at 16:11
  • 4
    @BartoszKP My understanding is that people who don't have any degree of the disorder aren't on the autism spectrum
    – Joe Friend StaffMod
    Commented Jan 10, 2018 at 17:51
  • 3
    Good answer but Autism Speaks is an autism hate group
    – cat
    Commented Jan 13, 2018 at 18:35
  • 2
    I have seen the term "identify with autism" before, but it's not a good term. Either you believe you have an autism spectrum disorder, have been diagnosed somewhere on the spectrum, or you haven't. I'd say adding both the options is the best way to go, as there's still a difference between believing you have an autism spectrum disorder vs being formally diagnosed with it Commented Jan 18, 2018 at 21:12
  • 1
    Yeah, the other "identify" questions were like sure okay, but come on, identify as autistic?
    – Trophonix
    Commented Jan 19, 2018 at 0:06
89
votes

Some of the same problems that have plagued old surveys still exist.


Approximately how many people are employed by the company or organization you work for?

My office is about 70. My subsidiary is about 400 or so. My parent company is 50,000. How do I answer? Personally, I answered with the 10,000+ answer, since my paycheck has the parent company's name on it.

I'm also surprised that there's no question about development team size. Although the parent company is 50,000 strong, the development organization is much, much smaller. Even if you consider just my local office (which is a development hub), only about 50% of the office is development.


Which of the following methodologies do you have experience working in?

One of the options is "Formal standard such as ISO 9001 or IEEE 12207 (aka "waterfall" methodologies)". As someone who has worked in aerospace (AS 9100, which is an industry-specific variant of ISO 9001) and pharmaceutical software (based in ISO 9001), I can tell you with absolute certainty that you can deploy the agile methods and still be compliant with ISO 9001 (and likely most, if not all, of the industry specific variants, like AS 9100).

Some of the other options also don't make much sense. Agile and Scrum are both options? Scrum is an implementation of the agile methods (along with Extreme Programming and a number of others). Pair programming is a technique, not a method.


Do you report or otherwise call out the unethical code in question?

The answers to this question are incomplete.

If one were to refer to the Software Engineering Code of Ethics and Professional Practice, the first priority of software engineers (people "those who contribute by direct participation or by teaching, to the analysis, specification, design, development, certification, maintenance and testing of software systems") is to the general public. However, one also has obligations to the client and the employer. The first step would be to report it internally, in order to avoid public disclosure of confidential information (unless there was an immediate danger to the general public). Then, escalating the disclosure as appropriate, perhaps even to the general public - depending on the situation, it may be ethical to disclose confidential information for the greater good of society.

These questions on ethics are not really nuanced enough to get into any kind of meaningful discussion. I answered "Yes, but only within the company" since it's the correct first step based on a number of ethical codes written for computing professionals (including the one I linked to). But it doesn't have to be the last step, and this question doesn't lead to that.


The fact that the survey still targets "people who write code" leaves a lot to be desired. That's the primary audience of Stack Overflow. But there are plenty of non-programmers involved in developing software. I would like to see this improved in the future with surveys that are inclusive of people who are involved in analysis and specification (product managers, business analysts), architecture (architects), testing (manual testers, automated testers), management (development leads, development managers), and process improvement (Scrum Master, agile coach).

I don't know what the inclusion of these other types of roles would look like, but this survey is advertised on the entire network via the blog post. It would be nice to see questions geared toward the software developers that participate on Software Engineering, Software Quality Assurance, Project Management, and maybe a few other sites because those areas are their expertise would be very nice.

5
  • 6
    In addition, ISO 9001 is hardly "formal" as quality management standards go. It's a silly little document and can't be used for software quality by itself. If someone says "our organization writes software according to ISO 9001", then you know for certain they haven't got a clue about what they are doing.
    – Lundin
    Commented Jan 9, 2018 at 7:40
  • @Lundin It's formal in the sense that it's a recognized standard with audits and a database of parties that are certified. You're right that it doesn't assure product quality - you can make garbage products with well-defined and improved processes. But it's totally possible to build a software product using a process that is compliant with ISO 9001 - I've written the necessary documentation to bring an agile process through the audit process. Commented Jan 9, 2018 at 9:49
  • 2
    ISO 9001 is some 30 pages of very vague text. You can make anything compliant to very vague text. I could easily ISO certificate my cat. Anyway, my point here is that ISO 9001 simply does not give the tools needed to define a software design process or improve its quality. It is the very same thing as putting up a sign of the door to your office saying "do not write bugs".
    – Lundin
    Commented Jan 9, 2018 at 10:10
  • 2
    I would totally buy a t-shirt with "I just ISO certified my cat." printed on it. Commented Jan 10, 2018 at 0:08
  • I felt the same way about the ethical questions. All of the answers seemed somewhat valid, just at different steps in the process. I put the same answer as you for the same reasoning, though I hadn't actually heard of that source you provided. It just seemed like common sense.
    – Trophonix
    Commented Jan 19, 2018 at 0:09
70
votes

The census badge description says "you have given your profile URL" but, in reality the current profile takes it without asking by passing the survey answer id to my current page (and thus, implicitly knowing who I am by sending me to a registered only page on the site).

It's not nice to do this without warning and it's actually a (well intentioned) lie to claim it's otherwise in the description. Please at least fix.

4
  • 4
    See here, although the post doesn't mention the implications of that.
    – user247702
    Commented Jan 8, 2018 at 16:34
  • 3
    I took the survey when I wasn't logged in to SO, so no badge for me... unless I take it again...
    – user812786
    Commented Jan 9, 2018 at 15:21
  • 25
    I took what I thought was an anonymous survey, and only at the end realized that everything I put is going to be forever linked irrevocably to my SO profile.
    – John
    Commented Jan 9, 2018 at 19:19
  • I could care less about my SO profile being linked to this survey, so I was infinitesimally peeved that after inputting the email address associated with my profile (after taking the survey otherwise anonymously and not signed in via my employer's machine), that I didn't have a badge of completion waiting for me when I signed in. Commented Jan 24, 2018 at 13:51
66
votes

There are design problems with survey (don't remember how it was last year).

Namely I am not obliged to select anything, but if I select and then change my mind - there is no option to "unselect".

E.g. a question regarding when did I update my CV last time. Sorry, I don't even have one, so I didn't tick anything, but if I would tick one of radio buttons I'd be unable to remove it.

While some other questions have 1 or 2 options to pass: "I prefer not to answer" or custom reason (possibly blank).

P.S.: and there is no funny part this year =(

1
  • 2
    I almost fell into this one as well; I don't have a CV, and my resume is unchanged from when I first wrote it (haven't had a need so far). No answers fit, and it would be even worse for a student who has yet to even write a resume.
    – Izkata
    Commented Jan 10, 2018 at 16:16
65
votes

Not a fan of the "rank these statements" type questions. I choose to participate in a survey in order to quickly give honest feedback for research purposes, but these felt more like an exercise and I genuinely dreaded them when they popped up.

Say I genuinely don't care about 3 of the options, I can't give them an equal ranking even though I equally don't care about them. So now I feel like I'm implying that I partially care about a couple of those options.

Similarly, I may have a couple of options that are of equal significance to me - I have no way to display that.

Going forward, I think phrasing the question in the way of "which of the following x do you find important enough to impact your decision on y ?" and a list of check-boxes will suffice, and actually provide better data. You could then follow up with an optional "which of these would you consider the most/least influential" (you get the idea...)


TL;DR

My brain is taxed enough trying to support a legacy WebClass application and the primal needs of my offspring. Now you want me to prioritise options on a made up job from a recruiter I'll never talk to? Urgh...

4
  • 9
    Totally agree. I gave up at the "rank these 10/11 things" questions. I'm even more agnostic than you: there's probably only two or three I'd care enough about to rank at all.
    – TripeHound
    Commented Jan 9, 2018 at 8:18
  • Already discussed in meta.stackoverflow.com/a/361614/1797006 Commented Jan 10, 2018 at 0:55
  • @KarlRichter yes but I wanted to give my alternative for the question layout with the context behind it. So I posted an answer. Commented Jan 10, 2018 at 0:59
  • At the very least, there should be an option not to answer (e.g. a checkbox that says "I prefer not to answer" or something along the lines of that). Though having the option to rank multiple on the same rank (and the ability to say a specific option isn't important at all/not applicable/whatever else would also be great to have) Commented Jan 18, 2018 at 21:29
49
votes

We added both Taiwan and "Other Country (Not Listed Above)" to the survey.

I can't even start the survey because my country of residence - Taiwan - is not present in the list. It's not there under any of the common names either (R.O.C., or the silly Chinese Taipei).

Please don't exclude a nation of over 20 million people from your survey.

6
  • This year, we used a standardized list of countries from our survey tool, Qualtrics. I know that there are some countries with complex identities. :/
    – Julia Silge Staff
    Commented Jan 8, 2018 at 21:07
  • 5
    I would at least expect the undignified "Taiwan, province of China" then, but even that's missing. I also noticed Hong Kong (SAR) is included but Macau (SAR) is not.
    – Anonymous
    Commented Jan 8, 2018 at 21:21
  • 5
    Also Bermuda...we may not have 20 million residents, but it sucks to not be able to continue with a Survey when it asks for geographical information based on the option I had to choose (UK)
    – brimstone
    Commented Jan 8, 2018 at 22:14
  • 1
    I face same problem on 2013. meta.stackexchange.com/questions/158543/…
    – 4b0
    Commented Jan 9, 2018 at 10:38
  • 7
    Taiwan is now on the list of countries! 🇹🇼 We apologize for our error in not including this from the beginning. Also, there is now an option of "Other Country (Not Listed Above)" to be able to include lots of small countries.
    – Julia Silge Staff
    Commented Jan 9, 2018 at 17:13
  • P.R. China will soon file a complaint that Taiwan shouldn't be a country :)
    – iBug
    Commented Jan 19, 2018 at 2:12
45
votes

We added "None of the above" to the questions about disability.

Add "Not disabled" to the list of disability options.

This allows the important distinction between a person that has a disability but selects "Prefer not to answer" and someone who doesn't have a disability who wants to answer such.

Currently, not answering at all and "prefer not to answer" don't allow collection of data that someone is definitely not disabled.

Same goes for the "behavioural problems" question.

5
  • 1
    Thanks for bringing this up; we have edited these questions to clarify them and will be analyzing the data so that leaving it blank is equivalent to "Not disabled" and "Prefer not to answer". This question will be kept private (not part of the public data release) and any data analysis will only use positive responses, i.e. respondents who identified with a difference/disability.
    – Julia Silge Staff
    Commented Jan 8, 2018 at 16:35
  • @JuliaSilge Wouldn't that approach introduce a lot of error into the analysis?
    – rjzii
    Commented Jan 8, 2018 at 21:16
  • 3
    With this kind of survey where people are very free to leave answers blank, we don't expect to be able to make strong statements about what proportion of developers have differences/disabilities. Instead, what is more possible is to say something like "The most common disability reported is X. It is Y times as common as Z. People were A times more likely to report mental differences/disabilities than physical." Things like that.
    – Julia Silge Staff
    Commented Jan 8, 2018 at 21:31
  • @JuliaSilge To be fair, any survey that is subjected to IRB allows for non-response to questions and it's just something that we have to deal with in the design and analysis. Basing the analysis on just the positive responses makes sense, but I do question if those questions should have even been included.
    – rjzii
    Commented Jan 8, 2018 at 21:39
  • 2
    “The most common disability reported is X” – Weren’t the answers far too unspecific for that purpose?
    – poke
    Commented Jan 9, 2018 at 8:03
37
votes

It says "10 is least important". Then what is 11?

bug

7
  • 47
    These go to 11
    – ivarni
    Commented Jan 8, 2018 at 14:48
  • 5
    @ivarni But why not just make 10 less important?
    – George
    Commented Jan 8, 2018 at 14:52
  • 17
    Hah! Let's pretend that was a subtle Spinal Tap reference and not a mistake. :)
    – kristinalustig StaffMod
    Commented Jan 8, 2018 at 14:59
  • 1
    That's so minor. I didn't even noticed. The hint part of such questions was clearly copy/pasted. How would this create a problem to anyone?
    – Sinatr
    Commented Jan 8, 2018 at 15:25
  • 1
    Can I ask a practical question at this point? Are we gonna do Stonehenge tomorrow night?
    – Randy Levy
    Commented Jan 9, 2018 at 5:02
  • 1
    @RandyLevy By answering this question, you've already done a Stonehenge :)
    – iBug
    Commented Jan 11, 2018 at 5:37
  • Its 1 extra unimportant
    – Alkarin
    Commented Jan 19, 2018 at 1:14
37
votes

I saw Shog9's comment that the intent of this survey is primarily to acquire marketing data; however, from a methodological standpoint this survey is very flawed if you want to comment on developers as a broader population. Since the survey is already in progress you should not change it! However, a couple suggestions for next year:

  1. Disclose in the introduction that the intent of the survey is to gather data for marketing purposes.
  2. Hopefully you did this, but given some of the feedback on Meta it is possible you didn't, do a small test to make sure the questions and answers make sense. Going through the survey there were a couple places where the answers "nudged" the respondent a certain way, that is bad.
  3. Make sure questions are clear to minimize respondent burden, it should be clear in the question how someone should answer if they have "x.5 years of experience" (example).
  4. Similarly, define what you mean by terms like "contribute" - is submitting a bug report to an open source project a contribution, testing, documentation, etc?
  5. Don't repeat questions. I seem to recall that I was asked about my education at least twice for basic information and demographic information.
  6. Make better use of your platform. For example, the question about programming languages could have allowed for free-text that was validated against a database. A measure of niche languages that are used could have been done with that approach and it would prevent respondent fatigue or misentry if someone clicks "Rush" instead of "Ruby."
  7. Your use of a convenience sample means you cannot say this is "the largest and most comprehensive survey of software developers (and anyone else who codes!) on Earth." IRB would have my head for making a claim like that.
3
  • 8
    I wasn't asked twice about education, but I was asked once about my education, and then once about my parents'. Perhaps this is what you're (mis-)remembering in point 5? Commented Jan 8, 2018 at 23:22
  • @RoddyoftheFrozenPeas Possibly. Generally I don't take notes when doing survey's unless people are explicitly asking for feedback on the design.
    – rjzii
    Commented Jan 9, 2018 at 5:36
  • I agree with the first sentence about making it clear this helps marketing... and then there was all that other stuff which I kind of disagreed with but didn't take notes on.
    – Travis J
    Commented Jan 9, 2018 at 22:20
36
votes

We added "None of the above" to the questions about disability.

Can we fix these questions?

Which of the following describe you, if any? This information will be kept private. If you prefer not to answer, you may leave this question blank.

They have a "I prefer not to answer" answer.

Having both could skew results.

3
  • 20
    Hey! Person who wrote the survey, here. We decided to have both here so that it'd be abundantly clear that the question is optional, and to make people as comfortable as possible with possibly sensitive questions. They won't be treated any differently in the data.
    – kristinalustig StaffMod
    Commented Jan 8, 2018 at 14:58
  • 5
    @KristinaLustig i felt that none applied to me which is different than "I prefer not to answer" Commented Jan 8, 2018 at 15:16
  • 5
    Totally - we're going to add in a "none of the above" option. It's a good point that someone else made in a different thread. Thanks for bringing it up.
    – kristinalustig StaffMod
    Commented Jan 8, 2018 at 15:18
31
votes

At the question: Think back to the last time you updated your resumé, CV, or an online profile on a job site. What is the main reason that you did so?

The question assumes everybody updates it CV because they need a (different) job. My employer wants me to update my CV regularly because I'm a consultant and when going on a new assignment the customer could ask for it. None of the options matched this IMO.

At the question: What is your current gross salary (before taxes and deductions), in {currency} I assume the salary question, asks for the yearly gross? Not really clear because it also says "If you are paid hourly, please estimate an equivalent weekly, monthly, or yearly salary. "

At the question: Which of the following tools do you use to communicate, coordinate, or share knowledge with your coworkers? Please select all that apply. Where is TFS? :( I mean, Trello is in there, shouldn't TFS be in there as well?

At the question: What do you do for childcare or dependent care while you are working?
Uhm... nothing? My children do not need to go to childcare, I never said they go there.

Please stop assuming so much in the survey, I skipped the survey last year because I got so irritated of all the assumption. I understand that is my problem, but still...

9
  • 45
    Apparently "email" is a thing of the past for communicating with coworkers.
    – ohmu
    Commented Jan 8, 2018 at 19:29
  • 2
    As far as the childcare question, perhaps your spouse takes care of your children? Or they go to school? People have kids and also work do typically have to figure out something to do with them during work hours. :)
    – Julia Silge Staff
    Commented Jan 8, 2018 at 20:50
  • 2
    @cpburnz Email is soooo 2017!
    – Mixxiphoid
    Commented Jan 8, 2018 at 21:21
  • 11
    The update CV also missed the simple option "I was up for a new challenge". The options provided were (all?) negative ones.
    – Jan Doggen
    Commented Jan 9, 2018 at 7:32
  • 7
    @JanDoggen I noticed that too, I interpreted most to be pretty similar. Someone could also update his CV, because he/she likes to keep stuff like that updated... There are a lot of reasons, so an options 'Something different [Textbox]' might have been better.
    – Mixxiphoid
    Commented Jan 9, 2018 at 7:56
  • 9
    What about using phones? What about not speaking to coworkers at all since you are doing a one man company?
    – Lundin
    Commented Jan 9, 2018 at 8:01
  • See Also: meta.stackoverflow.com/questions/361616 Commented Jan 9, 2018 at 11:37
  • @AshleyMedway I know, I already placed the link to this answer in the comments there.
    – Mixxiphoid
    Commented Jan 9, 2018 at 11:49
  • @cpburnz We use email for notifications, not communication (which I read two-way, not a one-way broadcast). So, yeah - thing of the past over here.
    – Izkata
    Commented Jan 10, 2018 at 16:23
30
votes

What ergonomic furniture or devices do you use on a regular basis? Please select all that apply.

Please add an option of "No use of ergonomic furniture". In that case how can one tell if no answer means no use or skipped question? I had to skip while I do not use any ergonomic furniture.


Please rank the following aspects of a job's benefits package from most to least important to you

The options of "Parental leave" and "Childcare benefit" do not apply to me - giving them any mark will influence the overall analyzing of these options (assuming I'm not the only one answering this question and am not a parent). I think there should be a question before that asking if I'm a parent or not any according to that include the options.

Maybe a better way to answer is by ranking only those that are important to me (and my order) and the rest as "not ranked" - if they do not apply to me (or any other reason such as mentioned by @Vogel612 - in Israel too some are mandatory and it makes little sense to rank them..)


What is your current gross salary (before taxes and deductions), in Some other currency:?

My salary is in a currency not in the list. It took me a long time to realize that the "Some other currency" is actually the option I selected before but not the text I entered.

I think it is best to place the text written before, or change the question before to be a drop-down with all the currencies


One minor improvement - after selecting an option/filling in text, I think pressing enter should proceed to the next question.

7
  • 9
    I, too, eschew the use of ergonomic furniture, much preferring the superior Reganomic designs that allow spilled drinks to trickle down to the base. No option for that either though.
    – Shog9
    Commented Jan 8, 2018 at 17:03
  • 4
    @Shog9 - sorry... but don't really get your point. The question is about use of ergonomics, at least it should have the option of no ergonomic stuff are used. About any specific type - I guess having a few common ones as options make sense and then if you really want the one that allows you to spill drinks then to have an option of "other" and specify... The whole reason I pointed it out is that they can't tell if I just skipped the question or actually don't use any. Commented Jan 8, 2018 at 18:13
  • 6
    Sorry @Gilad - I was making light of the fact that we're still using the term "ergonomic" in 2018 (chances are, you are using an ergonomic device of some sort unless all of your equipment is 30+ years old; it means "designed for human use" and has thus become a trite marketing term like "fresh" or "natural" - taken literally, it has only slightly more appeal than buying a can of soda emblazoned with "NO RAT POISON". Of course, the intended meaning was to differentiate new products from older ones that were often not designed for the wide variety of shapes and limitations in humans.)
    – Shog9
    Commented Jan 8, 2018 at 18:25
  • No kids here, either, but "parental leave" and "childcare benefits" are important to me because of what they say about the company culture.
    – Mark
    Commented Jan 10, 2018 at 0:09
  • @Shog9 "Of course, the intended meaning was to differentiate new products from older ones" - If that was the intended meaning, then that's what it should ask. All my furniture and devices are less than 5 years old, none of it labeled "ergonomic".
    – Izkata
    Commented Jan 10, 2018 at 16:31
  • 1
    The intended meaning of the term, @Izkata, when used in marketing. As in "ergonomic keyboard". Contrast with how the term is used by, say, OSHA, where it involves structuring both task and workplace to avoid causing injury to employees (trivia: software shops are exempt from those regulations). The point of the question is probably just to gauge how much cache the term retains in the former sense.
    – Shog9
    Commented Jan 10, 2018 at 17:03
  • 1
    Aside: probably worth reminding anyone reading this that there's a good chance you are responsible for the ergonomics of your own job; unless/until you're injured, your employer probably isn't required to do anything to reduce the toll your work takes on your body. So... It's worth your time to investigate your options ahead of time: take some measurements, make adjustments to how your work area is arranged, and buy equipment that fits your body - whether or not it has "ergo" on the label. Beats spending months in pain recovering from an injury later.
    – Shog9
    Commented Jan 10, 2018 at 19:07
28
votes

This survey takes a while and I want to save my progress to do later but it's not allowed

2
  • Thanks for your interest! You should be able to leave and come back to the survey as long as you use the same machine/browser; it should remember you based on a cookie (so long as you’re not using an incognito browser).
    – Julia Silge Staff
    Commented Jan 8, 2018 at 20:44
  • 7
    @JuliaSilge - thats not how it happened to me, I hit the back button on my PC to go back to previous question and it started all over again.
    – bhaskarc
    Commented Jan 9, 2018 at 16:58
24
votes

One of these options is not like the others.

bisexual || queer

Why are Bisexual and Queer lumped together? I am not aware of any inherent relationship between the two. Honestly I'm not even sure what "Queer" is supposed to mean in this context. The other "or" options appear to be at least somewhat related. Should there be an option for "straight, heterosexual, or maybe Matt from accounting if I've had a couple beers"?

26
  • 5
    Wha'?! Yeah, this doesn't make any sense. I don't claim any particular expertise in gay stuff, but per my understanding (and Wikipedia), "queer" is basically a catchall meaning "non-straight" or "non-cis" or "non-gender-conforming". Lumping it with "bisexual" is bizarre, and probably means that lots of gay or asexual people who also identify as queer (due to it being a catch-all) will tick both boxes.
    – Mark Amery
    Commented Jan 8, 2018 at 19:22
  • 4
    @MarkAmery that's exactly my point. I'm bisexual. I don't consider myself particularly queer. And yes, even if a Q has been added to LGBTQA++ (or whatever the acronym is now) depending on context Queer can definitely be used as a slur. I understand as programmers that (bisexual || queer) == true but that seems to be missing a certain je ne sais quoi of the the matter. Commented Jan 8, 2018 at 19:30
  • 44
    LGBTQA++ sounds like the gayest programming language ever.
    – Mark Amery
    Commented Jan 8, 2018 at 19:32
  • 33
    It's fabulous and it's got QA built right in. We can't go wrong. Until Microsoft comes out with LGBTQA#. Commented Jan 8, 2018 at 19:32
  • 7
    Also: why checkboxes? By definition if one identifies as X, then all other choices are off.
    – Sklivvz
    Commented Jan 9, 2018 at 1:41
  • 9
    "Or you could consider yourself to be Bisexual, Queer and Pansexual." That has about as much informational value as saying that you identify as a "Ferrari vice-janitor in the Jedi Temple on Coruscant".
    – ACEG
    Commented Jan 9, 2018 at 10:46
  • 4
    @Cristina without going full "I identify as an attack helicopter", there are distinctions between -sexual and -romantic identities. You can be bisexual but heteroromantic. If you're not "straight" things get.....a little messy sometimes. Commented Jan 9, 2018 at 13:14
  • 7
    @mikeTheLiar If you're not straight, then you're "other" (at least as far as such a survey goes). In my opinion, the categories cannot be changed every day just because one person out of 7 billion has come up with a new identity and doesn't feel accurately represented. That's also the reason why such categories never make sense -- there will always be someone who doesn't fit any of the little specialised niches.
    – ACEG
    Commented Jan 9, 2018 at 13:37
  • 9
    @Cristina "If you're not straight, then you're 'other'" I hope that you can revisit this statement and understand how condescending it is.
    – jscs
    Commented Jan 9, 2018 at 13:51
  • 6
    @Cristina gee, thanks for that. I really needed to get knocked down a peg this morning. Guess I'll slink back to the closet now. Commented Jan 9, 2018 at 13:54
  • 5
    "I just can't make sense of why these young'uns keep coming up with these new made-up categories when 'straight' and 'weird sexual deviant' are enough to adequately cover everyone!"
    – Mark Amery
    Commented Jan 9, 2018 at 14:42
  • 9
    @MarkAmery That's not at all what Cristina is saying and it's rude of you to suggest so by putting words in her mouth.
    – TylerH
    Commented Jan 9, 2018 at 15:05
  • 6
    @TylerH As far as literal meaning goes, it seems to me that in fact it is what's she's saying - she's said, explicitly, that "straight" and "other' are sufficient categories for the survey and that she doesn't understand why people need to keep inventing such categories. I'm distorting the tone and the connotations for the sake of mischief, certainly - but I think taking someone's literal meaning and framing it differently is a legitimate rhetorical device for looking at their beliefs from a different perspective, rather than just a mean way of slandering them. Your view may differ.
    – Mark Amery
    Commented Jan 9, 2018 at 15:26
  • 9
    @MarkAmery No, what she said was that this survey only considers people to be straight or some kind of 'other'. And then she goes on to claim a pretty progressive stance on the matter, which is one of entirely non-binary choices (e.g. no categories). For you to imply that she is old-fashioned, homophobic, etc. with your quip is wrong, disingenuous, and libelous (not slanderous).
    – TylerH
    Commented Jan 9, 2018 at 15:35
  • 7
    I agree that these options could be improved. If they are going to be non standard then Matt from accounting should totally be there. If not, then they should conform to the standard for these types of questions, already endlessly debated and available in the US census (documentcloud.org/documents/…). The options are: 1) Gay or Lesbian, 2) Straight (that is, not gay or lesbian), 3) Bisexual, 4) Other, 5) I don't know the answer/Prefer not to answer
    – Travis J
    Commented Jan 9, 2018 at 22:14
23
votes

Rank: What do you find most important in an ad

(I'm not sure this was the exact wording, feel free to correct)

I don't get what I should be answering here. I have never:

  • intentionally looked at an ad
  • continued reading an ad as soon as I realized it was one
  • intentionally clicked an ad
  • gotten any value whatsoever out of an ad

And so I'm not sure what this question is supposed to mean at all. I guess it is better when it doesn't intentionally pretend to be an ad, but other than that?

2
  • 2
    They are interested in improving their advertising. Those who are find close to nothing important in an ad (like me) will have difficulties answering this question. It's probably badly posed. Commented Jan 11, 2018 at 22:17
  • 5
    The most important thing in an ad from my POV is that it can be easily ignored or blocked.
    – JeremyP
    Commented Jan 16, 2018 at 11:58
21
votes

Plan for five years' time doesn't include full-time parenthood, volunteer work, or founding of non-profit organisations as possible goals.

As I type this - on my open-source OS written by altruistic volunteers, on a site whose entire raison d'être is enabling programmers to share knowledge out of goodwill - I can't help but feel sad about the answers that are left out on this question, and the fact that nobody has yet pointed it out:

enter image description here

What if I want to found a non-profit, rather than a profit-making company? What if I want to have kids and become a stay-at-home dad, working on open source projects for the good of the world while my wife earns the money to support us? I can't express any of those aspirations here.

4
  • 1
    "Working as a founder or co-founder of my own company" doesn't specify the company is profit-making, so I don't really see the dilemma there.
    – user812786
    Commented Jan 11, 2018 at 14:54
  • @whrrgarbl I imagine the definitions here are complicated and differ by jurisdiction (and I'm not going to pretend to have much of a clue), but not all (or even most? I don't really know.) nonprofit organisations are companies.
    – Mark Amery
    Commented Jan 11, 2018 at 14:57
  • There still needs to be an "Other" or "None of the above" choice. Commented Jan 12, 2018 at 20:53
  • @AdrianMcCarthy Yes, I suppose that's true. There are certainly possible aspirations that probably shouldn't be explicitly on the list, like "starting my own criminal enterprise stealing credit card numbers", and they need to be encompassed somehow. :)
    – Mark Amery
    Commented Jan 12, 2018 at 20:55
19
votes

I guess I work for a very old-fashioned company. Does anybody else work for a company that coordinates or shares information using telephone calls, or in-person conversations, or email, or is it all proprietary buzzwords that I've never heard of?

2
  • 2
    Yep, I used the fill-in-the-blank box for this one. "Email, phone, through the cubicle wall.."
    – user812786
    Commented Jan 11, 2018 at 14:40
  • 3
    I even dared to mention acoustic communication including near range direct visual contact. Commented Jan 11, 2018 at 22:18
18
votes

There are too many US-centric questions to feel comfortable. Where should I rank health insurance importance when it's 100% free and granted the moment you are born in 99% of the countries around me?

Same applies for parental leave, retirement funds, etc.: they're all enforced by law, and as far as I know, it's the same almost anywhere I might work.

The questions and results of this answer should be divided by continent and then by country, to not skew the data. Hope this can be fixed for the 2019 developer survey.

17
votes

I took the survey and here is what I think about it:

jobs:

I'm currently not looking for a job and, like last year, it annoyed me a lot (almost to the point of quitting all together) that the survey continued on about jobs and work experience even though I stated I don't have any (shouldn't be too hard I guess) however it made me sit through a ton of questions that I simply could not answer.

I've also stated in my survey that I'd like to personalize my experience on SO more. For example I'd love to be able to turn off SO Careers all together as it does not interest me, leaving me not only happier when using this site but also allowing SO to show me more relevant ads instead.

AI:

Why? I mean why so specifically AI? Are you guys up to something?

Personal Info:

Again why? Since when does a programmer form need to know my sexual orientation? I've never seen an ad about a sexual related thing on this site (as I don't think it would generate much traffic here)

Length:

I said that the survey was appropriate in length and nor difficult nor easy to fill. but thinking about it I might come back on that decision and say it is a bit too long...

Things that you could add next year:

I think the survey is created to generate more profit for SO (as it's promoting SO Careers and harvesting info which can be use to improve ads for the user) and I'm totally OK with that! However I would have liked to see something in return like:

  • Allow for more elaborating on answers. Questions about the way SO works could be answers with agree to disagree (and 3 options in between) but I was unable to explain myself (this even cause me to change from my answer as I was unable to justify my original choise)
  • Allow for a better feature request section (it was a one line question with minimal space to answer)

Other that that I think it was a good survey

17
votes

The followup question (missing from the survey for a short period due to a bug) now appears.

What is your current gross salary (before taxes and deductions), in Euros (€)? Please enter a whole number in the box below, without any punctuation. If you are paid hourly, please estimate an equivalent weekly, monthly, or yearly salary.

So what am I supposed to enter here? Weekly, monthly or yearly salary, however I choose? If I type in 10000, how would you know this is my monthly or yearly earning? Please, have your survey tested extensively before making thousands of people answer it.

Also, what is the use of this question? I am working halftime right now. The yearly salary (which is what I entered) looks as if I were underpaid. For a meaningful comparison, this question should ask for the hourly wage.

Also also,

Which of the following libraries, frameworks, and tools have you done extensive development work in over the past year, and which do you want to work in over the next year?

There are so many libraries out there, I could instantly name 10 which I have been using "extensively" and which havent been on this tiny list. Whats the point of this question?

4
  • Same here, I work only 10 hours a week (and of course earn much less than if it was 40 hours), but there is no way to specify it. Commented Jan 10, 2018 at 12:45
  • 4
    +1, how would you know this is my monthly or yearly earning?
    – Zanon
    Commented Jan 10, 2018 at 14:22
  • 2
    Briefly, there was a bug where the follow-up question ("Is this salary weekly, monthly, or yearly"?) did not show up. We will see what we can do with the salary info for people who took the salary during this brief window, but we may have to replace these with blank values. As a data scientist, this HURTS as you can imagine.
    – Julia Silge Staff
    Commented Jan 11, 2018 at 23:38
  • 1
    Briefly? I just went through the survey today and it didn't appear, and in reading the posts here various that other s sat didn't appear for me.
    – Rory Alsop
    Commented Jan 24, 2018 at 10:11
15
votes

Just completed the survey, I have two suggestions

About AI

You asked me if I'm actually interested in AI (and I'm not) but previously I had to answer to 3 questions about it: wouldn't be more logical to ask first if the user is interested about this topic so, depending on the answer, he could skip all the related questions?


Unethical code

Imagine that you were asked to write code for a purpose or product that you consider extremely unethical. Do you write the code anyway?

At a first reading this is how I placed the round brackets:

Imagine that you were asked to write code (for a purpose or product that you consider extremely unethical). Do you write the code anyway?

because several years ago at the beginning of my career I had to write code for a real unethical product: then, reading at the available answers I realized the real question (maybe?) was

Imagine that you were asked to write code that you consider extremely unethical for a purpose or product. Do you write the code anyway?

I feel this question could be better rephrased.


P.s. : no jar with the candies this year? "Mr. Anderson you disappoint me"

enter image description here

7
  • 5
    You are making me confused now about the ethical question. Not sure how I should have answered it now. :)
    – Mixxiphoid
    Commented Jan 8, 2018 at 19:15
  • 11
    the point is : this question is actually asking if you wrote unethical code (e.g. hundreds of popup, no support for assistive technology...) or if you wrote code for unethical products? there is a huge difference. Commented Jan 8, 2018 at 19:44
  • 1
    Welp, guess I misunderstood the ethics questions then.
    – poke
    Commented Jan 9, 2018 at 1:09
  • 11
    Maybe I'm the only one but I found the ethics portion extremely vague to the point where I was constructing various scenarios to try to make sense of it. e.g. could this be some sort of Edward Snowden situation, etc.
    – Randy Levy
    Commented Jan 9, 2018 at 5:36
  • @RandyLevy Similar here, my issue with this whole section was that there's little leeway for situations involving coercion. At least there was a text box to explain it later on, unlike several of the other sections.
    – Izkata
    Commented Jan 10, 2018 at 18:01
  • In common usage, that introduces a restrictive clause, which comes immediately after noun phrase it modifies. Thus the first reading was correct. It's the purpose or product that was unethical. I'm surprised anybody found this ambiguous. Commented Jan 12, 2018 at 21:03
  • Probably something on the answers made me think to the other possible reading, maybe because, as an employed coder, I am in control of the code and I have responsibilities for it, not about who should be my customer or my project. This led me to misunderstand the question. Commented Jan 12, 2018 at 22:49
14
votes

(Edit: This seems to be )

Guys give me a hint. Should I leave them blank or tick those boxes?

Bug 1Bug 2Bug 3Bug 4Bug 5

Plus another : Why am I given checkboxes instead a group of radio buttons for some of them? I suppose all of them should be radio buttons (from which you can select only one for each group).

9
  • 6
    A None of the above would've been nice here
    – George
    Commented Jan 8, 2018 at 14:56
  • 3
    Tick the box, but then untick it and leave it blank for 6 minutes, then type "I prefer not to answer" in the "own words" option
    – musefan
    Commented Jan 8, 2018 at 14:56
  • 1
    Checkboxes are on purpose here because people may want to check off more than one. Having none of the above does make some sense, I agree... I thought the "if any" might cover that but perhaps not.
    – kristinalustig StaffMod
    Commented Jan 8, 2018 at 15:01
  • 5
    @KristinaLustig The problem is with the wording, leaving it blank makes it seem like you'd prefer not to answer, when you may be more than happy to.
    – George
    Commented Jan 8, 2018 at 15:05
  • This was already posted as the first answer to this question, why post it again? Certainly the repeated screenshots are not necessary either.
    – user247702
    Commented Jan 8, 2018 at 15:17
  • Please make pictures smaller (if not remove some) Commented Jan 8, 2018 at 15:18
  • So post a second answer then. Or just remove the first bug, since it's duplicated.
    – user247702
    Commented Jan 8, 2018 at 15:22
  • Besides, obviously you cannot belong to multiple age groups, or answer both "yes" and "no" as to whether you have children. Radio buttons = one choice, checkboxes = multiple choices.
    – user247702
    Commented Jan 8, 2018 at 15:23
  • @Stijn But you can be gay and straight but not bisexual according to this survey (I'm assuming it was done like that to avoid any offence)
    – George
    Commented Jan 8, 2018 at 15:25
14
votes

The "how many years" question has non-contiguous answers, meaning some people can't select a correct answer because they're in one of the gaps between answers:

enter image description here

I've been coding for 5 and a half years. What answer am I meant to pick?

11
  • 4
    Isn't that "3 years <= experience < 6 years"? Commented Jan 8, 2018 at 19:11
  • 5
    @CamiloTerevinto maybe. Or maybe I should round to the nearest year. The survey doesn't say.
    – Mark Amery
    Commented Jan 8, 2018 at 19:12
  • 11
    int coding_years = 5 + 1/2; If you have coded for 5 years, you must have learnt about integer arithmetic :P
    – Lundin
    Commented Jan 9, 2018 at 8:05
  • 18
    Have you coded for 5 years? true. Have you coded for 6 years? false. Commented Jan 9, 2018 at 10:11
  • 2
    @AmaniKilumanga by that logic, I could just as well pick the 0-2 years option.
    – Mark Amery
    Commented Jan 9, 2018 at 11:34
  • @mark why would you do that though? Commented Jan 9, 2018 at 11:38
  • @AmaniKilumanga I wouldn't, but the point is that it can't be the correct logic to apply to interpret the meaning of these answers, because it doesn't identify a unique answer.
    – Mark Amery
    Commented Jan 9, 2018 at 11:39
  • 7
    Seems like this should be a slider, or freeform. Easier for entry, and then whoever analyzes it can use whatever bins they desire (including trying out different bins).
    – jscs
    Commented Jan 9, 2018 at 13:41
  • Proper list: 0-2.9999999999999999999999999999999 3-5.9999999999999999999999999999999 6-8.9999999999999999999999999999999
    – GEOCHET
    Commented Jan 10, 2018 at 19:08
  • @GEOCHET So you speculate. But maybe it's 0-2.4999999999999999999999999999999, 2.5-5.4999999999999999999999999999999 5.5-8.4999999999999999999999999999999. The whole point is that we don't know, and that different answerers will have interpreted it differently.
    – Mark Amery
    Commented Jan 10, 2018 at 19:10
  • I was disappointed it stopped at 30+. We're not all millennials! Commented Jan 12, 2018 at 20:59
12
votes

The whole "benefits package" question makes no sense to me.

enter image description here

I'll concede that some of these things - like parental leave, company-provided transport, or company-cooked meals - are quality-of-life benefits that different people might trade off differently against money based upon how much they personally value them. But they're a minority. All of these things, on the other hand, are strictly financial benefits, directly fungible with money:

  • Salary and/or bonuses
  • Retirement or pension savings matching
  • Fitness or wellness benefit
  • Conference or education budget
  • Childcare benefit
  • Stock options or shares
  • Health insurance

Thus, the obvious answer - and surely the only remotely rational one - is "I'll value them in proportion to the dollar value that they have to me, given my circumstances." But since the relative size of these things can vary wildly between employers, that means that ordering them basically consists of me guessing at what the dollar value of each benefit is at a typical employer and ordering by that, rather than the ordering in any way reflecting my values. I'm not sure that this process produces any useful data whatsoever.

If you want to elicit information about how much I value particular benefits, you'd be better off removing all the strictly financial benefits and then asking how much salary-or-equivalent I'd be willing to sacrifice for each of a series of fluffier benefits like having the company cook my meals and having a big equipment allowance so I can buy some nice monitors for my desk. But what do you expect to learn from asking me what category I'd prefer the $$$ I get paid to fall under on my payslip?

9
  • 1
    Can't you just order the list the way you like it? Converting items into "amount of $$$" and ordering by that is very programmish way. They probably just want to know what is the most/least demanding benefits to you personally, maybe there will be some interesting finding for certain groups of persons (e.g. by sex, by family status, etc).
    – Sinatr
    Commented Jan 9, 2018 at 8:32
  • 7
    @Sinatr why would there be any correlation with any of those things? A $50k salary and a $1k food budget is the same amount of money as a $51k salary, whether you're a man or a woman. The only place there's going to be a correlation is where there's a financial benefit that is only available to particular groups - a public transport budget is useless to me if I can only get into work by car - but even then, the value of a $1k public transport budget to someone who commutes by train is $1k; how are they meant to order that against salary in any way besides "salary's bigger, so matters more"?
    – Mark Amery
    Commented Jan 9, 2018 at 11:32
  • 1
    I answered this one as if I were comparing offers. "All else being equal, if Company A gives me $X for conferences, but Company B gives me $X fitness benefits, which would I prefer?" (Things I didn't care about went in to the bottom, e.g. childcare benefits since I don't have any dependents). Also in the US, healthcare can be difficult to assign a straight dollar value to - you might pay the same amount under two different plans but get very different coverage.
    – user812786
    Commented Jan 9, 2018 at 19:42
  • 1
    @MarkAmery Well, what I was looking for (flexible hours) isn't there, and is preferred by women more than men - so yes, there will be correlations outside of equivalent dollar amount, even when it's unrelated to group availability.
    – Izkata
    Commented Jan 10, 2018 at 18:15
  • 1
    @Izkata sure, flexible hours is a great example of something that you might trade off against salary but which different people (or demographics) might put very different dollar values on. I can think of lots of others - like ability to work remotely, or opportunities to contribute to open source, or job title, or whether the job involves managing other people, or commute distance, or length of work hours. But my whole point is that (besides "parental leave") the options here aren't things like that - they're all just "$$$" or "$$$ with strings attached".
    – Mark Amery
    Commented Jan 10, 2018 at 18:21
  • On this Q and on the adjacent drag-to-order Q there are a couple of items that are important and a couple that I really do not care about. For the remaining 5 or 6 the order is unimportant.
    – AdrianHHH
    Commented Jan 12, 2018 at 20:44
  • 1
    I disagree with the premise of the OP. Most of those benefits are difficult-to-impossible to assign a monetary value to. Stock, for example, involves a very different level of risk and potential reward and timing than salary. Some people would gladly take a lower salary for more stock on the possibility that they could make a lot more money on the stock. For others, the predictability of wages is more important. Even a bonus typically only has a target, and thus has risk and reward factors that are not directly comparable to wages. Commented Jan 12, 2018 at 21:14
  • @AdrianMcCarthy Yes, if I was interviewing at Google, I'd view stock options in a different light to interviewing at Yahoo. Also health insurance to somebody in the UK is different to health insurance to somebody in the US where it might be literally life or death.
    – JeremyP
    Commented Jan 16, 2018 at 12:09
  • Why did I not get shown this question...
    – Rory Alsop
    Commented Jan 24, 2018 at 11:34
10
votes

I wasn't sure how to answer the question regarding whether or not I contribute to Open Source.

I upload my personal projects to Gitlab under MIT/GPL/etc. licenses, but I don't contribute to third-party software. I don't know if that would count as "contributing", so I answered "no".

In the future, it might be better to distinguish between "I release my own code under a FLOSS licence" and "I contribute patches to projects with a FLOSS licence".

EDIT: Related: Survey question "Do you contribute to open source projects" should not be a binary response

10
votes

I am seeing an advert for the survey although I have already completed it. It's in German (which is probably correct, assuming I am in Germany), but when I took the survey initially, it was in English. There is no way to dismiss this advert, no voting or any other means to rate the ad like there is for external adverts.

German survey ad

In addition to that, I do not recall there being a question like the one in this ad in the survey.

Mac or Windows? [Your choice]

This is very misleading. (I would chose neither, by the way).

0

Not the answer you're looking for? Browse other questions tagged .