99

tl;dr:

In anticipation of our annual Developer Survey being ready to release any day this week (update: already live), we're happy to announce a new badge: "Census". This is a silver badge that can be earned multiple times - once for each year you complete our annual developer survey, starting in 2017.

The last few surveys have contained a field where you can (optionally) provide a link to your Stack Overflow profile; providing your profile link going forward will result in the badge being awarded to you.


Why do we need a badge for this?

Our Developer Survey results have become a de facto source of information for many companies that employ developers, or make tools for developers to use. We didn't set out with that goal in mind, we simply felt that it would be interesting to have a survey.

What remains is that the results of our survey help guide decisions that can have a profound and reaching impact for developers; we want to do everything that we can to help ensure that we have the largest, most diverse, and most representative group of developers possible participating in the survey.

Our hope is that a shiny silver badge will provide more incentive for more users to complete the survey, and help ensure that users that completed previous surveys continue to make time to participate.

Is the badge mandatory? How is it awarded?

No. At the completion of the survey you'll be prompted for an optional link to your Stack Overflow profile, as we've done in previous surveys. The difference is that this year (and going forward), providing a link will trigger the badge to be awarded to you.

If you're uncomfortable receiving the badge (which is displayed like any other badge), don't provide your profile link. This will be made clear on the survey.

What if I put in an incorrect link? What if I want to give it to my friend instead?

If you made a mistake entering the link, contact us and we'll correct it.

Badges found to be awarded incorrectly will be removed; if this looks to be intentional it might cause you a bit of grief. If shenanigans are obviously afoot you might even get a post (no pun intended) card from Tim ranting about you being the reason that we can't have nice things; but don't mistake that as incentive for tomfoolery.

I completed the last few surveys, can I get a badge retroactively?

No, we can't award the badge on a date prior to the creation date of the badge. But you can earn one very soon!

I (love/hate) this idea or I have questions that you didn't answer.

Post an answer or a comment and I'll do my best to help.

The Developer Survey is just days away, you said?

Check the blog, there will be an announcement soon!

38
  • 67
    I hope that users don't fill in crap just to get the badge...
    – honk
    Commented Jan 9, 2017 at 21:20
  • 63
    No one would do that. Next you are going to suggest that people post advertisements as questions or questions as answers or answers as comments!
    – Travis J
    Commented Jan 9, 2017 at 21:24
  • 47
    @honk Our data science team has gloves, no worries.
    – user50049
    Commented Jan 9, 2017 at 21:25
  • 9
    @honk: This is always a tricky balancing act. In the past, we gave out stickers and had contests for physical swag. We don't have any evidence that skewed results at all. We'll see how this goes, but my guess is that people who take the survey for the badge will go ahead and be honest on the survey too.
    – Jon Ericson Staff
    Commented Jan 9, 2017 at 21:25
  • 21
    Thanks for this. Maybe getting a new badge will make me feel a little better about losing my hat. Commented Jan 9, 2017 at 21:30
  • 43
    Does providing a link to your SO profile establish a connection between the survey data provided by the user and the profile? Stated another way: Once the link to a profile is provided, is the data anonymized prior to being passed to people (particularly third parties) for analysis or otherwise? More generally, other than the badge, what is the purpose of providing a link to the user's SO profile, and how will that link/connection be used?
    – Makyen Mod
    Commented Jan 9, 2017 at 21:42
  • 16
    @Makyen Yes, it's totally anonymized. It's nice of people to provide a link so we can get a better picture of their professional world, but that data is only used internally (with our thanks to you) or in aggregate form as we release results.
    – user50049
    Commented Jan 9, 2017 at 22:00
  • 63
    Can I earn multiple badges per year for encouraging other people to use my profile link when submitting their survey responses?
    – Shog9
    Commented Jan 9, 2017 at 23:00
  • 16
    @Shog9 My feeling. You hurt it.
    – user50049
    Commented Jan 9, 2017 at 23:03
  • 4
    @TimPost Does the survey require that all questions be answered? IOW, can questions be left blank? Since the answers will be associated with an account (either internally or externally), I would like to not have to answer questions which I don't want to.
    – Mysticial
    Commented Jan 9, 2017 at 23:03
  • 4
    @Makyen An example of how we could use the SO profile data (internally only) is to train and improve our algorithms that suggest relevant jobs to users. As one example, by looking at what kinds of questions & tags are visited by people who describe themselves on the survey as front-end web developers, we can better decide whom to show ads for front-end jobs. (That wouldn't be targeting those users specifically: it would be using what we learned from them to target everyone who visits the site).
    – David Robinson Staff
    Commented Jan 10, 2017 at 0:31
  • 4
    I'm sure this has probably been in a blog or Meta post somewhere, but I can't find it. What has historically been the general participation level of Stack Overflow users in the survey? It would be interesting to compare levels of participation before and after the badge, just to see how much shiny yet worthless Internet prizes actually incentivize participation. Perhaps I'm too cynical, but as I read this post, that does seem to be the primary purpose of introducing this badge.
    – Cody Gray Mod
    Commented Jan 10, 2017 at 9:44
  • 14
    @CodyGray Isn't the purpose of all badges (outside of tag badges) to incentivize a behavior users wouldn't otherwise do as much?
    – Magisch
    Commented Jan 10, 2017 at 12:31
  • 42
    Silver seems a little high for this IMHO. Commented Jan 10, 2017 at 17:44
  • 5
    @PaulG you can always fill the survey without linking it to your profile. Commented Jan 11, 2017 at 14:26

5 Answers 5

91

I'm generally in favor of this, but I have one (big) concern.

You say the developer survey is one of the key ways for employers to understand developers.

If I provide my profile link in the survey, didn't I just hand you a gigantic package of hugely personal, personally identifying data about me?

Can you ensure this data will not be shown to employers in any way? Say I state on the survey that I'm looking to leave my current employer, it would be oh so much a shame if that current employer had a way of finding that out, even indirectly. So can I be sure that an employer can't infer things about me that I told you in confidence, even if it's just by recommendations / matches that they go that data?

Putting this much valueable data in SO's hands (and it's a lot, a sincerely filled out developer survey is just about the most valueable pack of professional data anyone can get their hands on about a developer) requires a great deal of trust.

Obviously the ability to include the link isn't new, but now it seems like you're strongly encouraging it. Is this because you want to correlate the survey data with other data about me, like what questions I answer or what tags I'm active in, before anonymizing it?

6
  • 32
    The ability to provide a link to your profile isn't new. Survey data is anonymized, as Tim Post♦ explained
    – Cerbrus
    Commented Jan 10, 2017 at 8:25
  • 7
    That's why it's a silver badge and not bronze, I guess
    – Oriol
    Commented Jan 10, 2017 at 18:22
  • 10
    I agree with this point, and am troubled by David Robinson's eagerness to use this data on a per user basis to do analysis.
    – Travis J
    Commented Jan 10, 2017 at 20:02
  • 2
    Yeah, this point should have been made clearer in the OP. Commented Jan 11, 2017 at 6:28
  • 3
    The answer to your root question here, regarding making sure employers never see it is a very resounding YES. The point of this is to encourage the most diverse group possible to engage with the survey, there wasn't any other motive there. While the conversation did point out that we use the link if you provide it for internal research, it was purely a coincidence. The badge is there if you want to have it, and if you don't, we're totally fine with that. But this was all about the badge, nothing else changes, any results released anywhere are anonymized and presented in aggregate form.
    – user50049
    Commented Jan 11, 2017 at 15:48
  • 11
    I just hope anonymization works better than in my last company: the team leader would get anonymized results for its team about the "motivation" survey, excluding contractors. Needless to say, in teams with only one permanent employee apart from the team leader, anonymity wasn't exactly stellar. And then, it got compounded when they made it possible to break down the company-wide results by demographic: remind me, how many black women in a Senior Software Developer position have been in the company for 25+ years? Hum... Commented Jan 11, 2017 at 20:28
26

How will the survey control for people who will just fill in random options to get through the survey ASAP to get a badge?

For example, will it have attention filters? Measure timing? Something else?

6
  • 2
    I would expect that sheer sample size would normalize these... outliers?
    – Braiam
    Commented Jan 10, 2017 at 14:53
  • 2
    @Braiam That could be the case and a possible answer; dunno. Not making assumptions :-) I do happen to know that in general "people randomly filling in stuff to get foo (in this case a badge)" can be a big problem for some surveys, so I was curious how the SO people are handling this. Commented Jan 10, 2017 at 15:01
  • 51
    Or audits? You have made too many incorrect answers, Come back in 3 days and complete the rest of the form.
    – Bhargav Rao Mod
    Commented Jan 10, 2017 at 18:03
  • 3
    @Braiam that is true for robust statistics, but it could have noticeable effects on weaker statistics (say, if a group of jokers started screwing with COBOL data) As an example of joke data approaching real data numbers, note that the percentage of UK voters who claimed to be Jedi is similar to the percentage of US voters who voted for Jill Stein.
    – Jeutnarg
    Commented Jan 10, 2017 at 18:12
  • 22
    @BhargavRao "you have indicated that you have learned enough jQuery. This is incorrect; you always need moar jQuery. Badge no longer attainable for this survey."
    – TylerH
    Commented Jan 10, 2017 at 18:52
  • 5
    The survey itself doesn't lend well to people just blowing through it to get a badge, and our data team can pretty quickly identify and filter out noise. We believe that people will take the survey honestly, but if this (and I sincerely doubt it will) became a problem, I'd probably have some .. words .. with a select group of participants :) Let's just leave it at that.
    – user50049
    Commented Jan 11, 2017 at 15:40
25

The final question of the 2017 survey is:

Thank you for completing the survey!  This year we are awarding a silver ""Census"" badge on Stack Overflow to users who completed the survey.  If you would like to receive the badge, please paste a link to your profile page in the box below.  (If you are currently logged in to Stack Overflow, you can access your profile page quickly by going to https://stackoverflow.com/users/current.)  The badges will be awarded in a batch at the end of January. As a reminder, all survey results and data sets will be anonymized before being published. The link between your profile data and your survey responses will be kept confidential.

I've awarded the first batch of Census badges. So far 12,595 of the people who submitted their profile links have received a shiny new silver badge.

Trivia:

  • A few people who answered the question didn't get the badge because they entered something that I couldn't interpret as identifying a particular person such as:

    • "thanks" (You're welcome, by the way.)
    • A link to the survey itself. x 9
    • A link to a private YouTube video.
    • https://stackoverflow.com/users/current x 58
    • https://stackoverflow.com/help/badges
    • https://stackoverflow.com/#_=_
    • https://stackoverflow.com/
    • A common display name.
    • "OK" and "yes"
    • A rude word.
    • A string of characters I'm 90% sure are a password.
    • Email address we don't have any record of.
    • "Sure. :) Send me badges soon."
    • A link to a question.
    • The date.
    • Unsolicited comments on the survey/progress bar.
  • I can't really blame people for not reading the instructions.

  • Most people skipped the question for whatever reason. So the majority of respondents aren't motivated by badges. Unless they failed even more completely than the folks above who at least entered something.

If, by any chance, there was some error in awarding the badge, please use the contact form to let us know.

25
  • 8
    What was the private video about?
    – Pekka
    Commented Jan 19, 2017 at 22:14
  • 3
    I'd imagine Jon wouldn't know- Unless he meant "Unlisted" and just typed "Private". (Unlisted videos can be shared with a link, private videos can't be seen by anyone but the poster, IIRC) Or if by "private" he didn't mean the video's privacy setting...
    – Kendra
    Commented Jan 19, 2017 at 22:17
  • 18
    @Kendra: Correct. I was a bit nervous to watch a random video and was relieved when it wouldn't play. And then I was disappointed that I hadn't been rickrolled.
    – Jon Ericson Staff
    Commented Jan 19, 2017 at 22:26
  • 1
    I contacted SO about not having the badge yet, and got this reply: "The process to award the badges is a manual one and we are still collecting profile URLs to award the badge to." @Jon can you clarify the process a bit? Are there survey responses from the first week that still need to be processed?
    – user247702
    Commented Jan 20, 2017 at 12:58
  • 4
    @Stijn: Yep. I thought I had the entire week of results, but it was just the second batch of users. I expect to get the initial group later today.
    – Jon Ericson Staff
    Commented Jan 20, 2017 at 16:56
  • 1
    @Andy: I'm working on the first and third batches right now. It's taking way longer than I anticipated to clean up the data. If my family doesn't drag me away, I'll get them done tonight. Stay tuned.
    – Jon Ericson Staff
    Commented Jan 28, 2017 at 1:03
  • 2
    I can wait for a bigger number next to my silver pixels. Let your family drag you away. It's Friday. :)
    – Andy Mod
    Commented Jan 28, 2017 at 1:06
  • 1
    @Andy: Heh. I'm mostly doing it now so that I don't have it hanging over my head all weekend.
    – Jon Ericson Staff
    Commented Jan 28, 2017 at 1:09
  • 2
    @ChrisJester-Young: Unfortunately, no. :-( Sorry.
    – Jon Ericson Staff
    Commented Jan 29, 2017 at 2:03
  • 1
    @DonaldDuck meta.stackoverflow.com/questions/341214/… "I thought I had the entire week of results, but it was just the second batch of users. I expect to get the initial group later today."
    – Veve
    Commented Jan 31, 2017 at 16:33
  • 1
    @DonaldDuck: I don't have the final list yet. Once the survey closes, I expect I'll have your profile link and you'll get the badge a bit later.
    – Jon Ericson Staff
    Commented Feb 1, 2017 at 15:56
  • 1
    It says the badges will be submitted in a batch at the end of January. What happens to the people who took in the first few days of February when the survey is still open? When will we get our shiny silver badge?
    – mahacoder
    Commented Feb 6, 2017 at 10:13
  • 1
    I am pretty sure I did exact what is needed to get the badge, but I still don't have it yet. I completed the survey at the 1st of February somehow before 14:30, I posted after it about it, thatswhy I know the almost exact timestamp. Commented Feb 7, 2017 at 11:12
  • 1
    @ak31: I see you now have a shiny new badge. Enjoy!
    – Jon Ericson Staff
    Commented Feb 15, 2017 at 17:41
  • 1
    @АлександрФишер: I finally got around to the last set of users, which means you have a new badge. Thanks for participating.
    – Jon Ericson Staff
    Commented Feb 15, 2017 at 17:43
13

I'll try to put the security concern in perhaps a different perspective:

When will the link between the survey data and a particular user profile be deleted?

A couple possible answers to illustrate what I'm after:

  • As soon as the badge is awarded, the profile link is deleted forever (before any analysis).
  • We do the planned analysis, get the results (aggregated data) then delete the profile link - X weeks after the survey is taken at the latest.
  • We keep the raw data including possible profile links for unlimited time in case we come up with more ways to use it (but promise never to share non-aggregated data).

Also, do you consider accommodating for users that would like to get the badge, but don't wish their data to be linked to their profile for any other purposes (i.e. give us options "keep the survey data anonymous", "link the survey data to profile data and get a badge" and "keep the survey data anonymous but get a badge anyway" - for example by adding an option to only use the profile link for badge then deleting it immediately)? I understand that means less data for analysis, but it would be in line with the stated primary goal of making more devs participate in the survey.

3
  • Unless I'm mistaken, it has been an option to link the results to your profile in past surveys. As I see it, the badge is not a new attempt to get more personalized data, it's just a way to motivate users to cooperate with the existing method of getting more personalized data. Commented Jan 11, 2017 at 20:59
  • 4
    I understand that there used to be "provide anonymous data" vs. "provide data linked to profile" choice in past. Now it seems it will be "provide anonymous data" vs. "provide data linked to profile and get badge" choice. Hence my question about whether SO is considering adding a "provide anonymous data and get badge" option to the two current ones. Commented Jan 11, 2017 at 21:06
  • 4
    Thanks for asking this. Tim has said a few times that the data released to third parties is anonomised, but like you I would rather this data be 100% anonymous on collection. In this day and age of data breaches it's just one more thing to worry about
    – PaulG
    Commented Jan 12, 2017 at 8:36
7

Is the "Census" badge exclusive to Stack Overflow?

For example, can I provide a link to another site, say Stack Overflow in language X?

1
  • 6
    Unfortunately, the badge is only enabled on Stack Overflow for the moment. Some people included urls to their profiles on other sites and we gave the badge on SO if the user had an account there. If there was no linked SO account, we didn't award any badge at all. I hope that we can change the policy in future years. (Ideally, we'd have surveys that are more broad than our core programmer population.)
    – Jon Ericson Staff
    Commented Jan 29, 2017 at 3:51

You must log in to answer this question.