-168

As we prepare to launch the 2023 Developer Survey in the coming days, we wanted to provide an update to the community regarding the Census badge.

In case you're not familiar with the Census badge, this badge is awarded to users who complete the Developer Survey.

Beginning this year, we will no longer give the Census badge to those who complete the entire Developer survey. Additionally, as of 2023 May 4, we have removed previously awarded Census badges.

These actions are being done to better protect the privacy of those who will take the survey in the future, as well as the privacy of those who have completed previous surveys. When we reviewed the process for this year's survey, it was determined that assigning the Census badge added an unnecessary element of privacy risk. Although we understand this may be disappointing for those who spent time completing the survey to earn this badge, your privacy is of the utmost importance.

In the interest of ensuring user safety, we make it a matter of practice to not give a level of detail that would allow a bad actor to work backwards into the discovery of the very privacy issue that we're addressing.


UPDATE 2 by Philippe 2023 May 8

The census badges (one per person, as described in the first update) have been restored (and the badge retired, so it shows up in the "retired" section). Our thanks to the product and engineering team for their quick work in getting this done, and to everyone here who made clear their desire that we find a way to keep the badge.

UPDATE by Philippe 2023 May 5

OK, we've heard you. We have heard that the badge is important to you and that if we can find a privacy-compliant way to bring it back—at LEAST for the folks who already had it—that would be ideal.

And then we were approached by Machavity with an elegant solution. So here's what we're going to do:

During the week of May 8th, we will be re-issuing the Census badge to all community members who previously received the Census badge. However, as part of our commitment to privacy we will be making some changes to the badge:

  • Users will only be able to receive the Census badge once. If you previously had multiple Census badges, you will now only have one Census badge.
  • All Census badges will have the same issue date.

Beginning with the 2023 Developer Survey, we will no longer give the Census badge to those who complete the entire Developer Survey.

We believe that this provides sufficient anonymity that the chances of backtracking the data to individual survey respondents are extremely low. We hope that this will be an acceptable compromise and that you will know that we took the feedback very seriously.

A big thank you to:

  • Machavity, who first provided this solution as a potential workaround to the Stack Overflow team.

  • Everyone who shared their feedback about our previous announcement. While we recognize that the previous post may not have been well received, it was great to see community members engage with us and provide feedback in a constructive and respectful manner.

48
  • 101
    I'm honestly a little confused, at least from the perspective of a site like Stack Overflow, how the awarding of a badge could be a "risk" to the user's privacy. I can understand this stance for smaller communities, as if you have a census where only 100 people respond, you may well be able to do some work to identify the user, but on a site like Stack Overflow there is a much larger volume making the ability to identify an individual far harder.
    – Thom A
    Commented May 4, 2023 at 15:59
  • 88
    Privacy matters a whole lot to me, but I had to give this post a downvote. Any chance you can explain how this poses a privacy breach risk? Commented May 4, 2023 at 16:02
  • 50
    The badge had been awarded about 250k times; I'm not sure how many times the survey has been completed, but even if it was 10 times, that means that an average of 25k users respond each year; identifying a user from who got the badge would be a verydifficult/impossible task. Honestly, this feels like misuse of using "privacy" as an excuse to perform this action.
    – Thom A
    Commented May 4, 2023 at 16:03
  • 65
    "Additionally, as of 5/4/23, we have removed previously awarded Census badges." I'm also a little disappointed that this was just removed, without community input.
    – Cerbrus
    Commented May 4, 2023 at 16:07
  • 19
    good riddance. the survey hardly serves the community anyway.
    – Kevin B
    Commented May 4, 2023 at 16:07
  • 30
    "it was determined that assigning the Census badge added an unnecessary element of privacy risk" By who? Someone that doesn't understand what these words mean? Commented May 4, 2023 at 16:12
  • 16
    IIRC, there's at least one or more questions in prior years on MSO or MSE which discuss situations where the Census badge could be used, in combination with data from the survey, to identify individual users and their responses. I'd suggest looking for those questions and linking them here.
    – Makyen Mod
    Commented May 4, 2023 at 16:15
  • 43
    I'll weigh in here as well - I personally vetted the experience that we're talking about here, because I was prepared to shut down this situation and refuse to remove the badge. I was convinced through the demo that I was given that there was a significant enough chance to violate the privacy of our users that I would not feel comfortable leaving it as is. So while it was not an idea I originated, I did vet the problem and agreed that it needed fixing. I"m sorry that I can't give more details: as the text says, it would be trivial to work backward into the process from there.
    – Philippe StaffMod
    Commented May 4, 2023 at 16:22
  • 20
    Can we please be clearer than "unnecessary element of privacy risk"? If that is a risk of a PII leak, then it's one thing, we (at least I) can understand that. If it's a vague "privacy risk", then I fully agree with the reaction this announcement is getting right now.
    – 0Valt
    Commented May 4, 2023 at 16:39
  • 50
    Skipping over the obvious part where the badge doesn't leak any answers you gave, we already had the choice of whether or not to accept the badge. If people were really concerned about the privacy concerns, they could just reject the badge. Why take away that option from the rest of us?
    – vandench
    Commented May 4, 2023 at 16:44
  • 13
    @Cerbrus The individual responses are available in de-identified form in the full survey dataset.
    – Ryan M Mod
    Commented May 4, 2023 at 16:47
  • 26
    Is the privacy concern over some kind of ability to match a user to a set of survey answers by using the timestamps of the answers and the timestamps of the Census badge? If so, then change all the badge timestamps to the time that the survey closed in each year, and only aware the badges for future surveys when they close, using the same timestamp for each year.
    – rgettman
    Commented May 4, 2023 at 17:17
  • 48
    "as the text says, it would be trivial to work backward into the process from there." This sounds like the wrong solution has been taken in my opinion then @Philippe . If finding out the user was is so trivial then the problem is more likely the underlying design and data stored, not the badge itself. Instead of addressing the root problem you've removed the entire feature. It's like being upset about the colour a wall is painted and instead of repainting the wall you took a sledgehammer to it and destroyed the wall. Smells like an XY Problem.
    – Thom A
    Commented May 4, 2023 at 17:50
  • 12
    If you are getting rid of this badge because of privacy issues, why do we still have Not a robot? That gives you data that so and so was at place X on day Y. Commented May 4, 2023 at 19:22
  • 22
    A simple "We have become aware of a significant privacy risk caused by awarding these badges. Unfortunately, due to the nature of the risk, we are unable to share additional details at this time." would have greatly lessened the negative reception of this post.
    – gparyani
    Commented May 5, 2023 at 7:06

14 Answers 14

118

There probably needs to be a better explanation here.

  1. Stack Overflow shares the full survey data sets. That means you can see a given user's responses for every question. In other words, if you've ever completed a survey, those responses are public (minus any direct links, like your name or email).

  2. Generally speaking, it's not hard to de-anonymize large data sets

    “Our results suggest that even heavily sampled anonymized datasets are unlikely to satisfy the modern standards for anonymization set forth by GDPR [Europe’s General Data Protection Regulation] and seriously challenge the technical and legal adequacy of the de-identification release-and-forget model,” the researchers from Imperial College London and Belgium’s Université Catholique de Louvain write in the abstract to their paper, which has been published in the journal Nature Communications.

The badge gave anyone doing de-anonymization a leg up. You knew who had and had not completed a survey (and what years you had done so since badges contain a date/time stamp). Removal at least raises that barrier some.

18
  • 11
    Really, so they asked sexual orientation and then published the results linked to the account? Am I understanding this correctly?
    – bad_coder
    Commented May 4, 2023 at 19:42
  • 16
    @bad_coder no, no literal direct link to account. but inference isn't impossible particularly when looking at surveys over the course of many years and public user profile data
    – Kevin B
    Commented May 4, 2023 at 19:55
  • 11
    @bad_coder No, but it's not impossible to infer which answer is whose either. This just removes a data point, making it harder to infer.
    – Machavity Mod
    Commented May 4, 2023 at 20:31
  • 2
    So it is possible to infer what a user answered to each question.
    – bad_coder
    Commented May 4, 2023 at 20:53
  • 8
    Thank you for the elaboration. This makes the badge removal more understandable.
    – Stevoisiak
    Commented May 4, 2023 at 21:48
  • 20
    "Stack Overflow shares the full survey data sets" - Why? My assumption is "so people can do their own analysis rather than being spoon-fed the company's conclusions from the data"; but because it's a survey, that doesn't actually work - the questions were asked, very specifically tailored, in the service of the company's analysis (and any particular agenda driving that analysis, e.g. to prove a point about how they are faring on "diversity" metrics or to argue that X change results in Y marginalized group feeling more "welcome" - which, you know, risks offensive stereotyping). Commented May 5, 2023 at 1:58
  • 32
    Aside from that, if what you say is true, then it sounds like whatever exploit is still theoretically possible as long as that dataset is public, and like removing the badge is only a mitigation. Commented May 5, 2023 at 2:11
  • 27
    Still a meh explanation, the better approach would be to remove as many questions leading to rare and sensitive data points as possible (particularly disabilities and gender identity are data points that are sensitive and lead to certain people to be identifiable for rare combinations, e.g. how many Dutch gay developers that work with F# aged 55-64 with a visual impairment do you think there are?). These are way more problematic than a badge indicating someone completed a survey imho.
    – Erik A
    Commented May 5, 2023 at 7:30
  • 10
    Well, some survey questions are really questionable and really shouldn't be asked at all. They don't serve any purpose. Since anyone can use SE sites, there is no barrier to entry based on someone's skin color, sexual orientation or similar. There is no point in trying to make the sites more "women friendly" or anything like that. Only thing that could matter can be site accessibility aspect, but for that asking whether someone has disability is not relevant to whether person with disability can successfully use the site or not due to their disability.
    – Dalija Prasnikar Mod
    Commented May 5, 2023 at 8:24
  • 2
    @DalijaPrasnikar I actually saw something worrying me the other day. A user with a profile picture of a woman posted an off-topic question (not about programming). I posted a cv-pls in SOCVR; got closed within a few minutes. Shortly after, the user had removed their profile picture. I can imagine that user, or similar ones, responding in the survey that their gender was a reason for being unwelcomed. They had a few other questions in the past; they may not have had their profile picture at that time. Anyway; this is going somewhere else. Commented May 5, 2023 at 8:30
  • 7
    @Andreasdetestscensorship I am not saying that some users may not discriminate others based on some traits or that some users won't perceive some actions as discriminatory again based on their traits. But, anyone can be as anonymous as they want or present themselves as they want. there is no worth insights (for running SE sites) that can be gained based on those demographic data.
    – Dalija Prasnikar Mod
    Commented May 5, 2023 at 8:37
  • 2
    @DalijaPrasnikar Yes, definitely, I do get your point, and I've scratched my head at many of those questions, myself. I've found them severely irrelevant, or completely devoid of context needed to correctly assess their answers. Commented May 5, 2023 at 8:39
  • 2
    That's why the badge wasn't awarded automatically and had to be accepted. To those who care the choice was already given.
    – EvgenKo423
    Commented May 5, 2023 at 10:59
  • 5
    @KarlKnechtel The bell can't be un-rung (deleting things from the Internet is hard), but it's only been in the past few years that people have realized de-anonymization is possible. As to why SO made the full data sets public, the SO Blog from 2021 explains why. Mostly they're letting other people glean patterns of data they didn't look for themselves. I mean, SO has always made lots of data about the network available
    – Machavity Mod
    Commented May 5, 2023 at 13:44
  • 27
    First, this is enlightening. I can now at least understand why this decision makes more sense. However, I'm still pretty annoyed at it. I opted in to receiving the badge for my participation. Stack Overflow still gets to keep my data, but I don't keep the badge I got from them. I feel like I'm being penalized for their poor data stewardship.
    – Makoto
    Commented May 5, 2023 at 15:53
90

The cookie that I got for filling out the annual survey was that I got a badge to go along with it.

If I'm not getting the cookie, why am I willingly giving you my participation?

Why should I rejoice for you taking away my hard-earned cookie, given that I gave you my time on the condition that I'd get it?

When we reviewed the process for this coming year’s survey, it was determined that assigning the Census badge added an unnecessary element of privacy risk. Although we understand this may be disappointing for those who spent time completing the survey to earn this badge, your privacy is of the utmost importance.

I'm guessing that this is the NSA knows you were at a bridge and knows who you were calling and for how long, but not the contents of it level of metadata concern we're guarding against?

That's...odd. There's nothing really in the survey that anyone could use to build a case against anyone. Like...I don't get it at all. Sure, there are questions about one's self-identity, but...so what?

This decision makes no sense, I disagree with it, and I do wish the company hadn't reverted its long-standing convention of not deleting badges versus retiring them.


In response to the latest update, in which there is a return of the badge:

I'm glad that the badge is coming back, but the thing is that I gave you this data on multiple different occasions, and I'm only getting one badge for it. The fact that there's a security issue in which I could be identified for my responses is...well, bluntly, not my fault; I trusted Stack Overflow to be good data stewards and they let me down, and now I have to lose the additional badges that I received for it.

I mean sure, this pacifies whatever security concern y'all surfaced (it's not like this wasn't already talked about well in advance of this rolling out, and it's not like doing this now isn't going to prevent bad actors from finding things out), but since I've received the only cookie you're going to give me, I'm simply not going to partake in the survey. If the company is willing to alter the deal they make with us and then just...not be forthcoming over the fact that they really dropped the ball on keeping this kind of data as secure as they could, then there's a lot less incentive to trust you with this data or provide it going forward.

7
  • 25
    in my first read of the beginning of the post, I interpreted "cookkie" as "HTTP cookie" and not "analogy for reward". That was a misinterpretation on my part, yes?
    – starball
    Commented May 4, 2023 at 16:57
  • 41
    @user: Yes. I'm not a web browser, after all.
    – Makoto
    Commented May 4, 2023 at 16:58
  • 33
    Yes, that's how I interpret it. They are very good at finding new ways of saying "f- you" these days. I hope that talent is marketable.
    – Cody Gray Mod
    Commented May 4, 2023 at 22:07
  • 14
    ChatGPT, please write a 1000 word essay on the impact that saying "f- you" to one's users has on the marketability of a product.... Commented May 5, 2023 at 2:00
  • 2
    Where do you cash in your badges anyway... do I have to travel to NY or can they send them by mail? I don't mind trading mine for badge-shaped noble metals. I also have lots of Unicoins left from 2014-04-01, I'd be willing to trade those for any other cryptocurrency.
    – Lundin
    Commented May 5, 2023 at 10:32
  • @cottontail If it were for the badge, it doesn't automatically mean the wrong reasons or answers. It's not necessarily Next, Next, Next, ..., Finish...
    – EvgenKo423
    Commented May 5, 2023 at 11:14
  • 3
    Cannot agree with this more. If the only cookie is gone, I can't see any point in risking my privacy for absolutely nothing when filling out future surveys.
    – ray
    Commented May 6, 2023 at 2:17
77

How is that possibly a privacy concern?

Yay, you can see someone filled out an anonymous survey! That's data we can absolutely abuse!

I mean, if that's data that needs to be protected, what about:

I could go on, as there's a load of badges that show facts about a user that are technically private...

I just don't see the usecase here. I always thought of the badge as a little "Thank you" for filling out the survey.


Seeing as how we could choose which site to get the badge on, how about this:

Reinstate the badges, and add an option to the survey to allow users to choose not to get the badge.

8
  • 45
    add an option to the survey to allow users to choose not to get the badge. Wasn't that always an option?
    – Lino
    Commented May 4, 2023 at 18:57
  • 28
    It was? Then what's the problem...
    – Cerbrus
    Commented May 4, 2023 at 21:28
  • 4
    The Yearling information is entirely public; a user's profile discloses join date and reputation total. Theoretically, stalking a profile could determine that someone qualified for Fanatic/Enthusiast, although it probably couldn't be done retroactively. But I think they're more concerned with protecting demographic information, such as an individual's self-identified race, gender etc. Commented May 5, 2023 at 2:05
  • 7
    I guess the reason it's because questions like gender or disabilities were included in the survey. There might be options that have been ticked by only a very few people. Together with the other answers that might be enough to identify some people maybe. And these questions were more sensitive than say your favorite editor. Commented May 5, 2023 at 4:54
  • 2
    I think the point is that it isn't anonymous at the point where you fill in something that stands out, or a combination of answers that stand out together.
    – Lundin
    Commented May 5, 2023 at 10:19
  • 4
    These badges are also all privacy risks, if the Census badge was. In fact every badge that can be linked in some way back to a user action is.
    – TylerH
    Commented May 5, 2023 at 21:10
  • 2
    To be clear: what I'm proposing is that, if these badges are also privacy risks, then certain public non-badge information on user profiles is also a privacy risk. Commented May 7, 2023 at 21:44
  • If it's about survey question answers (and maybe combinations of them) that are rare, that would also apply to badges that have been rarely awarded so the information associated with that could only be traced back to a small circle of users.
    – shaedrich
    Commented May 10, 2023 at 10:05
72

I think this is a bad idea, and I alluded to some of my reasons in the comments, but let me write up all my points:

The cat's already out of the bag

In the data dump, the badges table lists badges per user. This is available on archive.org, under the CC BY SA license. Even after removal, people can still find who earned a census badge and in which year. Arguably, it'd be a little bit more difficult, but if you're going to download the survey data and try to track down users, checking if they earned a badge in the data dump isn't much of a stretch.

Even if you were to ask archive.org to remove this from the data dump, anyone who retrieved an older data dump is free to redistribute it under CC BY SA.

The Developer Survey was never close to properly anonymous

Especially in recent years, the developer survey has included personal data with rare categories, such as gender, sexual orientation and disabilities. If someone combines this with age, country and programming experience, a significant fraction of users can likely be identified definitively.

As an employer, where you have data on earnings, company size and education as well, your odds of uniquely identifying specific users (and checking if they're wanting to move away from the work they're currently doing) is even higher.

While I applaud the attempt to protect our privacy, I don't see how this achieves anything but annoy users that had this badge previously and draw attention to the poor privacy of the survey.


What can actually be done

For the data that's available, pretty much nothing, unfortunately. It's been made available using an irrevocable open license, so while availability can be made worse, it's out.

For future data:

  • Reduce collection and sharing of special category data as much as possible

    There's special category data (per GDPR) in the survey such as ethnic background, mental and physical disabilities and sexual preference. This should be avoided whenever possible. Even though it might be interesting, collecting and sharing it is always a risk, and IMHO it's not worth the reward here

  • Chunk the export without providing links between chunks

    When providing the export, you could not provide it as a single file, but a separate file with basic demographics, programming experience and payment details, a separate file with technologies used, country and payment details, etc. Aggregate the data to counts of combinations, and make sure all counts for specific combinations are above a threshold to make sure it's never directly personally identifiable.

  • Be clear about the potential for deidentification

    When entering data into the survey, this risk should be communicated to the user, so the user can make an informed decision on which data to share. Even if you don't share the data, in the Netherlands we've just had a major data leak of a polling platform, so users should be prepared shared data might become public

As a reach, there's some promising research in the medical field on generative AI for privacy preservation (feed the data to a model designed to generate non-identical data that preserves correlations found in the data), but I personally wouldn't go that route.

This is all barring that there's not some much worse privacy problem than what was hinted at, such as the order of users that got a badge being identical to the ordering of rows in the dataset or something. And even then, we'd deserve some extra details such as if the data dumps were affected.

0
40

As I mentioned in the comments, I can't see how the removal of the badge "protects" a user, at least on a site like Stack Overflow. According to a recent post, where someone was querying why everyone was missing, the badge had been awarded about 250k times. The first survey was in 2011 (according to my Google-Fu), which means that there have been 12 surveys so far, however, according to the description of the Census badge on MSE (thanks toolic) the badge was introduced in 2017; that means it has been about for 6 surveys and therefore on average over 40,000 users respond each year.

Being able to identify a user from their responses and then cross-referencing that against a list of ~40k users would make identifying a single individual very difficult if not impossible.

On smaller communities, where the numbers are significantly lower, then I can completely understand you may wish to at least obfuscate who has the badge, but on Stack Overflow? Honestly, the reason you state you've taken this action makes no sense; this is using privacy as an "excuse", not as a reason. Whatever the real reason is, I would suggest that it is not what the post here says it is; privacy is not a problem here.

8
  • 1
    I don't believe this badge was ever offered to sites besides Stack Overflow.
    – Makoto
    Commented May 4, 2023 at 16:18
  • 1
    It was on other sites, @Makoto , the linked question actually references a couple it was on. Both those sites also now give 404 errors.
    – Thom A
    Commented May 4, 2023 at 16:19
  • 1
    Oh. That's hella convoluted. So they opened up the survey to other sites and allowed you to get the badge on any one site you wished. But you'd still have to be technically inclined to want to participate, let alone have it awarded.
    – Makoto
    Commented May 4, 2023 at 16:21
  • 4
    @Makoto Not "any site" - as far as I'm aware, we limited it to tech-focused sites only. So AU but not Cooking.
    – Catija Staff
    Commented May 4, 2023 at 16:38
  • 3
    Even on smaller sites it's not a problem. You would still need cross-reference the responses against the list of ~40k users who got the badge on any site, since as far as I know it's not possible to tell which responses come from someone who got the badge on a specific site. Commented May 4, 2023 at 20:03
  • If the survey has run every year beginning in 2011, that makes 13 surveys so far - including the one that's currently open. Commented May 10, 2023 at 16:05
  • @TobySpeight at the time of writing, which can be seen in the post (4 May), the 2023 survey wasn't out (it came out 8 May). 2011-2022 is 12 years, not 13.
    – Thom A
    Commented May 10, 2023 at 16:10
  • Ah, that makes sense. I didn't realise the 2023 survey hadn't opened at that time. Commented May 10, 2023 at 16:22
36

Maybe there are only a few people like me who did the surveys because I would get the badge. I would understand that the badge will not be given from now on, but why do you need to go such length to remove the previous given badges? I earned them by giving you my time for many years.

It feels like you lured me into doing something for you and I will get something in return. Yes, but now you are going to take it all back. What?

2
  • 19
    Perhaps they should have waited until after you completed this year's survey to say "Surprise! No badge this year. And oh yeah, we went ahead and removed all the census badges you previously earned, because reasons. Thanks for playing." It'll be interesting to see if this announcement affects the outcome of this years turnout.
    – Drew Reese
    Commented May 5, 2023 at 6:56
  • 2
    I literally knew about the badge removal after I completed the survey, checked for the badges, and found out the Census was removed even from the badges page. What a freaking surprise and disappointment, what a headless solution from SO! Commented May 8, 2023 at 22:24
25

This is one of those baby with the bathwater scenarios. Sure, someone could in theory tie into a NSFNET router and observe anyone's data or bank data etc. etc. the world is ending.

There is no need to be concerned here, no one is at risk of being identified by this information, and what would be disclosed anyway? The tech stack they use? Their sexual orientation? Their pay rate in local currency?

Actually, that does sound a little interesting doesn't it. Perhaps these are the types of questions that shouldn't be part of a publicized survey... What does mine, or anyone's, sexual orientation have to do with the way that I use, consume, or interact with Stack Overflow? Just because some third party asks this of you, or even offers to pay for that, doesn't mean there is an actual obligation to take part in it.

Moreover, the only path towards identifying users here is going to be by salary. Salary is a fine question to ask, but why leave it so sloppily exposed? Let me show you, dear reader, what I mean here:

72335,I am a developer by profession,"Employed, full-time","Hybrid (some remote, some in-person)",Hobby,"Bachelor’s degree (B.A., B.S., B.Eng., etc.)","Books / Physical media;Other online resources (e.g., videos, blogs, forum);School (i.e., University, College, etc);Online Courses or Certification",Technical documentation;Blogs;Programming Games;Stack Overflow;Online books;Video-based Online Courses;Online forum;How-to videos;Written-based Online Courses;Coding sessions (live or recorded),Coursera;Udemy;Other,25,18,"Developer, front-end;Other (please specify):","10,000 or more employees",I have little or no influence,Start a free trial;Ask developers I know/work with,Brazil,BRL Brazilian real,32900,Monthly,HTML/CSS;JavaScript;TypeScript,HTML/CSS;JavaScript;TypeScript,NA,NA,NA,NA,Vue.js,NA,NA,NA,npm;Yarn,Yarn,Visual Studio Code,Visual Studio Code,Windows,Windows,Git,Code editor;Command-line;Version control hosting service web GUI,,,NA,NA,Rocketchat,Rocketchat,Unsure,Stack Overflow;Stack Exchange,Daily or almost daily,Yes,Less than once per month or monthly,"Yes, somewhat",35-44 years old,Man,No,Straight / Heterosexual,NA,None of the above,None of the above,Yes,Independent contributor,20,Agree,Strongly agree,Disagree,Disagree,Disagree,Agree,Agree,1-2 times a week,1-2 times a week,1-2 times a week,15-30 minutes a day,15-30 minutes a day,Just right,DevOps function;Microservices;Developer portal or other central places to find tools/services;Continuous integration (CI) and (more often) continuous delivery;Automated testing;Observability tools,Yes,No,Yes,Too long,Neither easy nor difficult,82008

This is a single line dump of the csv publicly made available by Stack Overflow. What is the single most useful bit of identifiable information here? "Brazil,BRL Brazilian real,32900,Monthly". This clearly indicates the respondent is in Brazil, or works there, or is paid with that currency at the very least. Age used to be a public data point in profiles, so it must exist. This person is between the ages of 35-44. They work with certain tags as listed. Because of the country label here, you can probably sort out a good guess (even without the census badge). Now, there may not be anything salacious about this connection between user and data, as this is a heterosexual male (not trying to out anyone), but you can see how if they privately responded as something else that could be made public. Not only that, this is a rather narrow window to look at.

All of this extrapolation exists primarily through the use of the country of origin for salary. Why not simply anonymize that into a USD conversion? It would be extremely simple, you could even bucket it into a round number to make sure no one answers with their exact annual salary for example. In the example shown, the salary is $6642.30 USD per month, or $79,707.60 USD annually. Why not just bin this into the $80k per year group?

If salary were binned by annual USD, and the sexual orientation question were removed, there would no longer be anything of value to discern here about users anyway. This should be the primary concern to address, if anything, when examining issues with the data dump.

8
  • 19
    "What does mine, or anyone's, sexual orientation have to do with the way that I use, consume, or interact with Stack Overflow?" If an appropriate number of people of certain sexual orientations use/consume/interact with Stack Overflow, the company gets to make positive PR statements about their "commitment to diversity"; otherwise, they get to yell at the userbase and iterate on the code of conduct again until everyone is "nice" enough to avoid (supposedly) disproportionately scaring those users away (never mind providing any evidence for that theory). What's not to like? Commented May 6, 2023 at 0:36
  • 9
    If I sound cynical, it's because I've seen this sort of thing play out again and again, across the entire Internet, since 2011 or so, based on ideology I've seen promulgated on the Internet since at least 2007. Commented May 6, 2023 at 0:37
  • 1
    @KarlKnechtel This has been an issue for SO in the past, but it's not (to my knowledge) the motivation for current changes.
    – wizzwizz4
    Commented May 6, 2023 at 11:37
  • 16
    It's almost like identity politics is a weapon of exclusion no matter which direction you're holding it. Hmm.
    – Cody Gray Mod
    Commented May 6, 2023 at 11:37
  • Well, for example, there's the gender pay gap. This isn't necessarily limited to gender (especially if it differs from sex). This survey is (maybe not exclusively, but still) for research and scientific purposes. To see, where situations of marginalized people can be improved in our developer community (not just stackoverflow but the business sector as a whole), can actually help them if we are willing to do the necessary changes. Btw, just by writing this comment, you can make assumptions about me. Do with this information what you will.
    – shaedrich
    Commented May 10, 2023 at 10:15
  • 1
    @shaedrich - Gender is different than sexual orientation, although I understand the reference. My point was that sexual orientation doesn't really have anything to do with a person's ability to be a developer. I suppose if you were interested in making some inference based on the wage they earn based on their sexual orientation, then perhaps that could be information of value... but wouldn't somewhere else that deals with wages be a little better suited for that, like indeed or glassdoor etc.
    – Travis J
    Commented May 10, 2023 at 19:19
  • Of the set of questions, that seemed to me to be one which had the most personal information while at the same time being the least related to someone's ability to be a developer. Also to note, I did say that this is probably not even an issue, but if personally identifiable information was a problem, perhaps start with the information people would more than likely want kept private.
    – Travis J
    Commented May 10, 2023 at 19:21
  • @TravisJ I never drew a connection between gender and sexual orientation. With sex, I meant sex as opposed to gender, not the orientation. Sex or gender doesn't have anything to do with a person's ability to be a developer either. However, gender roles in society often favor male individuals to choose the path of a developer earlier in life. If that's tied to socialization, this would be reversed for some trans people. I've read another study a while ago, that claimed that the gender pay gap works differently for homosexual employees. That could be an interesting point to prove and disprove.
    – shaedrich
    Commented May 11, 2023 at 9:55
18

Now that the full survey is online and we see its contents, axing the badge to protect privacy makes even less sense because very little personally identifiable information is asked (compared to previous years). As far as I can tell, the most personal questions are age, country of residence and salary but apart from country, they can be left empty, so there's really no danger of losing one's privacy here. The fact that one has to agree to take the survey in the first place makes this a non-issue.

1
  • 1
    "they can be left empty" Previous years getting the Census badge was only optional anyway, so anyone concerned about privacy can opt out of getting the badge even if fields in the survey can't be left empty. Commented May 13, 2023 at 15:03
10

So, if we're going to get a Census badge, with everyone's badge issued on the same day, why can't we get back all the Census badges we earned, with them all issued on the same day?

3
  • That would give away, how many times you participated, which is still one information more than a general, vague badge. However, the whole badge thing and the official information given in that matter are a bit clandestine to say the least.
    – shaedrich
    Commented May 10, 2023 at 10:21
  • 1
    @shaedrich sure, it's a bit more information. But tbh if anyone was smart enough to figure out who I was from that information, they could probably work it out from many other sources anyway.
    – Nick
    Commented May 10, 2023 at 12:01
  • 3
    Yep, that's the problem with the whole badge thing and SO's reaction to it
    – shaedrich
    Commented May 10, 2023 at 12:05
9

Looks like someone posted a bug report question about this well before I did:
Census page link is broken?


The help page for the Census badge has not yet been restored: https://stackoverflow.com/help/badges/6644/census

It also doesn't appear to matter whether the userid slug is included or not.

https://stackoverflow.com/help/badges/6644/census?userid=11047824

Stack Overflow 404 page

5
  • 4
    "and the badge retired, so it shows up in the "retired" section" – no, it's not as well. Probably, will be fixed along with this one.
    – EvgenKo423
    Commented May 9, 2023 at 4:51
  • We are currently aware of this bug where the Census badge returns a 404 error page. Once the badge has been formally retired the page should render properly. Will post an update once it's resolved, thanks for your patience.
    – tanj92 Staff
    Commented May 10, 2023 at 19:54
  • "The census badges have been restored" My badges have not been restored - lost 4, got 1. So 4 are still missing. @tanj92 Will we get one badge per survey or will there be only one badge per user? Commented May 12, 2023 at 15:04
  • @IlikedtheoldStackOverflow see the may 5th update in the OP
    – Kevin B
    Commented May 12, 2023 at 15:07
  • 1
    I'm not a native English speaker but when I read "The census badges have been restored" then I expect that all badges have been restored. But I had a "4 x" Census badge and after the pseudo-restauration I only have a "1 x" Census badge, where it should be "5 x" by now. That's not a full restauration, that's a compromise, a downgrade. @tanj92 So "The census badges have been restored" is marketing speech for "The census badges have been downgraded"? Commented May 15, 2023 at 8:22
8

I really knew that there will be no badge after I completed this year's survey. Can I get some compensation for spending my time aimlessly? Also, removing the badges (without getting any allowance from the user) means downgrading his/her reputation, which basically means showing big disrespect from SO to the users who spent lots of time, concentration, and effort on filling in these and giving datasets to the SO for many years.

Badges are much more important than some mythical privacy. Please return the badges back!

AFAIR, there was an option like "I don't want to get the badge" at the end of the survey. Why not keep it as is and break the system for the means of breaking and disrespecting your community?

One of the worst solutions in the history of SO. Totally disappointed!

2
  • 6
    One point that I don't understand, why are we restored only 1 badge back, even if we had multiple badges before? They did their data redaction by making the badge page inaccessible. I can't see the difference between keeping 1 badge or multiple badges now.
    – ray
    Commented May 9, 2023 at 14:09
  • 1
    @ray Exactly as I wrote, SO's intention is to show the hugest possible disrespect to the users, unfortunately :( Commented May 10, 2023 at 13:17
6

I feel like there should've been at least a pop-up with a message that for this year's survey will not grant you a badge, but I still see some of the points made. Will miss that badge tho.

1
  • 2
    One could argue, they forgot to do that but chances are, that was made on purpose. Will we ever know? Probably not.
    – shaedrich
    Commented May 10, 2023 at 10:20
6

I have to say that the Census badge is the biggest reason that I fill the survey, after filling out so many surveys, you decided to remove my badges? Feels like all my effort is worthless now.

3

This same info is still available in the data-dumps. I sent an email about it

I see that my request for removing the Census badge data from SEDE went through (or at least- it seems it did). But what about public data dumps? https://archive.org/download/stackexchange those don't seem to have been updated yet. The timestamps say "06-Mar-2023". I assume you also want to strip the Census badge data from there as well? If so, make sure to update all the sites where the Census badge exists- (which is more than just Stack Overflow).

The reply I got was:

While SQL is regularly updated to remove old data (the update you noticed was an automatic one), the public data dumps are not maintained, so no information is removed from them.

Why'd you make a big deal about the safety of users and lower the resolution about the badge information if you're not going to perform that same information resolution lowering everywhere else you have authority over? Anyone who wants to violate whatever safety you're trying to protect users against can trivially get this data from the data-dumps unless you do something about it.

2
  • 2
    Data dumps is not a location where SE "has authority over": Stack Exchange ceases to have control over data dumps after they get published. While they can pressure the Internet Archive to take down a given upload, they can't stop people from simply sharing previously-downloaded dumps. It's for the same reason that, for example, they don't remove attribution from data dumps when a user requests to be dissociated from a post.
    – gparyani
    Commented Oct 23, 2023 at 7:55
  • @gparyani yes, I know that once someone downloads something, there's no control over that, and that people can redistribute as they wish, but at the very least I'd have expected that they could update the source to mitigate. and this isn't a case of concern for the privacy/safety of one case of one user.
    – starball
    Commented Oct 23, 2023 at 7:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .