24

I just read (an old thing already?) about the ruling for the case hiQ vs LinkedIn (LinkedIn Vs. hiQ Ruling Casts A Long Shadow Over The Tech Industry for one of the many articles about this). It seems that LinkedIn (now owned by Microsoft) will have to allow hiQ's scraper full access to public user data.

I don't want to debate the merit of this (security implications, for example) but how will this ruling (if confirmed in appeal) change Stack Overflow's privacy policy? If I understand correctly (A Terms of Service update restricting companies that scrape your profile information without your permission) then the user profile data is now NOT available from SE API unless explicitly allowed by users and only for specific commercial companies. In the same post we also read that:

Profile Content that is NOT available via the Stack Exchange API ("Personal Profile Content") cannot be used for any commercial purpose, individually or in aggregate, or be republished without the explicit consent of the author of such Personal Profile Content or the explicit consent of Stack Exchange.

Does the sentence affect this rule or "...explicit consent of the author..." covers this case from a legal point of view (it's available, yes, but author has to give you the permission)? What about Developer Story data (which is pretty similar to LinkedIn content)?

Will this sentence change in any way SO/user's freedom to share or not profile information with these automated data mining entities? Note that (citing from above article):

In the course of this case, hiQ offered evidence that LinkedIn was developing its own version of hiQ’s talent-monitoring management software. The judge saw this for the attempt to stifle competition.

Well, that's exactly the purpose of Developer Story, then...

23
  • Not sure what you mean by the user profile data is now not available from SE API unless explicitly allowed by users...? Everything that's not deleted that you've posted on Q&A (be it questions/answers/comments) are available as well as any information you've put in your "about me" and profile.
    – Jon Clements Mod
    Commented Oct 12, 2017 at 9:21
  • 1
    @Jon the thing is that right now, Stack Overflow prohibits companies from scraping that information from the page (as opposed to the API) without their express permission and that may now become an illegal practice if that ruling holds.
    – Pekka
    Commented Oct 12, 2017 at 9:22
  • @Pekka oh is this focused about scraping? The information is available via a free API - I thought the whole issue about that was people arguing that linked in (public profiles) was public data and if they could access it another way without scraping etc... etc...
    – Jon Clements Mod
    Commented Oct 12, 2017 at 9:24
  • @Jon I think it was about scraping (and most headlines etc. seem to confirm that) but haven't read in depth about the case!
    – Pekka
    Commented Oct 12, 2017 at 9:25
  • I'm not a lawyer or speak for SO, but I'd have thought something like "We don't want people to scrape our site for user profiles (why should you use our resources for us to generate HTML and you have all the extra work of deciphering it), but if you want the information we happen to be providing, then access it here in a nice condensed JSON format"... shrugs
    – Jon Clements Mod
    Commented Oct 12, 2017 at 9:25
  • 6
    Oh wait... I'll shut up... There's a very obvious block of text in the MSE post saying: Profile Content that is NOT available via the Stack Exchange API ("Personal Profile Content") cannot be used for any commercial purpose, individually or in aggregate, or be republished without the explicit consent of the author of such Personal Profile Content or the explicit consent of Stack Exchange.
    – Jon Clements Mod
    Commented Oct 12, 2017 at 9:27
  • Ahhh... I think that's what the block I quoted above was trying to say... "Take whatever you want from the API - but don't try and take that data and then scrape developer stories" - that's pretty much the only things not covered by the API... so yeah... interesting :)
    – Jon Clements Mod
    Commented Oct 12, 2017 at 9:32
  • From what I understand, LinkedIn wants to do exactly what SO does to scrapers now: Remove access to ostensibly public data. If the court decides in the same way that the injunction ruled, this may be precedent that SO can't forbid scraping of anything publicly accessible and can't put any blocks to do so in place.
    – Magisch
    Commented Oct 12, 2017 at 11:36
  • @Magisch yes, that's what I'm worried about. Manual scrapping/cross referencing is expensive...I don't want an automated bot to send me e-mails or facebook messages or xyz or because my cheap data have been collected by a bot and sold for 1 cent! Commented Oct 12, 2017 at 11:40
  • 1
    They can't forbid it on basis of the "Computer Fraud and Abuse Act" (CFAA), but there's nothing forbidding SO from making it more difficult to scrape the public content.
    – Cerbrus
    Commented Oct 12, 2017 at 12:09
  • 1
    If I understood the sentence correctly they (MS) have to REMOVE any technical impediment or limitation. Yes, you may play near the edge between what is legal and what not or you may add recaptcha here and there but I don't think it can be a long term solution Commented Oct 12, 2017 at 12:11
  • The article really lacks specifics. I doubt they can do anything against, for example, (aggressive) rate limiting.
    – Cerbrus
    Commented Oct 12, 2017 at 12:32
  • 3
    IANAL, but those are 2 separate issues the ...be republished... message is based on copyright laws and it states what the copyright owner allows you to do. There are no laws against scraping it doesn't make any sense, you can't say to someone what to do privately with something you publish publicly. Microsoft tried to abuse the CFAA and failed. The part of SO message that states cannot be used for any commercial purpose has no legal merit. Someone who scrapes the site doesn't have to accept any agreement and as long as he follows copyright laws can do anything he wants.
    – Oleg
    Commented Oct 12, 2017 at 17:20
  • 1
    @AdrianoRepetti Sure but that's only after you accept and are legally blinded by a license agreement. Just putting something on a public webpage doesn't bind the people who view it that would be undemocratic. It's like me stapling a "give me 5 bucks if you see this" on my forehead. Does anybody who sees it owes me 5 bucks? If SO's statement had any legal merit they could sue google for indexing it's pages and making money by selling ads.
    – Oleg
    Commented Oct 12, 2017 at 19:04
  • 1
    @o11c Not yet, it's only about HiQ and LinkedIn but we might get there ;). It does make a DDOS attack by HiQ on LinkedIn legal though.
    – Oleg
    Commented Oct 16, 2017 at 22:54

1 Answer 1

5

I am not a lawyer (nor do I speak for Stack Overflow; I'm just a user).

Here's the court's final paragraph on the public interest followed by the conclusion (the ruling) for the case you are referring to - HiQ Labs v LinkedIn Corporation:

For present purposes, the Court concludes that the public interest favors hiQ's position As explained above, the actual privacy interests of LinkedIn users in their public data are at best uncertain. It is likely that those who opt for the public view setting expect their public profile will be subject to searches, date mining, aggregation, and analysis. On the other hand, conferring on private entities such as LinkedIn, the blanket authority to block viewers from accessing information publicly available on its website for any reason, backed by sanctions of the CFAA, could pose an ominous threat to public discourse and the free flow of information promised by the Internet.

IV. CONCLUSION

In sum, the Court concludes that: (1) the balance of hardships tips sharply in hiQ‟s favor; (2) hiQ has raised serious questions going to the merits of its UCL claim and the applicability of the CFAA; and (3) t he public interest favors a preliminary injunction. For these reasons, the Court GRANTS hiQ's motion for a preliminary injunction and ORDERS as follows:

  1. Defendant LinkedIn Corporation and its officers, agents, servants, employees, and attorneys are hereby enjoined from (1) preventing hiQ's access, copying, or use of public profiles on LinkedIn's website ( i.e., information which LinkedIn members have designated public, meaning it is visible not just to LinkedIn members but also to others, including those who may access LinkedIn's website via Google, Bing, other services, or by direct URL) and (2) blocking or putting in place any mechanism (whether legal or technical) with the effect of blocking hiQ's access to such member public profiles To the extent LinkedIn has already put in place technology to prevent hiQ from accessing these public profiles, it is ordered to remove any such barriers within 24 hours of the issuance of this Order.
  2. Defendant LinkedIn Corporation and its officers, agents, servants, employees, and attorneys shall withdraw the cease and desist letters to hiQ dated May 23, 2017 and June 24, 2017. LinkedIn shall refrain from issuing any further cease and desist letters on the grounds therein stated during the pendency of this injunction.
  3. This preliminary injunction shall take effect immediately.
  4. No bond shall be required, as Defendant has not demonstrated it is likely to be harmed by being so enjoined. This order disposes of Docket No. 23.

IT IS SO ORDERED.

LinkedIn tried to pin the majority of its defense on the CFAA, which outlaws unauthorized access - or misuse of authorized access - to a computer system, and the court smacked that down (rightfully) as far too broad an interpretation of the CFAA. If it had granted LinkedIn's CFAA argument, then half of the people on the internet would suddenly be guilty of a federal crime.

Key to LinkedIn's loss in this case were that

  1. They blocked HiQ on all levels from accessing their site and data after HiQ started scraping and
  2. LinkedIn's argument for privacy was undermined in several points by the fact that LinkedIn has previously sold its user data to a 3rd party without disclosing that to users, and LinkedIn could not show substantial user complaints about privacy, let alone user complaints about privacy related to HiQ's scraping.

Furthering HiQ's claim is that LinkedIn is literally a website where members go to display their personal data publicly and privately; that's the point, the function of the website. LinkedIn blocking a competitor's access to that was anti-competitive and therefore likely violating the Sherman Act and a host of other anti-trust laws. I doubt the same argument could be made by a scraper trying to sue Stack Overflow; we don't come to Stack Overflow to sell ourselves, we come to Stack Overflow to ask and answer questions. The fact that we have many users who have totally anonymous and blank user profiles lends itself heavily to that.

There were other small claims on both sides that the court also struck down (HiQ even tried to argue promissory estoppel WRT to public user profile data - arguing that, essentially, users were suffering from LinkedIn blocking HiQ, because LinkedIn "made a promise to users that they controlled their profile privacy settings").

With regard to the Developer Story... that's kind of exactly like LinkedIn's profiles, so this ruling may well affect Stack Overflow's ability to restrict scraping of those. But that depends on it being brought to court, and this ruling being used in a plaintiff's argument. It's likely that Stack Overflow will continue it's prohibitive policy until such time as it is challenged in court and found to be in violation of existing laws.

12
  • It says "date mining"?
    – Braiam
    Commented Oct 12, 2017 at 19:13
  • 12
    @Braiam You didn't know LinkedIn was the professional version of Tinder? Commented Oct 12, 2017 at 19:23
  • @Braiam Maybe a transcription error in the court statement, or maybe they actually meant dates. One of the discussion points during the case was the concern that users might have date-specific information that could be accurate at a certain time and not accurate at a different time, and could be harmful to the user if that information were associated with them at an incorrect time.
    – TylerH
    Commented Oct 12, 2017 at 19:26
  • @TemporalWolf Unfortunately that's more true than you might think... I have seen many instances where women have spoken out about harassment from men on LinkedIn, sending them inappropriate messages over the network's PM system.
    – TylerH
    Commented Oct 12, 2017 at 19:27
  • 5
    we don't come to Stack Overflow to sell ourselves People using the Jobs portion of the site are, and that's pretty integral to the issue at hand, since the jobs/dev story portion of the profile is precisely the content that SE states other organizations are limited in their ability to scrape. The non-jobs related stuff is all public personal stuff that SE includes in the API and has no restrictions on the use of.
    – Servy
    Commented Oct 12, 2017 at 19:27
  • 2
    @Servy That's true, and different aspects of Stack Overflow are probably subject to different considerations under the law; Jobs, like Developer Story, are separate products from the Q&A portion, so likely the same stipulation in my last paragraph would apply to Jobs. I've never used the API so I don't know what kind of data is excluded from it, but again this case was specifically about allowing HiQ the ability to scrape, not access via an API.
    – TylerH
    Commented Oct 12, 2017 at 19:32
  • 2
    @TylerH So the starting place here is to consider what public personal information SE has. There's your username, all of the info in your user card, and the about me section of your profile. All of that is made readily available to anyone, and companies are not prohibited from using it in any way that they want. But the stuff on a jobs profile, developer story, etc. are not made as readily available, and SO's policy places restrictions on how people are allowed to use that information (non-commercial use). Those restrictions sound very similar to this case here.
    – Servy
    Commented Oct 12, 2017 at 19:35
  • 2
    @Servy I agree, hence the last paragraph of my answer
    – TylerH
    Commented Oct 12, 2017 at 19:37
  • 5
    It's important to note that this is a preliminary injunction. When it was issued (August 14, 2017), the court had not yet ruled on the merits of the case, or in fact even heard the case. Its effect is limited to putting LinkedIn at risk of contempt of court in the event that they should fail to behave as ordered. Now, if HiQ ultimately wins the case then the final ruling will likely have implications for SE policy and procedures. What exactly those would be is something SE's counsel will need to consider. Commented Oct 14, 2017 at 4:00
  • 1
    @Servy one thing that I find important to note here is the fact that LinkedIn was actively blocking HiQ, and that was the main point of contention. Stack Overflow has set "rules" for commercial use of the information, but as far as I know they aren't actively blocking anyone from viewing information that is publicly available. So, so long as Stack Overflow doesn't actively block companies that aren't following the rules but instead take legal action against such companies, they should be in the clear. IANAL.
    – user4639281
    Commented Oct 14, 2017 at 23:04
  • The point being that Stack Overflow cannot prevent companies from viewing information that is otherwise viewable by the public. That doesn't say anything about the use of that information by said companies, which may or may not be able to be restricted through licensing or other means.
    – user4639281
    Commented Oct 14, 2017 at 23:06
  • @TinyGiant Sure, so the question simply comes down to whether or not SE actually enforces its own rules. If it creates the rules but never actually does anything to enforce them, then maybe that's not a problem, but if it actually enforces them, then it would be in the same boat as Linkedin (whatever that boat ends up being, when the case ends).
    – Servy
    Commented Oct 16, 2017 at 13:11

Not the answer you're looking for? Browse other questions tagged .