46

I understand my answers will now be used by OpenAI to produce LLM based AI(s), per "Our Partnership with OpenAI".

OK, but I licensed my answers to SO under CC BY-SA.

Will the output generated by the resulting LLM AI also be properly attributing the sources from which it generated the output, as required by CC BY-SA?

10
  • 13
    "I understand my answers will now be used by OpenAI to produce LLM based AI(s)," wait now?! Were you under the impression they weren't used before?
    – VLAZ
    Commented May 9 at 9:11
  • 5
    @VLAZ now it's official, so it reaches a different level Commented May 9 at 9:13
  • 6
    You may want to read the TOS, because you gave Stack Overflow an additional license to basically do whatever they want. Commented May 9 at 9:13
  • 3
    @SilentCloud but the situation of the content posted being used to train LLMs doesn't change. Yes, the OpenAI deal does change some things. But not this. Trying to pretend that only 3.5 years after ChatGPT was released would content from SO be used to train it is either an attempt to twist the narrative or pure and maybe even wilful ignorance. It sounds exactly like an early version of the post tried to paint ChatGPT. And if one is trying to claim an LLM is bad, it's best not to be like it.
    – VLAZ
    Commented May 9 at 9:16
  • 2
    How can you attribute the output of a probability model that was generated using millions of distinct inputs? It seems impractical at least, if not impossible. Commented May 9 at 18:12
  • 2
    You make a question in a website, 10 years later, they are feeding it to a carbon burning hype machine that very few people take actual value from. Commented May 9 at 18:33
  • @PresidentJamesK.Polk They could plausibly post-process LLM output with a RAG that adds links/citations to the oldest sufficiently similar content in the training database that is publicly accessible. This would help verifiability of the output, and help users to figure out how to attribute stuff or whether any part of the LLM output is novel (which is the case sometimes for GPT-4 and similar models). Of course, defining "sufficiently similar" would be difficult, but it would improve the value of the output potentially by quite a bit in many use cases.
    – Polytropos
    Commented May 9 at 22:48
  • @Polytropos: There is plenty of jurisprudence that copyright violations require more than a de minimis use. The problem now is that AI's depend in miniscule ways on billions of inputs, none of which exceeds the de minimis threshold. This is not hard to prove: train another model omitting one input, and its answers are still virtually the same. Hence, there is no legal link. Rinse&repeat.
    – MSalters
    Commented May 10 at 14:27
  • @MSalters I would never ever argue that they are by copyright required to attribute training data that has identifiably been the source of some idea. But it would be nice to do, improve the usability of their product, prevent potential plagiarism by model, and seems much more doable than implied by the comment I answered to. Indeed, my understanding is Copilot does something a bit like this by suppressing outputs that have too much overlap with snippets from GitHub.
    – Polytropos
    Commented May 11 at 7:07
  • @PresidentJamesK.Polk "How can you attribute the output of a probability model that was generated using millions of distinct inputs? It seems impractical at least, if not impossible." They could simply decide not to do that if it's too much work and not use the CC material for training. Otherwise a link to a website with lots and lots of names and links seems also plausible somehow. I would be happy with a generation of say 5-10 of the most important authors and links to the original resources for every output in some footnotes as a compromise. Or simply: CC attributed to the World. Commented May 15 at 20:39

2 Answers 2

25

At least in the US, IMO, yes. The output of AI generation cannot be copyrighted, because it's not the work of a human.1 Thus, all such AI generation output is in the public domain, which means that licensing it really doesn't exist as a concept. However, it will be a derivative work of CC BY-SA content,2 so the license would still apply, due to the fact that it's a derivative work. Basically, the copyright holder of the original work retains rights in the derivative work, such that permission is needed from both the original copyright holder and the creator of the derivative work. The original copyright holder has granted a CC BY-SA license for the original work, so complying with that would be required.

As to the output of what will result from Stack Overflow's partnership with OpenAI described in "Our Partnership with OpenAI" being properly attributed, Rosie, the Stack Overflow employee who posted the above announcement on Meta Stack Exchange, has said:

Attribution is something that we believe strongly in. Having credit attributed is a non-negotiable for us, and is a critical part of any and all partnerships of this type. There aren’t specific details yet because the work is just starting, but making sure attribution is happening (in a license-compliant way) is a commitment we require and have received from our partners. This is the very heart of socially responsible AI.

Note that basically all existing posts on the Stack Exchange Network have already been used by companies to create LLM based AIs, and those posts have been used from the beginning, just not with the company's participation or explicit permission. The partnership which the company has announced gives the company a bit more control over what happens and allows the company to be more proactive about making sure that attribution happens. Without such a partnership, the company has no legal right to try to enforce that there is attribution, because only the copyright holder (i.e., individual contributors, or their agent, which Stack Overflow is not) has legal standing to enforce the license agreement.


  1. Per multiple rulings by the U.S. Copyright Office, as discussed in Copyright Registration Guidance: Works Containing Material Generated by Artificial Intelligence.
  2. In my opinion, the AI model is a derivative work of the training data and the output is a derivative work of the AI model. There are counter-arguments to this, with most AI-generation companies claiming that using basically anything as training data is "fair use" (a US copyright law concept). "Fair use" in the US is something that can only be decided in a court case. How things ultimately turn out is up to the courts and/or new legislation which directly address these issues.

Note: I am not a lawyer. This is not legal advice. The above is largely just my opinion and contains various assumptions on my part. The area of law where copyright intersects with AI generation is currently being litigated in multiple lawsuits, even when just looking at the US, and will need to happen in nearly every jurisdiction. I would not expect resolution in this area of the law for many years, possibly more than a decade.

13
  • Logic dictates, AI generated products are the products of the owner of the AI.
    – peterh
    Commented May 9 at 2:12
  • 11
  • 2
    Overall, as I said in my answer here, this is a very new area of the law and courts will need to make rulings that are specifically on these points and/or legislators will need to modify laws to account for these new areas.
    – Makyen Mod
    Commented May 9 at 2:59
  • 6
    In this opinion-blog from Creative Commons, they argue that training AI falls under fair use, and that means that attribution is likely not necessary: Fair Use: Training Generative AI Commented May 9 at 9:16
  • @Makyen NB: Copyright can now, in some cases, be applied to generative AI works in a limited capacity. The AI-generated portion (e.g. the particular organization of source content) is sometimes eligible, where the source content itself used in the AI work is still not copyrightable. tl;dr - a 100% genAI book cannot be legally copied and re-published verbatim without permission, but you can take 100% of the contents of that book, reorder those contents, & legally republish that
    – TylerH
    Commented May 9 at 16:16
  • "because only the copyright holder (i.e., individual contributors, or their agent, which Stack Overflow is not) has legal standing to enforce the license agreement." If that's what your lawyers told you I'll defer to them, but SO as a sublicensor of the original authors would at least have standing to enforce its own TOS against the harvesters. So it really isn't helpless here and I'm sure many of us would've preferred that that route had been tried first. Commented May 11 at 15:52
  • @EmperorEto Yes, the company can enforce its ToS, but being able to enforce that has nothing to do with being able to enforce the license of the contributions by individual users. That SE also holds a license to use the data doesn't give them any rights to try to enforce the license which other entities obtain, even when the other entity gets the data from SE. When another entity gets the user contribution, a completely separate, but identical, license agreement between that other entity and the author comes into existence to which SE has no part and no legal ability to enforce.
    – Makyen Mod
    Commented May 11 at 16:11
  • @Makyen, I don't disagree that SO can't make a copyright claim on behalf of the authors. However I'm disputing the notion, implied or otherwise, that SO had no other legal recourse to protect its users besides inking a deal with OpenAI. Commented May 11 at 17:53
  • @EmperorEto What legal recourse was available for SE to do that? As far as I'm aware, there's nothing. SE can have issue with some methods of the other entity obtaining the data, particularly if the entity bypasses existing restrictions on access, but there are plenty of existing methods for them to get the data. Once they have the data through a valid method, they have a CC BY-SA license, which SE has no standing to try to enforce, just like any other copyright claim. Even if the access method wasn't valid, they have a valid license as long as the license of the source was valid.
    – Makyen Mod
    Commented May 11 at 18:35
  • I don't want to get into a legal argument here, esp. since it's a live issue. But in a former life I was an IP lawyer and I have read the SO public network TOS s.6 and I don't think your assessment of who's licensing what to whom is quite ight. If SO's lawyers advised otherwise I'm certainly not trying to offer you or SO contrary advice. But for the sake of the users reading this, I think there were avenues SO could've pursued to protect its users' rights (and its own) short of contracting with OpenAI. IMO it was a business decision to do so - the wisdom of which I'm in no position to judge. Commented May 11 at 18:55
  • @EmperorEto regarding the "license enforcement", that's what SE told us about reporting scraper (outside of OpenAI deal).
    – Andrew T.
    Commented May 12 at 15:32
  • Thanks @AndrewT! Yes it's clear their lawyers have taken the position they have no standing to make a copyright infringement claim for user-generated content. In my opinion this is a cop out though because there are other types of claims available as well as a copyright claim on the site as a whole as a heavily moderated collective work, which SO does own and licenses. Again this was a business choice, albeit one informed by legal calculations including the cost of litigation and weighing the odds of success. Commented May 12 at 16:05
  • Just one last thing I want to add is I'm sure SO is not the only site in its class to take this view. I expect Wikipedia would too. They have legitimate self interest in not claiming a copyright on the collective site and not taking on the role of enforcement including preserving s230 protection and dmca safe harbor. It's just much more complex than "we don't have standing" though that makes for an easy if incomplete public explanation. Commented May 12 at 16:49
6

Agree with most all of Makyen's answer. Specifically regarding whether the results are "derivative works", though (the question at hand):

How things ultimately turn out is up to the courts and/or new legislation which directly address these issues.

Agreed as well, and I can see this going one of two ways in the courts (and probably both ways, since different laws and precedents will exist in different locales).

First, yes, there's the "obvious" conclusion that if one information-system uses data from another information-system, then it is a derivative work.

(Standard "I am not a lawyer" disclaimer here, but from my discussions with attorneys on IP negotiations) Then there's also the concept of "residual knowledge" that might be applied here. As humans, we learn things over time that ultimately enter our residual knowledge.

Compare two situations:

  • First, take a common Stack Overflow example of knowing how to set an environment variable in Linux via ~/.bashrc. If I was answering a question about this, I'd be doing so from my residual knowledge, not from looking it up in Google (and certainly not ChatGPT! ;-). I probably wouldn't remember where I initially learned it (although I think I do remember).

    In my Stack Overflow answer, there would be no expectation that I would need to attribute that knowledge to a source.

    There are, as a result, literally millions of answers here on Stack Overflow that do not have any attribution to a source. That's expected and normal.

  • However, when I was answering a question about why VSCode can only operate as the default Windows Subsystem for Linux user, I Googled, searched GitHub, and found some existing issues that had information relevant to the problem. After confirming those by testing on my own system (and looking for a workaround), I posted an answer with attribution to the GitHub issue.

    In that case, an attribution would be expected and required.

The question that has not yet been determined is whether the result of training an LLM is equivalent to residual knowledge or not. When OpenAI claims "fair use" on training on web-based content, this is likely the core principle they are using. The input is so extensive, and the results of an LLM's output are such an amalgamation of that input, that it is far more like residual knowledge.

  • Note: Given the existing definition of residual knowledge, anything stored in a computer system would not be residual knowledge, because it must be in the human-brain. However, if this was being decided in court, OpenAI (et. al.) would be likely to argue that LLMs are different enough from traditional information-systems that the prior definition should not apply.

And perhaps as with human knowledge, the result will be "it depends". If the LLM determines that most of its output for a given prompt came from a particular source, it would attribute it, and otherwise it would synthesize a result.

Who knows? It might be argued that the "hallucinations" we hate so much are signs that the LLM synthesized the result based on residual knowledge. If an LLM result were more like a "search result", it would be closer to the original.

5
  • 1
    My understanding was the current LLMs do not have a "link" between the source and the result -- the token predictor doesn't "know" why it's predicting what it predicts. If this understanding is correct how is it going to attribute anything?
    – chx
    Commented May 10 at 14:07
  • @chx Right. A GPT works on probabilities. At each step, it randomly chooses a token from a small pool of tokens which give a high score if chosen as the next token in the token stream it's generating. And those scores are derived from literally millions of token sequences in the training data. However, token sequences that have high scores are quite likely to exist as sequences in the training data. So when the AI generates a chunk (sentence, paragraph, a function in code, etc) it just needs to do a search in the current SO data to see if it can find a near match.
    – PM 2Ring
    Commented May 11 at 4:54
  • Obviously, this process isn't foolproof, and it requires fine-tuning so it doesn't waste effort looking for attribution for common knowledge. But IMHO it's a hell of a lot better than the current situation, where LLMs are using our content and giving zero attribution.
    – PM 2Ring
    Commented May 11 at 4:54
  • If you believe they'll attribute anything then I have a bridge to sell you.
    – chx
    Commented May 11 at 10:21
  • It's all not settled in courts, so we simply don't know and need to wait. My naive approach would be that even fair uses can become unfair uses if done automatically on a very large scale. And commercially profiting from content creators without somehow letting them participate in the profit is not ethical. What would OpenAI be without the content. A human can surely do the same but only to a very limited extent, one might make a distinction there in the future. And if the current laws don't say that, maybe they can be made to say that. Content creators will be thankful. Commented May 15 at 20:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .