-4

I know this sounds counterintuitive, who would out themselves? We are all aware of the Temporary policy: ChatGPT is banned. However, this post has to do with the plagiarism aspect of using ChatGPT.

While it is a bannable offense to post content from ChatGPT (the first offense seemingly being a week suspension at the time of writing), it seems we would benefit from discussing the plagiarism involved in posting material from ChatGPT.

Posting entirely from ChatGPT is one thing, and a week suspension is a nice slap on the wrist for that which will hopefully educate users on the policy. However, copy pasting content from another tool without citing it is plagiarism.

Should there also be additional penalties for users plagiarizing ChatGPT, meaning that they posted the content which was not authored by them without citing its source?

For example, let's say I were to include a paragraph from ChatGPT. I asked it to write an example blurb for meta Stack Overflow and it gave me this:

As Stack Overflow continues to grow, effective question tagging is becoming increasingly important. Tagging helps users find the information they need and ensures that experts can quickly locate questions within their area of expertise. -ChatGPT Feb 13 Version.

Shouldn't this be cited (proper or different citation can be debated), as sourced from ChatGPT, in order to avoid plagiarizing the tool or its sources?

OpenAI's user policy does state that it should be, as was also linked in Makoto's answer in the original banning post, https://openai.com/api/policies/sharing-publication/#content-co-authored-with-the-openai-api-policy.

Creators who wish to publish their first-party written content (e.g., a book, compendium of short stories) created in part with the OpenAI API are permitted to do so under the following conditions:

  • The published content is attributed to your name or company.
  • The role of AI in formulating the content is clearly disclosed in a way that no reader could possibly miss, and that a typical reader would find sufficiently easy to understand.
  • Topics of the content do not violate OpenAI’s Content Policy or Terms of Use, e.g., are not related to political campaigns, adult content, spam, hateful content, content that incites violence, or other uses that may cause social harm.
  • We kindly ask that you refrain from sharing outputs that may offend others.

Outright 100% reproduction of ChatGPT should still be banned in my opinion.

That said, it seems it is proper to cite this type of content, and it would also make it easier to recognize when posted. Perhaps citing it could also lead to a situation where partial inclusion was perhaps not as problematic as when authors attempt to claim they wrote the entire response.

21
  • 3
    It's not our (or SO's) place to enforce policies from other sources. That said, we can and should enforce our own. Given that these users are suspended for a lengthy amount of time and have all of their chatgpt posts deleted, I'm not sure what an additional penalty would really do
    – Kevin B
    Feb 27 at 20:07
  • 3
    Output from ChatGPT has no copyright, as only a human can produce a work with copyright, a computer ML program cannot, nor can a monkey. The most OpenAI can do is kick you off their platform, they don't own any copyright to the output of ChatGPT. Plagiarism is stealing someone else's work, but ChatGPT legally isn't 'someone'. Feb 27 at 20:08
  • 1
    @KevinB - It isn't really enforcing another sources policy, so much as noting that it isn't just free form content. Also, week isn't really a lengthy amount of time.
    – Travis J
    Feb 27 at 20:11
  • @vandench - Note that the requirement is to cite the source and role, not necessarily claim ownership of the content.
    – Travis J
    Feb 27 at 20:12
  • 1
    Let me put it another way. You're effectively suggesting users who don't openly disclose that they are violating our policies should receive a stronger penalty, yea? doesn't that sound a bit silly? I mean, if we ever dropped the ban i could see putting in that kind of rule... but we're not there yet.
    – Kevin B
    Feb 27 at 20:13
  • 7
    @vandench That being said, any content ChatGPT generates itself is almost certainly guaranteed to be in violation of copyright itself since the data it was trained on was certainly not entirely legally consumed.
    – TylerH
    Feb 27 at 20:15
  • I can honestly say I haven't seen a single user who has posted ChatGPT content clearly state that the the content was generated by ChatGPT. If, and only IF, users start doing that I don't see any benefit of even thinking about whether an answer that contains attributed ChatGPT vomit is "ok", because I don't believe that such users ever will be honest about it. The fact that the users don't check the work before they post the (unattributed text) is another problem that needs to be solved before we consider this.
    – Thom A
    Feb 27 at 20:15
  • @KevinB - No, it does not. At least if you cite it and use a narrow version of the content an argument can be made to whether the post originated entirely from ChatGPT versus merely using a small portion. If users intend to pass themselves off entirely as ChatGPT, then a 1 week ban doesn't seem effective, as this content is everywhere right now.
    – Travis J
    Feb 27 at 20:15
  • @Larnu - That's a fair take, but it is also still analyzing that the post is entirely sourced from ChatGPT, which as I state above should be banned. Mostly, this is to address the possibility of including small portions, which would or should have been author reviewed.
    – Travis J
    Feb 27 at 20:17
  • There's no value in that content existing, if it isn't the primary content of the answer. Saying "Chatgpt said X" provides no value whatsoever in an answer. In a question, I don't think it's strictly forbidden to include a wrong answer from chatgpt when including things you've tried
    – Kevin B
    Feb 27 at 20:18
  • I'm not saying I disagree with the idea, however, I feel that the suggestion is a little putting the cart before the horse. We need the users to learn to crawl (cite properly), walk (test), and run (add their own user contrnt) to allow this, and they haven't learned how to crawl yet.
    – Thom A
    Feb 27 at 20:19
  • 7
    there's no value whatesoever in using chatgpt as a reliable source worth citing, a good answer would never use chatgpt as a source because a good answer would have it's answerer and official documentation as sources, not a chatbot. I see no value in even creating a policy that suggests answerers should cite chatgpt content.
    – Kevin B
    Feb 27 at 20:20
  • 3
    I don't care if it's all the vomit that ChatGPT generates or just a cup of the vomit, the content ChatGPT generates is complete garbage. I have yet to see a answer, generated by ChatGPT, that wasn't complete and total garbage. I have identified at least 3 dozen answers generated by ChatGPT. ChatGPT sucks at identifying context, so it often attempts to answer a question on a different subject, than what the actual question was asking about. Only a human is able link two subjects, and determine from the context, what the actual topic of the question is. Feb 27 at 20:26
  • 5
    @vandench You don't need a court ruling to know they stole content without proper attribution or permission, let alone violating users' privacy. There are gazillions of blogs and articles out there that document how it did so. In fact I think GPT-3's authors itself even acknowledge this.
    – TylerH
    Feb 27 at 20:55
  • 1
    I would like to say one thing: ChatGPT is not guaranteed "vomit", to follow the terminology used here. I have seen plenty of seasoned veterans use ChatGPT really effectively to the point where it scares me since I myself cannot, I think I'm missing some classic engineering training in my education. It is how the average Stack Overflow visitor would use it (lazily) that makes it crap.
    – Gimby
    Feb 28 at 12:52

3 Answers 3

20

All AI generated content is "the work of others" and requires following the referencing requirements.

This is Stack Overflow's, the company's, policy as explained by animuson in answer to: "Is attribution required for machine-generated text when posting on Stack Exchange?" and applies to all Stack Exchange sites, including Stack Overflow.

This is regardless of the copyright status of the AI content.

It was initially established for Stack Overflow, the site, in the consensus in answer to: "Is it acceptable to post answers generated by an AI, such as GitHub Copilot?".

This means you need to:

If you copy (or closely rephrase/reword) content that you did not create into something you post on Stack Overflow (e.g., from another site or elsewhere on Stack Overflow), make sure you do all of the following:

  • Provide a link to the original page or answer
  • Quote only the relevant portion
  • Provide the name of the original author

Note: "Temporary policy: ChatGPT is banned" and "Why posting GPT and ChatGPT generated answers is not currently acceptable" still apply, so even if it's only a portion of the post and is properly referenced, content generated by ChatGPT is still banned.

Trivia: In my experience, about 99.9% of the ChatGPT generated content that's posted on Stack Overflow doesn't follow the referencing requirements.

9

There is no value in having a policy specific to ChatGPT requiring citing sources. We already have such a policy for the general case.

That said, there's also no value in ChatGPT answers being provided as sources because a valuable answer would be citing reputable sources, not a chat bot. Citing ChatGPT in a given answer would literally do the answer harm in terms of whether or not it is a good reputable answer.

I wouldn't necessarily be against users pointing out when they've relied upon ChatGPT to generate an answer, however that should only lead to its deletion if they're using it as a source of information.

4
  • I agree with almost all of what you are saying here, except for the one aspect I was hoping to discuss in this post. Namely, this situation where only a single blurb comes from ChatGPT. As an example, a primary source of some of the learning from ChatGPT came from w3schools. That isn't really a good endorsement, if you are familiar with that as a source, however, it does produce the very similar guidance on simple subjects. Including a small phrase of explanation there to accompany the use in code whereby the answerer wrote their own code and also explains the code is perhaps beneficial.
    – Travis J
    Feb 27 at 20:45
  • My concern is that entirely banning all versions of content from ChatGPT is going to become extremely burdensome to our moderation team here. It may be extremely difficult to detect if it is remixed into only part of posts. In that regard, perhaps it makes more sense to at least provide guidance for proper use.
    – Travis J
    Feb 27 at 20:46
  • 2
    my argument is, it would be more beneficial to cite w3schools in that case than it would be to cite chatgpt.
    – Kevin B
    Feb 27 at 20:47
  • I would say yes if it were a direct citation. However, ChatGPT does at times provide a composite of several sources in a condensed form which can be more efficient to use. It also does not disclose where its information comes from (which imo is a separate but problematic issue).
    – Travis J
    Feb 27 at 20:48
4

So I'm gonna get pretty rules-laywer-y on this. Let me state up front, from what I've considered presented here:

  • Not ascribing your sources is plagiarism, and plagiarism is not allowed anywhere in the network.
  • Having a user cite that they got their information from Source X is pretty routine stuff in the normal swing of things.

What I can't really offer is if citing that you got your answer from ChatGPT makes the thing better™.


Let's take this part by part. The policy on why ChatGPT is banned is as follows, emphasis mine.

The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.

In reading this, the main justification as to why ChatGPT is banned is centered around two factors:

  • Correctness of answers
  • Volume of answers

In short, we're getting bombarded by a deluge of crap answers. People deciding to cite their work on that isn't really going to improve matters too much and it wouldn't really tip the scale in one direction or another on people's use of the tool.


Let's take away factor 1 from this equation - let's say that there is a "correct" answer that is generated by ChatGPT. In that context, not citing your source is still tantamount to plagiarism, and there's just no guarantee that this makes the situation any better to what's already going on. At best, the community loses something useful because someone took something else's work and tried to pass it off as their own. At worst (and I mean this is worst case scenario, so hyperbole abounds) we now have people with whole site curator powers who are not expert enough to help the community identify issues with answers at any sensible scale.


Not that long ago we had someone try to do this thing where they walked the tightrope between having AI-generated answers and human curated responses. It fell flat on its face because the human didn't human, and I don't recall their answer having the right citations attached. Let's suppose then that this person had done all of the things correct:

  • They had properly attributed their answer to ChatGPT in the manner required,
  • They had curated it for correctness and indicated that they had proofread it, and
  • The answer was good and useful for others.

On its face this avoids the reason that ChatGPT was disallowed in the first place:

  • The answer was validated to be correct, and
  • the volume of answers provided to the site is low per capita.

However, this highlights another issue which I touched on earlier - you're giving someone powers and privilege on the site who doesn't really have the same level of expertise as someone who doesn't use ChatGPT.


So all in all, I'm not sure if this makes the situation any better. Even if users cited that they got their information from ChatGPT, and even if the answer was correct, there's a different problem brewing behind that which the original policy doesn't cover, either in spirit or in letter, which would require some extra thinking.

For all I know, maybe Stack Overflow doesn't care if a person really knows what they're talking about. ¯\_(ツ)_/¯

1
  • 2
    This is essentially right where my thinking is too. The aspect of site privilege is an interesting one that I hadn't considered. I guess I just thought that anyone able to accurately glean information that was correct would also have a high level of subject knowledge.
    – Travis J
    Feb 27 at 22:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .