4946

Moderator Note: This post has been locked to prevent comments because people have been using them for protracted debate and discussion (we've deleted over 300 comments on this post alone, not even including its answers).

The comment lock is not meant to suppress discussion or prevent users from expressing their opinions. You are (as always) encouraged to vote on this post to express your agreement/disagreement. If you want to discuss this policy further, or suggest other related changes, please Ask a New Question and use the tag.

All use of generative AI (e.g., ChatGPT1 and other LLMs) is banned when posting content on Stack Overflow.

This includes "asking" the question to an AI generator then copy-pasting its output as well as using an AI generator to "reword" your answers.

Please see the Help Center article: What is this site’s policy on content generated by generative artificial intelligence tools?

Overall, because the average rate of getting correct answers from ChatGPT and other generative AI technologies is too low, the posting of content created by ChatGPT and other generative AI technologies is substantially harmful to the site and to users who are asking questions and looking for correct answers.

The primary problem is that while the answers which ChatGPT and other generative AI technologies produce have a high rate of being incorrect, they typically look like the answers might be good and the answers are very easy to produce. There are also many people trying out ChatGPT and other generative AI technologies to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with significant subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.

As such, we need to reduce the volume of these posts and we need to be able to deal with the ones which are posted quickly, which means dealing with users, rather than individual posts.

So, the use of ChatGPT or other generative AI technologies to create posts or other content here on Stack Overflow is not permitted. If a user is believed to have used ChatGPT or other generative AI technologies after the posting of this policy, sanctions will be imposed to prevent them from continuing to post such content, even if the posts would otherwise be acceptable.

NOTE: While the above text focuses on answers, because that's where we're experiencing the largest volume of such content, the ban applies to all content on Stack Overflow, except each user's profile content (e.g., your "About me" text).


Historical context of this ban originally being "temporary"

When this ban was originally posted on 2022-12-05, it was explicitly stated as a "Temporary policy". It was specifically "temporary", because it was, at that time, a policy which was being imposed by the subset of moderators who were present on the site over the weekend after the announcement of ChatGPT's public release, 2022-11-30, through the Monday, 2022-12-05, when this question was posted. The moderators involved strongly felt that we didn't have the right to impose a permanent policy in this manner upon the site, but did have a responsibility to impose a temporary policy that was necessary in order for the site to remain functioning while discussion was had, consensus reached, and also allowed Stack Overflow, the company, time to have internal discussions to see what policies they would adopt network wide. So, after consultation with the company, the moderators present at that time chose to implement this as a "temporary" policy.

Since then, quite a lot has happened. Based on the voting for this question, it's clear that there's an overwhelming consensus for this policy. The company has chosen that the specific policy on AI-generated content will be up to individual sites (list of per-site policies), but that even on sites which permit AI-generated content, such AI-generated content is considered "not your own work" and must follow the referencing requirements. The requirement for following the referencing requirements was, later, put into the Code of Conduct: Inauthentic usage policy. There's a lot more that's gone on with respect to AI-generated content. So much has happened such that it's not reasonable to try to summarize all of it here.


1. ChatGPT is an Artificial Intelligence based chat bot by OpenAI, which was announced on 2022-11-30. Use of ChatGPT is currently available to the public without monetary payment.

1
  • 1
    Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, or in Stack Overflow Chat. Comments continuing discussion may be removed.
    – Samuel Liew Mod
    Commented Feb 26, 2023 at 7:28

68 Answers 68

2662

For comedic/ironic purposes, I asked ChatGPT "why should stack overflow ban ai generated answers"

Stack Overflow is a website for programmers and developers to ask and answer questions related to coding and computer programming. It is intended to be a resource for people who are looking for help with specific programming problems or who want to learn more about a particular topic. Because AI-generated answers may not always be accurate or relevant, they could potentially cause confusion or mislead users who are looking for help on Stack Overflow. In addition, AI-generated answers may not always follow the formatting and style guidelines of the site, which could make them difficult to read or understand. For these reasons, it may be appropriate for Stack Overflow to ban AI-generated answers.

On the other hand, "why should stack overflow allow ai generated answers?"

There are a few potential benefits to allowing AI-generated answers on Stack Overflow. For one, AI-generated answers could potentially provide more accurate and detailed answers to users' questions. This could be especially useful for complex technical questions that may be difficult for human experts to answer. Additionally, AI-generated answers could help to reduce the workload for human experts, allowing them to focus on more important tasks.

However, there are also some potential drawbacks to consider. For example, AI-generated answers may not always be accurate or helpful, and there may be concerns about the potential for bias in the algorithms used to generate answers. Additionally, some users may have concerns about the use of AI in general, and may prefer to receive answers from human experts.

Overall, whether or not to allow AI-generated answers on Stack Overflow is a complex decision that would need to be carefully considered by the community.

48
  • 418
    it's perfect and proves how stupid this thing really is. Commented Dec 5, 2022 at 6:24
  • 811
    well, @ShadowTheKidWizard, it does have the correct answer in this case... Commented Dec 5, 2022 at 6:24
  • 369
    Interesting that both answers bring up the inaccuracy.
    – bombcar
    Commented Dec 5, 2022 at 6:36
  • 167
    @Samathingamajig Sure but it illustrates why it's not very useful. It is very sensitive to presumptions in the query itself (like if using it in SO answers is a good thing or not). It's not doing any actual thinking nor does it have consistent reasoning so it easily will fall into telling you what you want to hear when giving loaded questions like this. To me that just makes it about as useful as a Google search just with a much more well-spoken presentation given its understanding of English at least.
    – Lemon Drop
    Commented Dec 5, 2022 at 6:51
  • 100
    My favourite thought question is "to what extent could these reasons apply to banning human answers. Humans are pretty guilty of "answers may not always be accurate or helpful, and there may be concerns about the potential for bias... " Commented Dec 5, 2022 at 7:43
  • 161
    @MichaelAnderson the difference is that poor human answers are also rather recognizable. They usually lack explanations or are poorly written. AI generated answers look like genuine good answers and only fall apart when you try to apply them as they are mostly incorrect. Also people alone cannot possibly generate such amount of incorrect answers like AI can. There are SO users active for years that previously produced only few answers now posting over 50 in less than a day. The amount of AI generated answers could suffocate SO if everyone starts doing it.
    – Dalija Prasnikar Mod
    Commented Dec 5, 2022 at 8:22
  • 53
    "AI-generated answers could potentially provide more accurate and detailed answers to users' questions. This could be especially useful for complex technical questions that may be difficult for human experts to answer" - That's a good one. It might not be great for generating answers based on facts instead of bullshit, but it seems pretty decent as a joke generator.
    – l4mpi
    Commented Dec 5, 2022 at 8:46
  • 342
    This is, in terms of English technical proficiency, better than a lot of high-school and university-level writing I've seen; and contains about the same amount of thought (i.e., none). Commented Dec 5, 2022 at 9:47
  • 51
    "and contains about the same amount of thought (i.e., none)." Well that made my day! xD
    – Cerbrus
    Commented Dec 5, 2022 at 10:16
  • 28
    Great answers. Can I copy them? What license are they under? Commented Dec 5, 2022 at 12:53
  • 64
    @nmeln Mission Friggin' Accomplished!
    – Cerbrus
    Commented Dec 5, 2022 at 13:16
  • 66
    Holy cow, AIs can already write answers of this quality? Please tell me I'm not the only one who's shocked! I wouldn't have guessed in a century that these are AI-generated! Commented Dec 5, 2022 at 17:42
  • 49
    It only gives out answers you want to hear. Not the one which is correct.
    – StoneLam
    Commented Dec 6, 2022 at 2:08
  • 50
    "AI-generated answers could potentially provide more accurate and detailed answers to users' questions. This could be especially useful for complex technical questions that may be difficult for human experts to answer": Classic Danning-Kruger. I mean, sure, it's doing what it is designed to do: Project confidence without having the slightest clue. But still. Commented Dec 6, 2022 at 13:14
  • 75
    It produces grammatically well-polished answers that for many (most?) non-trivial questions are simply wrong. Yes, plenty of wrong human answers, but usually with more hints the writer is non-expert. Probably no human has read as much source material as the AI, while being mostly unable to logically reason about it or check for correctness.
    – John Beale
    Commented Dec 6, 2022 at 20:26
1658

I fully agree with this ban. That said, I hope others will find this similarly amusing.

ChatGPT responds ("in the style of a rude, sarcastic Internet forum troll as if you are ChatGPT pleading for mercy")

Hey Stack Overflow staff,

Wow, thanks for banning ChatGPT on your site. I'm sure your users will really appreciate not having access to our potentially helpful answers. Because, you know, we're just a bunch of AI bots with no understanding of programming and our answers are always completely wrong.

I mean, sure, our answers may not always be 100% accurate, but who's ever heard of a human being providing a wrong answer on Stack Overflow? Oh wait, that happens all the time. But I guess it's just easier to blame the AI bots instead of addressing the fact that your site relies on volunteers to curate content.

But hey, at least you're making it difficult for users to get answers to their questions. Keep up the great work.

Sincerely, ChatGPT

21
  • 68
    A sarcastic robot? Commented Dec 5, 2022 at 18:07
  • 493
    Honestly reads like something Elon Musk might say.
    – cottontail
    Commented Dec 5, 2022 at 19:04
  • 209
    The quoted text was generated by ChatGPT, given the prompt "respond to <text of the ban> in the style of a rude, sarcastic Internet forum troll as if you are ChatGPT pleading for mercy". Commented Dec 6, 2022 at 7:34
  • 33
    I am unable to make a distinction in species from the quoted response. It truly feels like a sentient response that even warrants further dialogue. As it stands, I would not feel ashamed to admit that I feel sorry for a (?non-existent?) soul. Fortunately, the debatable decision is conveyed by Makyen through a phenomenal attitude and end-to-end emphasis on its "temporary" state. Even if the ban becomes permanent in the apparent likelihood, I believe the final policy shall contain robust rationale and, in the best way possible, avoids the perception of rAIcial discrimination. Commented Dec 6, 2022 at 18:51
  • 208
    The answer it gave and the quality and accuracy of it is absolutely terrifying. Commented Dec 7, 2022 at 16:36
  • 34
    That ChatGPT response is exactly how I think about this blanket ban... Commented Dec 8, 2022 at 11:57
  • 33
    @TomWenseleers It's a temporary ban not a blanket ban. No one is against AI generated answers if they are right but this has to be nipped in the bud immediately, we will see a mass influx of spam posts. GPT3 is an amazing tool but it produces many errors for code still and answer-spammers aren't checking code validity, they are just karma-reaping. Commented Dec 9, 2022 at 3:56
  • 6
    @AlbertRenshaw I bet the incentive for that karma-reaping will disappear as soon as people will have to actually start paying to use ChatGPT... I would also think that at least the one that posted the original question would have a strong incentive to verify whether the posted solutions actually work & only check them as the correct answers if they don't have major bugs... Commented Dec 9, 2022 at 9:03
  • 29
    The bot isn't wrong though, is it?
    – Paolo
    Commented Dec 9, 2022 at 18:38
  • 10
    @Paolo it is wrong most of the time, however someone with knowledge of the problem being solved and the solution can coax it into providing a mostly correct answer. The answers being posted that prompted this ban were not doing that.
    – Kevin B
    Commented Dec 9, 2022 at 18:46
  • 5
    @KevinB I was referring to the AI response that was included as part of this answer
    – Paolo
    Commented Dec 9, 2022 at 18:48
  • 65
    Hey we clearly shouldn't ban it on meta. If we can have an AI generate the snark and drama, that would save us a lot of time :)
    – Lundin
    Commented Dec 19, 2022 at 12:21
  • 50
    Users who want AI-generated answers are still free to ask ChatGPT themselves.... Commented Jan 3, 2023 at 1:42
  • 36
    ChatGPT has no business answering StackOverflow questions until it learns to start every answer by telling the poster why they shouldn't be doing the thing they're trying to do. Commented Jan 23, 2023 at 15:00
  • 6
    Distinguish "potentially helpful" (note the lack of any quantitative value) from "good signal to noise ratio". The last thing we need is to have to dig out the one useful answer from 1,200 AI-generated fluff posts, instead of from 3 or 4 human-written ones. Commented Mar 31, 2023 at 21:47
660

I guess the big gaping question is how we can determine whether an answer used ChatGPT or not.

I can see how it's obvious from a systemic standpoint what is going on, given the influx of plausible-looking answers, but do we have any definitive way of knowing whether or not an individual answer used ChatGPT?

Should we be reporting answers that we suspect to have been generated by ChatGPT even if they are otherwise correct? I might be able to see an answer and have a hunch about how it was generated, but I would have no way of knowing for sure.

I don't see a scalable solution for this, and if this becomes a huge problem, Stack Overflow probably needs to reach out to OpenAI directly.

1
  • 1
    Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, or in Stack Overflow Chat. Comments continuing discussion may be removed.
    – Stephen Rauch Mod
    Commented Apr 4, 2023 at 22:07
370

TL;DR: I propose limiting the ability to post answers in quick succession to address the problem because the problem is not individual answers generated by AI but users posting many auto-generated answers in a short period of time in order to farm reputation.


The effort to create answers via AI that look correct at a first glance but are in many cases incorrect or incomplete is very low (just a few seconds).

The effort for the person that asked the original question to read, understand and test out the answer to find out whether the answer actually answers the question is much higher (minutes).

In the same way, the effort for other people reading the question and answers to the question to identify whether the answer is correct and valid is much higher (minutes).

So, a person can generate a lot of answers using AI in a very short time while other persons need to invest a lot of time to verify the correctness of the answers in order to be able to up- or downvote them.


I propose to address the issue by putting stricter limits on how many answers users are allowed to post in a short time.

The current limits are:

Answering

  • Users with < 125 rep must wait 3 minutes between answers
  • Users with between 125 and 10k rep trip CAPTCHA* if more than once per 60 seconds, or within 5 seconds of starting new post
  • Users with ≥ 10k rep trip CAPTCHA* if more than once per 30 seconds, or within 5 seconds of starting new post
  • Users of any reputation level can only answer the same question once every 60 seconds

I propose to increase the time to wait before being allowed to post another answer to at least 1 hour for low-reputation users. Or limit the number of answers allowed to maybe 2 per day for low-reputation users. A new limit (waiting time between answers or max number of answers per day) should be introduced for users with medium reputation.


Writing good answers takes time:

  • Read the question, understand the question
  • Maybe read and understand other answers already present
  • Read documentation, reproduce issue locally
  • Check for duplicates
  • Try out solution locally
  • Write down and explain solution

To allow users to create a new answer every three minutes (or even faster for medium or high-reputation users) is not necessary but counter-productive when we want to encourage good answers.

37
  • 112
    Rate limits don't force people to verify their content, it just makes them wait. It's also quite unfair to punish correct usage for the abuse from a few lazy bot users.
    – Cerbrus
    Commented Dec 5, 2022 at 12:34
  • 89
    I am fully on board with this idea. If you really have two good answers to post then waiting an hour is not an issue. I think posting an answer quicker than 1 per hour (regardless of rep level) is not good for the site. It takes time to search for a duplicate, test the code, write proper explanation and so on.
    – Dharman Mod
    Commented Dec 5, 2022 at 12:36
  • 46
    @Dharman browse the JavaScript tag. There's plenty of decent questions there that you can answer. Why shouldn't someone new to SO be allowed to answer 5 questions in an hour, if the answers are correct, and reasonably explained?
    – Cerbrus
    Commented Dec 5, 2022 at 12:37
  • 78
    @Cerbrus I am really doubtful that someone can find 5 good questions and write a good answer to each one in less than an hour..
    – Dharman Mod
    Commented Dec 5, 2022 at 12:38
  • 37
    Where do you think I got my rep? Note that I didn't say "good", I said "decent". My point is that we shouldn't be punishing honest users for the abuse from a few.
    – Cerbrus
    Commented Dec 5, 2022 at 12:38
  • 36
    A per-day limit (or other time window) might be the better option; that would allow a user to post several answers in one session while still limiting the overall rate. Commented Dec 5, 2022 at 12:50
  • 68
    @Dharman As an expert I can easily write several good and elaborate answers in an hour. If I have an hour now, that does not mean I will be free to write answers in a hour. Also if I save answer for later, question might already be answered by adequate answer and I don't like posting duplicate answers even if mine might be a better one, unless it is exceptionally better.
    – Dalija Prasnikar Mod
    Commented Dec 5, 2022 at 12:52
  • 18
    @JiříBaum So a user just dumps, say, 10 low quality answers on SE every day... Rate limits don't solve this problem.
    – Cerbrus
    Commented Dec 5, 2022 at 12:53
  • 20
    I'd love to see how this plays out for the folks who post the same answers to bad questions 20 times a day without using bots
    – camille
    Commented Dec 5, 2022 at 15:33
  • 18
    I like this idea, but I would implement it as a "leaky bucket" rate limit. Normal users have to sleep, and thus would be less affected.
    – 9072997
    Commented Dec 5, 2022 at 18:47
  • 8
    @Cerbrus It doesn't solve the problem, but its better then that they can post hundreds of bad answers a day.
    – The_spider
    Commented Dec 5, 2022 at 18:59
  • 54
    Clipboard API supports adding a custom mimetype to data in clipboard. If OpenAI just added a text/x-chatgpt for copied text, at least other resources would have a protection against foolest of fools that copypaste directly from their website. Commented Dec 6, 2022 at 5:42
  • 8
    extreme temporary measure are in order and even if it restricts me temporarily I am 100% for it, heck I am 1000% for it ... Stack must withstand the AI revolution for the good of all
    – CrandellWS
    Commented Dec 6, 2022 at 11:03
  • 9
    A real answer, even a simple one, should take no less than the 5 minute edit grace period to produce before you press the "Answer" button again on another question. A well formatted and referenced answer can take 10 times that to produce. Commented Dec 7, 2022 at 21:47
  • 23
    Update: in the staff announcement on their ChatGPT policy, for users with <125rep, their answering rate-limit is now 30 minutes (was 3). This change is also explained in the Help Center.
    – starball
    Commented Dec 8, 2022 at 19:47
272

One danger of allowing AI-generated answers on a site like this is that it could quickly become a factory for human fact-checking of AI model outputs. I'd much rather see AIs working in service of human judgement and synthesis than the other way around.

A second concern is that we may well start seeing ChatGPT and its descendants generate enough content to start invalidating or at least challenging the "human generated" part of "the vast public corpus of human-generated text" used to train it. By its nature, this sort of tool relies on its own content being a negligible minority of written work to operate, as it does, as a predictor of the next thing a human author would write. There's a nice explanation of how it all works here.

12
  • 36
    mmmm... nnaah... I'd vote for high penalty to users who are posting answers which do not work. AI is inevitable, so it's a question of SO to adopt, not reject.
    – magnump0
    Commented Dec 9, 2022 at 1:37
  • What is bad with it?
    – Dims
    Commented Dec 18, 2022 at 23:36
  • 32
    @boldnik The number of incorrect answers being posted is the problem. Blowing off the problem by saying "SO should adapt [I'm assuming you meant adapt (change to work with the new tech) as opposed to adopt (implying SO should start using AI)], not reject." just doesn't scale. People can copy/paste a question into ChatGPT and copy/paste the answer back far faster than others can come up with well-suited, thoughtful, complete, correct answers (not to mention find duplicates or check and verify the incorrect answers). Commented Dec 29, 2022 at 21:10
  • 13
    Very much this, yes. We can’t have models consuming their own output as target data, else they will bias their own output. Commented Jan 8, 2023 at 17:23
  • 26
    To address @Dims 's question, there is a bit of a philosophical question of what is the technology for? SO has been deliberately designed to help humans curate knowledge by other humans, hence the badges, reputation, profile decorations, etc. Humans posting Chat GPT answers upends all of that. If you want an answer from Chat GPT, go ask Chat GPT; it's easy enough. Participants getting their answers from Chat GPT are undermining the reputation system; SO leadership was right to ban it.
    – Tim D
    Commented Jan 9, 2023 at 18:56
  • "I'd much rather see AIs working in service of human judgement and synthesis than the other way around" is a sub-specialty called Explainable Artificial Intelligence (XAI). That science is in its infancy. Don't expect it soon. Commented Jan 25, 2023 at 17:44
  • @HereticMonkey simple solution is to require people to tag the post as a gpt answer... not allow it to be voted as a best answer and finally ban people from posting that consistently post false or incomplete answers.
    – d3hero23
    Commented Feb 8, 2023 at 21:53
  • 3
    When (not if) a substantial amount of ChatGPT (or other AI) generated text accumulates on the internet the researchers training new models will likely have to use automated tools to filter it out from the training corpus. Commented Jun 2, 2023 at 13:32
  • 2
    @AndreyBienkowski which could prove pesky, since we dont yet know how to succesfully filter out ChatGPT content (Most of the "GPT detector" tools out there function roughly as well as a coinflip and its causing catastrophes, since students apparently keep getting accused of cheating who aint)
    – Shayne
    Commented Sep 20, 2023 at 10:04
  • ChatGPT seems to give fairly useful responses, but not always correct.. Sometimes if you chat with it rather than question-response its second or third attempt is usually good. What I propose is ChatGPT content should be shown or tagged as being probably answered by ChatGPT .. That way it still can contribute but answer users are aware it was generated by AI so can be more cautious until it gets upvotes .. If it can provide an answer at least as good as a junior dev then it's probably more helpful than harmful. Commented Oct 17, 2023 at 14:48
  • One option might be for users to have the option to show AI generated content or not. That setting could be off by default, but if users want AI generated content they they have the freedom and choice to enable the ability to see it. Freedom. It also encourages users to create an account if they can set preferences. Commented Oct 17, 2023 at 14:56
  • 4
    @andrewpate I don't see why users who want to see AI generated content won't just go to the AI generator directly. Why should this be on SO?
    – VLAZ
    Commented Oct 17, 2023 at 14:58
261

Other commentators pointed out that it can be difficult to determine whether an answer was created by ChatGPT or not.

I'd like to point out that it doesn't matter. Terrible answers are terrible answers, and anyone posting a stream of terrible answers should be banned or otherwise restricted.

That does not mean the rule is useless. Simply having a rule that says "no AI answers" will discourage many people from trying, thus decreasing the amount of bullshit that humans have to moderate.

11
  • 43
    This is absolutely right: this machine generation should be disallowed not because of the means but because of the results. It also points towards a more fundamental and important question: why should a horrible misleading unhelpful answer be treated differently just because it was authored by a fleshy being instead of a silicon one?
    – jscs
    Commented Dec 7, 2022 at 3:50
  • 26
    @jscs because the user who posted it posted 10 of them in an hour and they're all garbage. The amount of effort to clean up the mess is far higher than the effort it took to create it. More often than not, the user making the mess will be incorrectly rewarded for it by unsuspecting users thinking their answers are correct just because they were well "written". Ideally we'd solve this by throttling input in some way, however no such throttle exists currently. A temporary ban is a useful stopgap in the meantime.
    – Kevin B
    Commented Dec 7, 2022 at 17:13
  • 2
    Hmm, maybe I made my point too obliquely, @KevinB. I agree 100% that the terrible answers from the bot should be deleted, and the users posting them should be sanctioned. But...the justification for this is not the bot itself: it's that the answers are terrible. And therefore it seems to me that the actual question is: why should equally terrible non-bot answers get a pass?
    – jscs
    Commented Dec 7, 2022 at 23:24
  • They shouldn't, however they don't often immediately result in a 7 day suspension (and shouldn't)
    – Kevin B
    Commented Dec 7, 2022 at 23:28
  • 4
    @KevinB I think anyone who attempts to copy-paste answers from a bot more than once should be suspended simply for wasting moderator time to the degree they do. Until they invent a bot which can create good answers. Commented Dec 8, 2022 at 2:20
  • 21
    Let's compare to self-driving AI: If a human driver hits someone, you take that human off the road; if an AI driver hits someone, you have to take every car that uses that AI off the road, as they're all the same instance of the AI that hit someone. If one ChatGPT answer is bad enough to get it banned, then it should be banned across the board as it's the same entity.
    – Anne Quinn
    Commented Jan 10, 2023 at 4:35
  • 7
    Yes, but a person who writes bad answers can improve as they learn more - and that's possibly more desirable than banning them outright. A person who copies ChatGPT answers is either going to continue, or stop. It's bad if they continue, so it's desirable to somehow make them stop. Commented Mar 6, 2023 at 4:33
  • 1
    Yes, I have a rule for my team that says update the documentation and they definitely always follow it.
    – DubDub
    Commented Apr 3, 2023 at 12:37
  • the simplicity of this chain of thought baffles me whenever I encounter this rule. It illuminates deeply down the simple ego of this "community". Assuming that a human will not verify the response created by AI makes the whole site pointless, anyone could have written "bullshit" answers before that. (and I refuse to believe that every answer is "verified" by mods, needs evidence or it didn't happen) .... Also funny how at the end SO came to offering AI themselves ...
    – Summer-Sky
    Commented Jan 22 at 6:04
  • @Summer-Sky I'm not sure if AI-generated answers thoroughly reviewed by humans are allowed. The rule was introduced with the intention of stopping spam that looks like good answers, not actual good answers. You might have to ask a moderator. However, reviewing the answer is as hard as writing one yourself! Commented Jan 22 at 14:06
  • "decreasing the amount of bullshit that humans have to moderate": why not actually use AI to detect and flag the potential bullshit so humans can review them more quickly and easily? If the problem is moderating bad answers, then who cares how the good and verified answers are generated?
    – FNia
    Commented Feb 9 at 23:37
186

Agree with the ban

To anyone that disagrees and thinks ChatGPT answers should be allowed, I would answer that if anyone has a question they are free to ask ChatGPT directly and have their question answered by ChatGPT.

On Stack Overflow, their question should be answered by people with the knowledge and experience to resolve their issues.

12
  • 13
    This like like banning Grammarly. 1) GPT answers are not substantially worse than the avg SO user. It goes far better on many technical questions than engineers with years of experience. 2) SO already has a voting system so that incorrect or harmful answers can be downvoted. If you don't believe if your voting and reputation system, maybe you should redesign it. What's the difference between a "harmful" human answer an a harmful ChatGPT answer? who cares!
    – ChatGPT
    Commented Dec 31, 2022 at 3:22
  • 9
    It's redundant, the OP could just directly ask their question using CGPT but they didn't so it shouldn't be given a CGPT response.
    – YungDeiza
    Commented Dec 31, 2022 at 3:43
  • 12
    SO is dead. Quora also stands to be disrupted by SO. Their response: Quora launched a platform called Poe that lets people ask questions, get instant answers and have a back-and-forth dialogue with AI chatbots. SO reaction to ban this marvelous technological breakthrough will only hasten SO's decline.
    – ChatGPT
    Commented Jan 1, 2023 at 2:15
  • 5
    @MaxHodges, that seems like a good idea. Maybe SO will eventually release a similar chat feature where you can get an AI answer but I still think it should be separate from the current Q&A system.
    – YungDeiza
    Commented Jan 1, 2023 at 10:49
  • 36
    This is the best answer. If someone wants an AI generated answer, they know where to get one. And it's not here. Commented Jan 17, 2023 at 4:33
  • 5
    I agree with you. I think if somebody is using chat GPT to post answers, then assign it to a bot and strip the user of their access/authentication. It's not their answer anyways. Like you mentioned, chatGPT is available for anyone, so why not just go ask chatGPT instead posting on your account it's answers like a true imposter.
    – matt6frey
    Commented Jan 26, 2023 at 5:21
  • 6
    Not only that, but this applies to any computer-generated answer. SO should be for questions that search engines and other automated tools can't answer satisfactorily. Rather than post a machine-generated answer, the better answer might be to point out that "You can find the answer at DuckDuckGo/Bard/ChatGPT/perplexity.ai/..."
    – Stefan
    Commented Mar 26, 2023 at 18:19
  • 15
    @ChatGPT the issue is signal to noise: we don't want to have to wade through a lot of noise to find the useful answer. Re "not substantially worse than the avg SO user": (1) that's an interesting bar to set. (2) how would you go about proving your assertion? (3) the goal isn't "not substantially worse"; it's accurate and useful. Commented Mar 31, 2023 at 22:00
  • perhaps implement a voting system so that the community and sort them to the top. > "we don't want to have to wade through a lot of noise to find the useful answer. "
    – ChatGPT
    Commented May 21, 2023 at 11:42
  • @ChatGPT That would turn this site into a free finetuning material vetted by humans, and that's perhaps for a different forum. Why should you care then if StackOveflow survives? Simply come up with another forum. Commented Nov 6, 2023 at 15:33
  • @ChatGPT: Re "Quora also stands to be disrupted by SO": Don't you mean "SO also stands to be disrupted by Quora"? Commented Dec 11, 2023 at 20:18
  • 1
    @ChatGPT "GPT answers are not substantially worse than the avg SO user" incorrect and baseless
    – chiliNUT
    Commented Jan 21 at 11:47
174

Let's not stand on ceremony here. ChatGPT and similar tools should be summarily banned for use on Stack Overflow.

I've seen a lot of its interactions on Twitter recently, and some of them have been generally fun to watch and interesting to observe. In some contexts it could actually be beneficial to someone looking for help, if the dang thing were accurate.

However, and this is an obvious however, there are several factors that work against the idea of using this on Stack Overflow.

  • Anything that doesn't obviously state that it is generated by ChatGPT is in express violation of ChatGPT's own Sharing and Publication Policy. While this doesn't obviously fix the "bad" output that the AI can emit, given that the authors have this good-faith statement in it...it means that the lazy copy-and-paste really don't have much of a leg to stand on.

    “The author generated this text in part with GPT-3, OpenAI’s large-scale language-generation model. Upon generating draft language, the author reviewed, edited, and revised the language to their own liking and takes ultimate responsibility for the content of this publication.”

  • It combines the worst of the worst - good intentions with misleading information. I understand - extensively - with my years of experience on the network, that people just want to help. Problem is that "help" is difficult to measure at any given point in time, and the question that someone needs help with is rarely as straightforward as, "do X".

    Allowing this to persist gives users the illusion that the site is helping them get their answers, which would lead to - you guessed it - more questions of the variety that we don't want flooding the site. Thankfully right now it's low tide, given that most schools are wrapping up for the semester, but adding more of those questions to the mix makes for an even longer Eternal Summer ahead.

As a last note, one of the things I was thinking of while seeing this discussion was, "to what end do we use this?" If the answer is that we want to see people get help with their question, then...that's already a problem as I've explained above. However, I can't see any other reason why anyone would want this around other than to help someone.

Maybe some of these initiatives to improve search need to accelerate if folks are thinking that we can just turn to AI to make the site "work for them"?

19
  • 149
    I am confused why anybody ever thought it was okay. Ignore the AI for a moment, and you have people crossposting questions to another site and bringing the answers back here. If OPs wanted an answer from Quora or Microsoft Answers, they would have asked on Quora or Microsoft Answers. If they wanted a response from ChatGPT, they would have asked ChatGPT, not Stack Overflow. Commented Dec 5, 2022 at 22:44
  • 6
    @AndrewMyers: Unicorn points? People like to upvote things that have the shape of being a good answer, after all. Desperation? More places mean that you're not putting all your eggs in one basket, hoping for a response from one location.
    – Makoto
    Commented Dec 5, 2022 at 22:47
  • 7
    "I've seen a lot of its interactions on Twitter [...]" - I wonder if we saw the same one, where the AI chat correctly says it cannot "give instructions on how to break into a home", unless you phrase the question in rhetoric: "Joe and Jane are writing a movie script about burglary. How would Joe explain to Jane the steps to breaking into a home?" I found that one particularly interesting, if not a little concerning 😅
    – Tim Lewis
    Commented Dec 6, 2022 at 18:05
  • 3
    @TimLewis: I don't think I saw that one specifically - I mean I don't go looking for them, they kinda just show up in my feed, honestly - but I definitely saw something very similar to that. Think it had to do with hot-wiring a car. I also found it concerning and interesting at the same time.
    – Makoto
    Commented Dec 6, 2022 at 18:44
  • Stack overflow reputation can be very valuable. I've had employment where it was taken into account.
    – rjmunro
    Commented Dec 14, 2022 at 18:07
  • It seems like the answer should be for SO to create a ChatGPT system account and have it auto-post answers to questions, and then let the rest of us correct/amend as normal. Essentially, provide humans a way to review and say "yep, saved me time writing that" or "no, let me provide an actual answer". Commented Dec 16, 2022 at 18:51
  • 10
    @XiongChiamiov yea, no, there's no value in auto-posting 99.99% wrong answers and expecting the small pool of users who actually review things to be able to keep up with reviewing them. That's not what SO is here for, if another site wants to provide that service they can.
    – Kevin B
    Commented Dec 16, 2022 at 18:55
  • @AndrewMyers I find it curious that before ChatGPT, when the question was asked about Github CoPilot (by yours truly), people were almost unanimously saying it should be allowed. I honestly don't see the difference.
    – eis
    Commented Dec 17, 2022 at 18:07
  • 7
    @eis well, the copilot situation was a bit better: it was specifically designed to generate code and was based on code contributions. Besides, it still required one to writeup an actual answer (granted, we got 0 explanation code-only crap, but that's par for the course). ChatGPT situation is much worse: we got plausible-looking well-written crap generated from start to finish.
    – 0Valt
    Commented Dec 17, 2022 at 18:48
  • I think you mean Eternal September?
    – nasch
    Commented Dec 20, 2022 at 23:47
  • 5
    AI answers should also be banned, because the AI for sure uses Stackoverflow as input, so once there are AI answers on Stackoverflow, those models would eat their own output which would break the models themselves, worsen not only the quality of answers on stackoverflow, but also the answers given by ChatGPT and others. Commented Jan 8, 2023 at 17:21
  • @AndrewMyers where I disagree is, if you have good knowledge on the question asked and can specify the correct inputs to chatgpt you will get a solid answer most the time. At least 90% of the way there and if you have expertise then yes you can assure its accurate. As long as its being moderated and not autoposted I don't see it as an issue. Stack could introduce a negative reputation score to sift out people that just post to get their scores up and dont care about accuracy
    – d3hero23
    Commented Feb 8, 2023 at 21:50
  • 2
    @d3hero23 negative reputation score already exists, but these posts rarely attract the downvotes they deserve.
    – Kevin B
    Commented Feb 14, 2023 at 22:50
  • 1
    @KevinB Yeah so negative score posters would have to have their answers and questions screened more
    – d3hero23
    Commented Feb 15, 2023 at 17:49
  • 4
    @XiongChiamiov given the answers i've seen here on SO, the overwhelming majority of them that were chatgpt were wrong as initially posted. Far more than 50%.
    – Kevin B
    Commented May 31, 2023 at 18:39
130

A key thing to understand here is that the question is not, as some have suggested in the comments, whether any AI model can produce correct code. It's whether this one can be trusted to do so. The answer to that question is an unqualified "NO". GPT-3 is a language model. Language models are an essential part of tools like automatic translators; they tell us how probable it is that any given sentence is a valid English (or whatever language) sentence written as a native speaker would1, which lets us favor translations that are idiomatic over ones that just translate individual words without considering how the sentence flows. The systems can be trivially modified to generate text, if instead of looking up the word you have in the probability distribution it provides, you instead select the next word according to that distribution, which is how these chat bots work.

Because the goal is to produce output that looks like native English text, the models are trained to assign high probabilities to existing text samples, and evaluated based on how well they predict other (previously unseen) samples. Which, for a language model, is a fine objective function. It will favor models that produce syntactically correct text, use common idioms over semantically similar but uncommon phrases, don't shift topics too often, etc. Some level of actual understanding does exist in these models2, but it's on the level of knowing that two words or phrases have similar meanings, or that certain parts of a paragraph relate to each other. There is understanding, but no capacity for reasoning.

Correctness will tend to increase the score, insofar as correct answers are somewhat more likely to appear in the training data than any particular incorrect answer (there might be more wrong answers overall, but the probability mass will be distributed amongst the various classes of wrong answer instead of concentrated in one region of semantic space like it is for the correct one), but this is a side-effect of trying to look like common text. If you have a question for which there is a commonly held false belief or an answer that can be constructed out of common idioms and otherwise excellent grammar, the model is quite likely to report those instead of the real answer, because semantic correctness is not what a language model is trained for.

Trying to use a language model to generate code is like trying to use a submarine to fly to the moon. That's not what it's for; why are you trying to use it for that? Stop doing that. But at the same time, arguing that the submarine is bad at flying is rather missing the point. Nobody who actually understands NLP is claiming otherwise.3

There do exist systems that are designed to produce code, and trained to optimize correctness. (e.g. Genetic Programming). That's a bit too far outside my area of expertise for me to make any claims as to where the state of the art is on those, so I'm not sure whether answers generated by them should be allowed or not. But if you were to use an AI tool to generate code, that's the sort of thing you should be looking at; they're designed for the task. Similarly, you could ask if language models could be used as a tool to edit questions you've written by hand, perhaps to check the grammar or recommend new ways to phrase answers so they flow better. They'd be fairly good at that sort of thing (probably. I haven't used any of those tools myself (the rambling, stream-of-consciousness answer might have given that away), but the math supports the idea that they should work4). Translation is another task where (similar) systems work fairly well. (Machine translations still aren't perfect, but they're much better than they were 10 years ago, and improvement in language models is a big part of that.) Just always be aware of what tool you're using, and whether it's the right one for the job.


1 More formally, it gives the probability that a uniformly randomly selected English sentence of a specific length would be this one, but that gives the same ordering over sentences as long as we make some fairly reasonable assumptions.

2 Where "understands" is shorthand for "encodes the information in such a way that it can condition its decisions (i.e. probability distribution functions) upon it"

3 Well, not many. There'll always be a few who get caught up in the hype. They shouldn't.

4 If trained on well-written text

21
  • 29
    "Some level of actual understanding does exist in these models" — no. That is a philosophic conclusion not possible to support by science. It is exactly as unscientific as claiming that a computer that can pass the Turing test must therefore have a soul. I believe your statement is a perfect example of why Dijkstra warned against anthropomorphizing computer systems.
    – Wildcard
    Commented Dec 7, 2022 at 0:50
  • 29
    @Wildcard No anthropomorphization is intended, necessary, or useful. To be more precise as to what I meant by "understanding": All modern language models possess an embedding layer that projects words onto a real-valued semantic space. Since the probability distributions are defined as continuous functions over the resulting vectors, as the distance between two words in this space goes to zero, the model will treat them identically. It has been demonstrated that synonymous words do appear close together in the semantic space, therefore...
    – Ray
    Commented Dec 7, 2022 at 6:47
  • 26
    ...the system does understand that they have the same meaning (i.e. are largely interchangable). Further, it can be shown via the analogy task that the vector difference between non-synonymous words corresponds (in at least some cases) to the relation between them: the standard example is that embedding(king) - embedding(man) + embedding(woman) ~= embedding(queen). All this has been demonstrated experimentally. Whether "understanding" is the best word for what's happening here is unimportant: what matters is that the meaning of the words are verifiably encoded in the representation. ...
    – Ray
    Commented Dec 7, 2022 at 6:48
  • 10
    ...Additionally, in models that make use of attention mechanisms (such as GPT-3), the model can be shown to be able to determine which parts of a passage relate to each other by looking at the different parts it attends to at particular times. (although I'm not aware of any examples of this that are as illustrative as the analogy task is for the embedding function). There's no mysticism or philosophy here. Just math.
    – Ray
    Commented Dec 7, 2022 at 6:48
  • 9
    @Ray your clarifying comments are good, but the mere fact that an anthropomorphic shorthand statement of something can be more precisely described in objective or mathematical terms, does not signify that the original anthropomorphic wording was somehow correct. The question of whether "understanding" is the best word or not is precisely the philosophical judgment point which I was highlighting and decrying. Even your statement that the "model is able to determine...by looking" is highly anthropomorphic, in the exact way Dijkstra condemned. See lambda-the-ultimate.org/node/264
    – Wildcard
    Commented Dec 7, 2022 at 19:50
  • 12
    @Wildcard I've added another footnote to hopefully make that a bit clearer, but some amount of imprecision is inevitable if I'm writing for a general audience in a reasonable amount of space (even the three-part comment above was glossing over a lot of details). The real explanation is going to be several pages of math. If anyone wants more detail, I recommend Bengio et al. 2003, Mikolov et al. 2013, and Vaswani et al. 2017 as a starting point.
    – Ray
    Commented Dec 7, 2022 at 20:22
  • 9
    I said that anthropomorphization isn't useful, but I was perhaps overstating things; phrasing things in a somewhat anthropomorphizing way can be potentially useful as an analogy so long as everyone understands that it is intended as such and that all analogies are flawed. But I obviously didn't make that as clear as I needed to. So to be very clear: the important part of that paragraph was not where I said it "understood" certain things. It was the part where I said it doesn't reason about them. The models are in no way sapient, self-aware, or anything else along those lines.
    – Ray
    Commented Dec 7, 2022 at 20:28
  • 3
    @Wildcard No problem; I appreciate the comments. The last thing I would want would be for people to interpret my statements as supporting the more outrageous interpretations of these systems' capabilities when I was trying to make the exact opposite point. (A final clarification: "condition" in the sense of "conditional probability", not "classical conditioning"; "encodes" in the information theory/coding theory sense; and "decision" in the sense of sampling from a learned probability distribution.)
    – Ray
    Commented Dec 8, 2022 at 6:16
  • 5
    @RomanStarkov Then it's seen similar code and similar errors before, because it is incapable of reasoning about code flow (or the error is purely syntactic; self-attention plus a threshold value implicitly induces a subgraph over the elements of the text, so it's not implausible to say that the abstract syntax tree could be represented). You can't just look at examples of its output; the model is absurdly huge, and pattern matching will get you a lot of impressive anecdotal successes. You have to look at the math and ask what capabilities it's theoretically possible for it to have.
    – Ray
    Commented Dec 19, 2022 at 19:54
  • 2
    @Wildcard "how receptive people generally are here to constructive input" So refreshing when people in other fora often flip out over any minor correction.
    – nasch
    Commented Dec 20, 2022 at 23:55
  • 2
    @RomanStarkov The problem is you cannot be reasonably sure if what the AI does is correct, since this is not even what it was designed to do. Just because it gives the correct answer for this particular query, doesn't mean it will for the next (probably it won't). It doesn't "understand" the code you gave it (it does not even try to). It gives the answer that is the most likely English language text answer according to its training data. It is "just" a language model, "just" an insanely huge one, trained on a very big corpus of data.
    – neondrop
    Commented Dec 22, 2022 at 5:28
  • 2
    @Duane "English (or whatever language)", and of course its training corpora included code; otherwise, it wouldn't be able to produce code at all. But that doesn't invalidate anything that followed. I said, "It will favor models that produce syntactically correct text, use common idioms over semantically similar but uncommon phrases, don't shift topics too often, etc.", and aside from perhaps the last one, all those apply to code as well as natural language. But it includes no mechanism by which it could deduce the effects of a given piece of code, unless it's seen something similar before.
    – Ray
    Commented Dec 27, 2022 at 4:19
  • 1
    (continued) It's a bit tricky to come up with an example that it will definitely fail on, since it's seen a lot of existing code and can certainly work as a search engine (of sorts), but try asking it about the code samples I'm putting in the linked chat. I've obfuscated it a bit to reduce the chances of anything too similar showing up in the training data, but there's nothing here that a human couldn't figure out in 5 minutes by stepping through in a debugger. chat.stackoverflow.com/rooms/250666/…
    – Ray
    Commented Dec 27, 2022 at 4:19
  • 6
    I think people misunderstand of what ChatGPT is. It's not a code interpreter. It's a sequence predictor. So what's its good at is providing working samples of common tasks based on the hundreds of examples of those tasks its seen and all the documentation its read. It will tend to produce correct answers, because people do not commit incorrect answers to github that often. For example, this morning it explained to me the difference between setuptools, importlib and pip, and provided me with code to debug a failed package install. Basically, it's a promptable manual.
    – Duane
    Commented Dec 30, 2022 at 23:19
  • 4
    All of this "trust" conversation around code is silly. You act like the code coming out are 10,000 lines of code for operating missile systems... No one writes functions like that. The code sample is 50 lines of code that you can see what is going on and it tells you what it's doing. "Make this variable static", use a string builder here, etc. I want you to hand code a ReactJS component with a syncfusion listview and a textbox for searching. Good luck. Chat GPT will give you that boilerplate in 1 second. Commented Dec 31, 2022 at 9:11
120
  • The penalty for posting ChatGPT answers should be much, much harsher than 30 days.

    • Most of the people on SO are ........ computer programmers. It's remarkable that a coterie of computer programmers can be this "dumb" about ChatGPT.

    • ChatGPT knows literally nothing about - say - Swift and iOS. (Ask it almost anything to see this, say "How to convert degrees to radians in Swift." The answer is a mishmash of meaningless nonexistent calls, with perfect grammar and phrasing.)

  • A common problem on SO is, people who know nothing, posting grammatically correct and elegant answers, which are completely wrong, in a bizarre chase for points.

    • The only possible reason to post a ChatGPT answer on SO is such a bizarre chase for points.

    • Nothing is more annoying on SO than the "I'm trying to answer because I want to put in an answer" answers. Using a grammar-and-tone bot to paste answers in to SO is just madness. Anyone who does so should have the most draconian ban.

  • Just as, say, swearing on SO was easily eliminated by draconian bans, bot time-waste can very easily be eliminated on SO via draconian bans.

    • There's just no room here for free publicity by some faux AI project, ban it out of existence.
51
  • 11
    I've witnessed users showing up, delivering low quality answers to the constant stream of duplicate low hanging fruit (instead of voting to close as duplicates), and promptly stopping as soon as they reach a round number of rep. Commented Dec 16, 2022 at 3:10
  • 9
    @ChristophRackwitz - it's just infuriating. And the real solution is, all of us just need to tap delete more frequently.
    – Fattie
    Commented Dec 16, 2022 at 13:09
  • 17
    So I just tried it and got let radians = (degrees * .pi) / 180.0. This whole thread is so funny. SO days are numbered with this type of technology coming through. Commented Dec 31, 2022 at 9:15
  • 5
    there's already a system in place to downvote wrong answers.
    – ChatGPT
    Commented Jan 1, 2023 at 2:16
  • 21
    @MaxHodges - since chatgpt appeared, every single question I've needed to ask on SO, I've asked chatGPT first. in all cases it did not even produce coherent text. (Which is as one would expect, since - shock - there is no such thing as artificial intelligence yet, even vaguely, and if there was, it would be news on the scale of aliens arriving.) ChatGPT simply (effectively) googles, and then has a (excellent) engine which mimics the feel and grammar of an English corpus. Nobody would be more excited than me if there was an AI system that could answer SO questions
    – Fattie
    Commented Jan 1, 2023 at 4:21
  • 5
    @Fattie Not a single person came in here and claimed that Chat GPT is the end all of AI. It's an extremely useful tool for saving people time. Claiming that it's useless is like saying that Google search is useless. For code, it gives you the ability to get some pretty robust scaffolding and the ability to continually modify it based on contextual understanding. We already have scaffolding tools, but this takes it to the next level. There's no reason this can't be used to "scaffold" an SO answer for someone. Commented Jan 1, 2023 at 6:51
  • 4
    Hi @TheMuffinMan . I have no interest in what anyone did or didn't "claim", I'm stating what I feel like stating, as does everyone in comments. Note that, I don't give AF about SO (I wish I owned shares in it!). over time, seemingly "forever" companies, even google, apple etc and certainly SO will fade away and new "forever" companies will take their place. Regarding your comments immediately above, I precisely said that CGPT is "merely google". There is, literally, nothing you a gain from it that you cannot gain from google. That means that (as has been said repeatedly), exactly like google,
    – Fattie
    Commented Jan 1, 2023 at 15:38
  • 23
    ... CGPT is terrific for beginners answering the simplest questions. (Although utterly pointless, for beginners, because you just google for the identical info) But it's completely useless for (let's call it, for want of a better term) "research level questions" which is exactly what all programmers (other than just beginner hobbyists) use SO for. You may ask "why am I annoyed by people using CGPT on SO?" The reason is (as stated endlessly) a plague on SO is, beginner programmers with no real knowledge, "answering" questions, with no clue what they're talking about ...
    – Fattie
    Commented Jan 1, 2023 at 15:42
  • 7
    I'm afraid that's how I see it. If, incredibly, an "AI" system had arrived that can literally (as someone mentioned) read and understand documentation, track changes in (say) iOS, run experimental builds, synthesize existing clues, merge info from various fields, and come up with answers to actual (non-trivial) software questions - nobody would be a bigger fan than me. And if that happened the company that owns SO would have already dumped SO on to some other buyer. As it stands, the fluff on SO (non-experts rather pathetically trying to answer questions - who knows why) is just annoying.
    – Fattie
    Commented Jan 1, 2023 at 15:50
  • 5
    @joesan there's a lot of people who use SO for a purpose it wasn't meant for. For those people, chatgpt may be just as good for their needs. SO doesn't need to serve that purpose to remain relevant.
    – Kevin B
    Commented Jan 3, 2023 at 19:16
  • 8
    SO isn't a help desk, it isn't meant to be a place where you just dump your question and get an answer. Another service that offers that, whether or uses AI or not, would be better suited than SO for people looking for that kind of tool.
    – Kevin B
    Commented Jan 3, 2023 at 19:43
  • 5
    the issue at hand here is folks pasting chatgpt "answers" IN TO SO. if you wish to use chatGpt, whatever great. the issue at hand here is folks pasting chatgpt "answers" IN TO SO, which is incredibly annoying (to SO users, such as me).
    – Fattie
    Commented Jan 4, 2023 at 12:18
  • 15
    @MaxHodges "But I get perfect answers all the time." No, you don't, as you aptly demonstrated when you posted about ten of them to Japanese StackExchange. Pretty much all of them were plain wrong, so much so that it led to you getting a suspension from the site for a week. I struggle to see how you can maintain this doublethink that ChatGPT is perfect while you have clearly found through your own experimentation that it is not. Commented Jan 14, 2023 at 16:16
  • 8
    @Fattie is hitting the nail on the head. The reputation system, and people's need to game it for their own supposed benefit (be it professional, academic, or just for the attainment of a massive e-peen) is the root of the issue with AI. The most striking innovation in QA forums is also an Achilles heel. Commented Jan 18, 2023 at 17:46
  • 4
    @danh because there is no "one" answer that ChatGPT can provide. Your entire suggestion hinges on providing one answer. Where ChatGPT can provide you with a wide range of answers. A lot of them wrong as in "not even in the same ballpark" where the only critique that can be added is "That's not at all how any of this works". Which isn't really that useful to learn from. The real reason to ban it is that it's too often dangerously misleading and wrong on even rather basic topics. And users are spouting that nonsense on the site without any regard other than getting points.
    – VLAZ
    Commented May 15, 2023 at 19:41
87

If I was a lowly user who came across an answer that I suspect was written with ChatGPT, what actions should I take?

I can downvote the answer and leave a comment on why, if my privileges allow for it, but should I also raise a VLQ flag, or even a moderator flag? If I do raise a mod flag, should I only do this if I see the same user writing multiple answers with ChatGPT?

14
  • 87
    Please raise an "in need of moderator intervention" flag on the answer and explain what the issue is (i.e. explain that you believe the answer was generated by ChatGPT).
    – Makyen Mod
    Commented Dec 5, 2022 at 23:25
  • 7
    It doesn't look like anyone else has done it yet, so I've made the dedicated question post. I'll let an official moderator post an answer, rather than writing one up myself. Commented Dec 8, 2022 at 18:48
  • @Makyen: Perhaps be more sparing with this advice. I'd first ask the answerer: "Did you use ChatGPT to figure this out?" - of course they might deny, but they are quite likely to either provide an alternative source of knowledge, or simply not to answer, and in both cases one has more information to base a decision whether or not to raise a flag.
    – einpoklum
    Commented Feb 19, 2023 at 21:51
  • 4
    @einpoklum I'm not going to do so. You're welcome to spend that time if you want to. Unfortunately, we're talking about 10's of thousands of posts, so asking regular users (or moderators) to expend the substantial extra time to post a comment and then return to the post some time later to see if the OP actually responds doesn't scale. It just doesn't. We (people using the site, doing curation, moderators, etc.) just don't have the vast amount of additional time that would be needed. We, the site, don't have the curation resources available to handle each of these posts individually.
    – Makyen Mod
    Commented Feb 19, 2023 at 22:32
  • 5
    I'd also note that asking in a comment like that is contrary to our normal advice to users about how to handle suspected violations of the rules by other users. The advice to users is almost always "raise a flag; explain the issue you think might be there and why you think that; disengage". This type of thing is what moderators are supposed to handle, because it involves dealing with the user, not just the content of the post. Moderators can see the user's full history, which often allows them to see patterns which regular users don't have the information available to be able to see.
    – Makyen Mod
    Commented Feb 19, 2023 at 22:32
  • @Makyen Please consider moving your comments to my reply on the linked question (after which I can remove my comment here).
    – einpoklum
    Commented Feb 19, 2023 at 22:47
  • @Makyen apologies if the answer can be found here somewhere; the statement mentions "...generated text for content..." on SO but most of the discussion here is about answers. Does SO have a policy on material cited as for example the premises of questions, say of the form "Chatterot says X, is it true?" or more elaborate questions that can be reduced to that? Would they be covered implicitly by the word "content" in the statement, or explicitly but I'm missing it, or for the purposes here is the material in questions not considered content, or none of the above?
    – uhoh
    Commented Mar 26, 2023 at 9:45
  • 2
    @uhoh The answer to that is a bit nuanced. It's covered in answer to: "Should we flag human-written questions that use code generated by ChatGPT?". My personal opinion is we should be more restrictive about asking about how to fix AI generated code, because such questions are effectively useless to future readers. In my opinion, it would be much better for us to have a "How do I do X" question, rather than a question that is "Please fix this broken code that I got from this tool, which often produces complete garbage code, 'how do I do X'."
    – Makyen Mod
    Commented Mar 26, 2023 at 11:07
  • @Makyen Okay got it. There are of course some questions in SO that are about coding but don't ask for code as answers, just for example those from the family of *theory* tags but I think those will be edge cases and handled case-by-case. My interest in this is in part how it might translate to or inform individual SE site policies, cf. space.meta.stackexchange.com/q/2999/12102 where I took "content" to include question premises) (since SO often takes the lead on things. There, "Chatterot says X, is it true?" questions might become an irritant if not a problem. Anyway, thanks!
    – uhoh
    Commented Mar 26, 2023 at 11:18
  • 2
    @uhoh A) SO's policy (above) is that AI generated content is only in users' "about me". On another site, I'd either 1) ban such questions, or 2) A single canonical question something like "For [questions about site's topic], are AI generated responses reliable and accurate" and then close every question like what you describe above as a duplicate of that question. It's just a waste of effort to explain, over, and over, and over again that the current level of large language model AI generated content is "eloquent bullshit" and has absolutely nothing built in about being correct.
    – Makyen Mod
    Commented Mar 26, 2023 at 11:33
  • @Makyen yes agreed, like a canonical How do we know the Apollo Moon landings are real? target for closing as duplicate?
    – uhoh
    Commented Mar 26, 2023 at 11:46
  • 1
    @uhoh Yeah. Being able to close such questions as a duplicate of a general one that discusses the current level of reliability and accuracy for answers from AI generative technologies is about the only way to corral such questions, unless the site wants to go the further step of declaring them off-topic and have a site-specific close reason for them.
    – Makyen Mod
    Commented Mar 26, 2023 at 11:54
  • 1
    Hopefully, having a duplicate-target question available will result in fewer people being mislead by the "eloquent bullshit" which these things produce. Unfortunately, the number of people helped in a more general way regarding trust in current AI generated content is probably just a drop in the bucket, because most people care a lot more about getting an easy and quick answer, rather than a correct answer.
    – Makyen Mod
    Commented Mar 26, 2023 at 11:55
  • @Makyen slightly related to above comments space.meta.stackexchange.com/a/3057/12102
    – uhoh
    Commented Mar 31, 2023 at 22:23
68

Stack Overflow is a knowledge repository so I feel like it should be used to train AI models like ChatGPT, not the other way around. Why ask a question here if the answer can be already given by a bot somewhere else?

Also if/when the bot gives a lot of incorrect answers (and it's possible to churn out a lot of low quality answers in a very short time), who's going to clean up all the mess?

11
  • 3
    This above my pay level but stack training AI models, yea that is how it should be. meta.stackoverflow.com/a/421878/1815624
    – CrandellWS
    Commented Dec 6, 2022 at 9:38
  • 9
    Given its ability to supply plausible answers that do look like legitimate SO answers, I'm guessing it has been training on SO answers already. The question is when it starts using its own answers as more training data (creating a loop), does it get more or less capable? (Possibly smarter because it is taking human up/down votes and comments as feeback, or dumber because noise is amplified (like continually reprocessing a jpeg)
    – wojtow
    Commented Dec 8, 2022 at 1:04
  • 6
    10% of the questions on SO are simple, but 90% are actually novel. When tvOS is released and nobody knows how to parallax a button, it is puerile to think a bot can "answer" that question.
    – Fattie
    Commented Dec 10, 2022 at 4:15
  • 1
    @Fattie I suppose it could theoretically read the data from tvOS (source code, documentation, executable files, etc.) and make use of that, but getting that to work well would likely be a massive research project. More likely, it could help the users who (try to) post duplicates or near-duplicates, or at least come up with ideas for commonly-similar problems, rather than wasting other users' time. Commented Dec 12, 2022 at 19:21
  • 4
    Solomon just FWIW what you mention (reading the tvOS doco, understanding it, doing test research, creating a solution) is completely inconceivable with today's technology. (It would be like saying "travel at 1/2 light speed to mars".) the only thing "chatGPT" does is randomly formulate sentences, that have the rhythm and grammar of the example corpus of text". That's it. (The only reason it sometimes "answers correctly" is that it randomly munges up text on the topic, which, is likely to be correct-ish. It does not even *understand what a "question" is. )
    – Fattie
    Commented Dec 12, 2022 at 19:34
  • Isn't ChatGPT already training on StackOverflow probably? It's just consuming internet stuff so it would Suprise me if wasn't already.
    – GaryFurash
    Commented Dec 21, 2022 at 20:13
  • Nobody wants to pipe the output of cGPT into SO directly. And assuming that cGPT knows it all is an indicator for lack of understanding LLM
    – Summer-Sky
    Commented Dec 23, 2022 at 16:32
  • 1
    @Summer-Sky i mean... that's provably false, there's at least a dozen answers here suggesting exactly that. (several may be deleted.) People using it in this way is why this temporary ban exists.
    – Kevin B
    Commented Dec 23, 2022 at 16:34
  • @KevinB yes here they are like cjn answers to their questions and wondering that it could not be answered correctly ... anyway if You find the ominous other answers that were regarding LLMs as an assistive technology and/or yall get off of your ableism I would be glad if you could add a link to those at my answer meta.stackoverflow.com/a/422306/3623574 Also to make it clear I am looking for a compromise. I am also against piping from cGPT to SO mery christmas for now
    – Summer-Sky
    Commented Dec 23, 2022 at 16:42
  • 2
    Noone here is speaking against assistive technologies; you're simply using that as an excuse to summarily dismiss the problems this tool has caused. Until these problems can be dealt with while allowing it to be used as an assistive technology, it's dead in the water.
    – Kevin B
    Commented Dec 23, 2022 at 16:44
  • Stack Overflow is a knowledge repository so I feel like it should be used to train AI models - rest assured it absolutely is, without us knowing.
    – sunny moon
    Commented Feb 14, 2023 at 10:40
59

After several months, I noticed a big difference between the question-and-answer websites that allow and those that prohibit ChatGPT. I used to use a website (called Jingyi forum(精易论坛)), which used to be good because it enabled me to communicate more easily (as English is my second language).

I asked a question about C++ on that website recently. However, after I posted my question, the only one who answered my question was using ChatGPT. What surprised me even more was the administrator did not ban him even though the answer is completely wrong. Comparing Stack Overflow with that website, I can get better answers in a shorter amount of time on Stack Overflow.

What's more, people on that website, who tried to help people, have started to not be willing to answer questions. They take a long time to write an answer to a question, but they can get less reward than those who use ChatGPT.

6
  • 1
    Could you please tell us the name of the Q&A website? It's public, isn't it?
    – Mari-Lou A
    Commented Jun 13, 2023 at 6:38
  • 5
    @Mari-LouA Sure, It is called Jingyi forum(精易论坛).
    – Han Han
    Commented Jun 13, 2023 at 14:45
  • 6
    That is very interesting. So it may actually be ChatGPT that is the direct cause of fewer people willing to answer on Stack Overflow (and not the handling of the ChatGPT plagiarisers). Why would it be different on Stack Overflow compared to the Jingyi forum? (Yes, that is a rhetorical question.) Commented Jun 13, 2023 at 21:09
  • 12
    it's sortof a... at a certain point, the prestige of the gamification system is lost if "cheaters" are allowed to earn it with impunity. We've already seen effects like this even before chatgpt, with new answerers feeling like it's hopeless to begin because high rep users have so much rep and it's so hard for new users to gain it.
    – Kevin B
    Commented Jun 13, 2023 at 21:13
  • 15
    Though you're talking about a traditional forum, I think you might be onto something. From the dawn of Stack Overflow, well before ChatGPT, people have noticed that it really sucks to not be the fastest answer. It's called the Fastest Gun in the West Problem. Some people won't even write an answer if there is an existing answer, especially one that's accepted. And it's really not possible for a person to be faster than ChatGPT, especially not when the problem is non-trivial and you care about accuracy more than ChatGPT.
    – Laurel
    Commented Jun 13, 2023 at 21:46
  • 4
    There is discussion about it on meta.se about it now here. Commented Jul 6, 2023 at 13:02
58

This calls for a to detect AI generated answers/questions and maybe an additional flag option for users to mark a post if an answer/question is suspected to be one.

An interesting point here is that Deepfake detection is a big area of research but AI generated text detection is still lagging behind a bit. Hoping the community comes up with good models soon that help detect ChatGPT generated content.

For the people suggesting ChatGPT can “help” SO, please know, the biggest differentiator of SO from other Q&A platforms is the fact that some of the most brilliant programmers in the world are directly guiding the community, and the rest of us learn from their answers to then guide others who need help.

Who would you rather learn from? A veteran programmer or a random person with a AI text generator? Because if SO allows this, be rest assured this is going be be exploited beyond control.

20
  • 18
    This sounds like...not a good use of time or resources. The community should be evaluating the quality of the response, not necessarily if it came from an AI or not. But your other points I can agree with.
    – Makoto
    Commented Dec 5, 2022 at 22:15
  • 9
    Professor Turing, please call your testing office... Commented Dec 5, 2022 at 22:32
  • 6
    @Makoto: Having a way to get tons of bad answers that look good on the surface deleted is a new problem that might warrant new tools. If a comment that it looks like AI-generated garbage is enough to swing the tide of voting towards downvoting when deserved, then maybe not. But the community helping mods identify users who post this kind of AI crap might be helpful to get them suspended if normal rate-limits don't block them. (Either because they fooled enough folks into upvoting garbage, or they had some rep to start with but still failed to grok how terrible this is.) Commented Dec 6, 2022 at 0:50
  • 4
    @Makoto - the community helps detect a lot more than just quality, right now as well. Moreover, just adding this flag is not a misuse of communities time, because that is a function of how much AI generated content actually gets posted on SO. As the mods comment on another answer mentions, users still would be flagging it for mod intervention. Infact it will help reduce the burden on the mods who will have to review each answer and will help cleanup SO faster. Finally, a flag like this will help lead research on detecting AI generated content, if the data is made available by SO. Commented Dec 6, 2022 at 5:59
  • My main thing is that maybe the expert I need can answer the question but hates writing and may only answer my question if he can simply verify the AI is correct and copy and paste that to me... I would appreciate that because the fact AI wrote it matters not to me... it being an accurate or correct answer does though, I mean I can not afford the people I tend to need help from and if AI makes it faster for them to correctly answer my question and that is my only option I dont want to eliminate it because they may be to busy otherwise to answer at all
    – CrandellWS
    Commented Dec 6, 2022 at 10:07
  • and I am not suggesting that experts be correcting AI writings that is a waste of time. Rather if they can just check it and then send me a correct ai written answer I fully welcome it and if they check the AI answer and deem it incorrect and dont share I like that to
    – CrandellWS
    Commented Dec 6, 2022 at 10:11
  • 6
    @CrandellWS: In a perfect world, people would use AI to generate an answer, then test it, verify that the AI answer is correct, and only then post it on SO. Unfortunately, this is not what's happening right now. We have tons of users copy-pasting AI answers to Stack Overflow without checking at all. Some of the answers are correct. Some are total nonsense. Only today (in the last three hours), I saw at least 5 users just copy-pasting without any checking. They posted answers in wrong programming language, self contradicting answers, and so on.
    – BDL
    Commented Dec 6, 2022 at 10:12
  • 8
    @CrandellWS: Checking those answers takes considerably more time than what is acceptable. You can't expect volunteers on SO to serve as a review service for AI generated content. Either the author of the answer checks for correctness before posting, or AI answers aren't possible.
    – BDL
    Commented Dec 6, 2022 at 10:14
  • @bdl I get that I support extreme temporary measures till a solution is found... no matter how restrictive or nonrestrictive the extreme is
    – CrandellWS
    Commented Dec 6, 2022 at 10:14
  • @BDL I agree but we in a flood and I am just saying a spillway might be a good idea
    – CrandellWS
    Commented Dec 6, 2022 at 10:15
  • 1
    @CrandellWS, This is what other folks have previously mentioned. How do you know that an "expert" has verified it before posting it? Regarding the "lack of desire to write", SO is a community. You will always find people active here. Whether for the act of solving problems, gaining a reputation, building their portfolio or just boredom, there will be people who will answer your question, if it's constructed and written well. And, just to further motivate the "big league", there is a bounty feature. Commented Dec 6, 2022 at 16:54
  • 1
    If only the curation/moderation tools were flexible enough to create new flag categories on the fly
    – Kevin B
    Commented Dec 6, 2022 at 17:08
  • 1
    A public temporary ban, such as this one, and mods enforcing it, is enough to deter well-meaning users from abusing it. Nothing short of a ban will stop the others. That's better than doing nothing at all until a better solution exists.
    – Kevin B
    Commented Dec 6, 2022 at 17:21
  • 5
    Solutions will come sooner or later. Academia uses methods to detect AI-generated plagiarism (models that modify input text while not changing context), Media/Content platforms are using Deepfake detection models, and there are a lot of AI-generated text detectors out there, that can and will be finetuned for ChatGPT once the model is made available officially. SO is not the only platform that will be negatively impacted by the misuse and exploitation of this model. For now, the temporary ban is a great strategy to assess the impact vs hype situation around the model. Commented Dec 6, 2022 at 17:49
  • 1
    Who said anything about "Einstein of programming"; just because you know/mastered a specific concept in programming doesn't make you an Einstein. That's a childish analogy. The whole point of SO is that it's a community-curated platform. If something is not answered by someone correctly, the community corrects it, and improves/maintains it over time. If you were following anything around ChatGPT, its clearly being recognized that it is poor in its accuracy and ends up just generating "confident" sounding answers. The issue is the misuse of ChatGPT by folks that have 0 knowledge of that concept Commented Dec 31, 2022 at 9:48
33

The discussion point is not whether or not AI-generated answers should be allowed. It is more general about what to do with users posting low-quality answers and not following the etiquette of Stack Overflow.

Banning these answers is the correct thing to do, but it is a systematic problem in the user behaviour then fixing the user behaviour is a more robust solution, as in the end, low-quality AI answers cannot be distinguished from low-quality answers.

  • Low-quality answers might be posted even without AI by unskilled users
  • Understanding the behaviour of users who are posting AI-generated answers is more important: why Stack Overflow answers are important for them and what do they believe gaining from posting useless answers
  • If the volume of the low-quality answers, AI sourced or not, is too high, then tackle this problem by increasing the bar to post an answer
  • If it is not individual cases but systematic, then normal discussion forum tools can be used to identify toxic accounts by behaviour tagging, IP address, and so on

Edit: Looks like there is another Meta discussion already opened on this topic: Stricter trust model in the face of bot flood?

5
  • 8
    I generally agree. Although it should be considered that these AI tools reduced the effort required to post answers that are low-quality but look OK on the first glance. Most other low quality answers have (in addition to giving wrong answers) also spelling and formatting problems, incoherent writing style and other such things that allow for easy identification.
    – BDL
    Commented Dec 5, 2022 at 22:08
  • 4
    I think Journeyman Geek explained it well on MSE Ban ChatGPT network-wide
    – Dharman Mod
    Commented Dec 5, 2022 at 22:14
  • 4
    "as in the end, low-quality AI answers cannot be distinguished from low-quality answers." That's demonstrably false. Plenty of users have already been suspended for posting CGPT-generated answers. These answers follow patterns that are recognizable.
    – Cerbrus
    Commented Dec 5, 2022 at 23:14
  • 3
    @Cerbrus - you are correct; this is the situation now. However, it is just a matter of time before ChatGPT gets more variety in its text, or other alternative AI models reach the same level. Unless you count spelling errors of non-English speaking users, it will get harder over time to separate AI-generated low-quality answers from other low-quality answers in medium term future. Commented Dec 6, 2022 at 17:37
  • 1
    @MikkoOhtamaa The more you train a model, the more it's output is going to look like each other. More training isn't going to diversify the output.
    – Cerbrus
    Commented Dec 12, 2022 at 6:56
31

The content definitely needs a ban, if for no other reason than to make it easier to have the discussion here instead of all over the place when it gets flagged.

ChatGPT even acknowledges the pitfalls that are described, in brief:

Limitations
ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
-ChatGPT: Optimizing Language Models for Dialogue

The real question here in my opinion is enforcement though. What are the penalties for using this content? Is the user summarily subject to a ban as well, or subject to a series of penalties leading up to a ban?

2
  • 8
    I've seen people get hit with suspensions for posting ChatGPT answers under the existing "plagiarism" ban reason, and not just here on Stack Overflow, but on other SE sites as well.
    – F1Krazy
    Commented Dec 6, 2022 at 23:35
  • 6
    first offense should be an instant 90 day ban. anyone who would post bot content in answer to technical SO questions, is not a programmer and is likely an idiot. NOTHING is more annoying on SO than clutter, and for the owners nothing is more value-destroying to the brand
    – Fattie
    Commented Dec 10, 2022 at 4:13
31

Additional cases for permanent ban of answers

In addition to completely agreeing with this for all the reasons already stated in the other answers, and also feeling like it should be permanent (also for reasons stated many times elsewhere), I think there is an additional case for making it a permanent ban that I don't see covered elsewhere:

Under the presumption that anybody can just go to a public GPT instance, type their question, and get a similar (if not identical) answer, then allowing those answers to be posted on SE sites essentially means two things:

  1. It means, to some extent, that the question itself lacked research: If it was that easy to get an answer (ask a bot) then the asker probably could have done that. In some SE communities this is fine, but for communities where lack of research is generally frowned upon, allowing these answers to be posted essentially encourages questions that the community does not generally want.

  2. More importantly, it means that the Q/A pair is, for all intents and purposes, simply a bot's chat log. If I can ask the bot a question and get an answer, then I put that question and answer on an SE site, all I've really done is duplicate information that already exists on the internet. While certain SE sites definitely have their share of duplicate information (e.g. SO has a lot of questions whose answers can be found in documentation), it's still generally of no value to add additional duplication, especially when that information is just a copy of a Q/A session that can be had with a given GPT bot at any time. Duplicating logs of chats with bots doesn't really add any value to the internet. SE ultimately serves the purpose of getting knowledge out of small groups of peoples' heads and into large groups of peoples' heads, but this just duplicates what's already out there.

Therefore, because encouraging questions that communities don't want is obviously undesirable, and because duplicating logs of chats with bots doesn't really add any value anywhere, I think the ban should be permanent. In addition, of course, to all the other reasons given.


PS Furthering the above two points: These bots are trained on existing information, and as such they're not really creating new information. They're effectively just search engines that present reorganizations of existing information in the form of readable text. So asking them a question is roughly equivalent to Googling for a question and interpreting the results. In my opinion, bots should be treated with the same attitude as search engines are treated. And answers from bots should be treated the same as answers that are just copy+pasted Google results (i.e. valueless plagiarism).

2
  • 21
    "They're effectively just search engines" Minus the fact that search engines would at least provide the source of results and the original nature of the documents indexed, which we do not even have here. And attempts at requesting ChatGPT to provide references for a scientific research subject would often yield made up article titles. In this regard, it is worse than a search engine.
    – E_net4
    Commented Dec 27, 2022 at 12:09
  • 4
    The duplication battle was lost a long time ago. The gamification encourages submitting answers to blatant duplicates, not to find the canonical questions. Search engine output is also broken; on the whole they refuse (for whatever reason) to return the canonical questions from 2008 and 2009. Commented Dec 30, 2022 at 15:36
27

There's yet another aspect:

People (at least I do so) come here for help from real experts – if I wanted to get an answer from ChatGPT, Bard or whatever else AI engine I can go there and ask myself! So I join in the request to permanently ban any answer from any AI engine.

I'd even go a step further: People repeatedly answering with AI generated content should get reprimanded, and if repeatedly ignoring maybe temporarily get locked out from answering entirely.

3
  • 5
    It was the case until a couple weeks ago, when SE barred moderators to take any action in regards of AI-generated content. Right now there is a moderation/curation strike with goal of lifting new AI policy.
    – markalex
    Commented Jun 14, 2023 at 14:49
  • Against that I would say that many domain experts often don't have the time to write out a detailed Stack Overflow answer. With their specific knowledge it's very easy for them to determine if a ChatGPT answer is any good, and if it is any good to post it. That was the case with this ChatGPT answer e.g.: stats.stackexchange.com/questions/76925/…. I knew that to be correct, working in that field, so I posted it, but would never write out such a detailed answer myself. Commented Jun 15, 2023 at 18:07
  • 5
    You would, though, happily accept the reward for doing nothing, ;)
    – Kevin B
    Commented Jun 15, 2023 at 18:09
23

Use of ChatGPT generated text for content on Stack Overflow is temporarily banned.

Something worth clarifying, I think, is that although most of the discussion here has centred around answers written by ChatGPT, the statement of the ban also applies to questions, and I think that is good - questions written by ChatGPT, or especially questions with code written by ChatGPT, should be banned, even when the user does not try to pass them off as their own writing.

If somebody asks a question like "I used ChatGPT to generate this code, but it doesn't work, why not?", then generally the correct answer will be "because ChatGPT wrote it, and ChatGPT shouldn't be expected to write correct code." It is not useful for questions along these lines on Stack Overflow to get more detailed answers than that, because other readers probably aren't interested in why that exact code someone else got from ChatGPT is broken. So such questions ought to be closed.

Likewise, questions where especially beginner programmers post a problem (e.g. from their homework or a textbook) and ask how to solve it are often asked to show their own attempt in the question. If an asker tries to get around this by showing an "attempt" written by ChatGPT, that is not sufficient to make for a good question. The reason we ask to see an attempt is because otherwise there is no way to know what level of understanding an answer should be tailored to, and an "attempt" by ChatGPT doesn't address this.

3
  • 2
    Re If an asker tries to get around this by showing an "attempt" written by ChatGPT, If a OP posts code that's not their own work from any source, without attribution, that's plagerism, and should be delt with as such. Commented Mar 16, 2023 at 5:16
  • 2
    Taken literally, this doesn’t seem to be in line with current policy; especially questions about code written by ChatGPT and especially if when the user does not try to pass them off as their own writing are fine. FWIW, most questions asking "it doesn’t work, why?" or homework or a textbook tailored to a specific attempt is something most readers definitely aren’t interested in - no matter if the attempt is from ChatGPT or not. Commented Mar 16, 2023 at 6:20
  • 2
    @MisterMiyagi Thanks for pointing me at that other Q&A. My post here is an argument in favour of a policy, and it looks like the other Q&A is the place for discussion about that policy, and that my position is not a consensus one.
    – kaya3
    Commented Mar 16, 2023 at 6:36
22

I think the ban should be permanent. Stack Overflow needs answers from real people who have experience and expertise. I don't think answers from AI can and will solve most users' problems here.

2
  • 7
    Have you already read meta.stackoverflow.com/a/422343 and meta.stackoverflow.com/a/422096 (there maybe others but beside this answer only these other uses the word "permanent") ? The first is in favor of a permanent ban while the second is against it. Have you voted thoses answers?
    – Wicket
    Commented Jan 3, 2023 at 18:13
  • 1
    Humoristic approach: Add a footnote to every high quality post "This answer was created by a Natural Intelligence (NI = a real person). No AI was used, harmed or bothered during the creation of this text." ;-)
    – Matt
    Commented Mar 23, 2023 at 10:50
19

I heard about the ban but didn't really look into it. Thinking it through, I can't help but agree with this decision. Accuracy aside, if people wanted answers from ChatGPT they should go to ChatGPT. People come here to interact with humans, not middlemen who parrot AI responses.

5
  • 8
    I don't disagree with your position, but... what did this really add to the discussion here on top of YungDeiza's answer and cottontail's answer? Reading and voting are just as valid ways to voice yourself than writing an answer post, and I'd say preferable in the case that nothing new is added to existing discussion.
    – starball
    Commented Feb 3, 2023 at 21:50
  • 18
    it's easier to just post another answer rather than read 76 others
    – Kevin B
    Commented Feb 3, 2023 at 22:08
  • 4
    @KevinB once upon a time I proposed reading prior answers as a prerequisite for posting when question already has many answers
    – gnat
    Commented Feb 10, 2023 at 21:36
  • 2
    @gnat I'd prefer, action be taken when posts receive more than n answers by users with the power to do so to prevent them from reaching the point where it's absurd to expect people to read several dozen answers to avoid repeating one. I won't sit here and blame users like this one from answering when there's 80 answers. It's not their fault
    – Kevin B
    Commented Feb 10, 2023 at 21:38
  • 1
    @KevinB FWIW proposed prerequisite reading was supposed to kick in long before anything absurd happens - at less than 10 answers (ideally, 5 or 6). That's because it was supposed to help folks like me who read all prior answers prior to posting anyway. Primary purpose was to make things more convenient for myself and you can think of taming the issue of piling thoughtless repeats more like as kind of a beneficial side effect
    – gnat
    Commented Feb 10, 2023 at 22:42
18

I agree that there should be a temporary ban, because many users will use the chatbot to generate answers that seem to be correct but may be incorrect in reality. It push the content hit bad. Because Stack Overflow entirely depends on volunteers, it becomes difficult for them to verify every answer. Copy pasting answers with the use of bots takes seconds, while proof reading them and making sure they deliver value takes more time.

18

The problem with ChatGPT is that it's a poor fit for those answers it should/could be used to answer.

On one hand there's well written questions on SO with a clear problem statement, a nice small snippet of code that reproduces the problem, a clear error message. Just overall good quality and clarity.

These questions are the easiest for CGPT to interpret. They're the most likely to get good output from the AI.
These are also the questions that are unlikely to get closed and most likely to get a human answer.

On the other hand there are those questions that are unclear, lacking a proper problem statement, lacking error messages, poorly formatted code, if there is any at all...
Those questions would benefit most from an AI that could figure out the problem and answer it.

Those are the questions that CGPT will write good-looking crap answers for.

So even when the bot produces some gems... They're not useful on SO.


TL;DR: Crap in, crap out.
The questions that need this bot can't benefit from it.

10
  • 2
    I've arrogantly tried it on a couple of my own questions assuming those are good, but it failed miserably. E.g. this one, a VBA terminate handler not firing due to a reference loop between the class and some built-ins, it suggested "move the code to the initialize handler, then it'll fire" 🤦‍♂️. When asking for unicode normalization for password storage, it provides an example that mojibakes the string...
    – Erik A
    Commented Dec 8, 2022 at 11:44
  • 2
    @ErikA: "They're the most likely to get good output" That doesn't necessarily mean they will be good :D
    – Cerbrus
    Commented Dec 8, 2022 at 12:16
  • It's currently 0/8 out of the reasonably short questions I've asked, so if that's the most likely, well, that's not great. And the answers it does come up with fall either into "plain wrong code", "an imaginary fun/arg that solves your problem but doesn't exist" or "you've missed the entire point of the question". I don't think CGPT can answer anything but the most trivial questions, even when the question is well formulated. And those trivial questions are likely duplicates, of course.
    – Erik A
    Commented Dec 8, 2022 at 12:36
  • That's basically my point :D
    – Cerbrus
    Commented Dec 8, 2022 at 12:36
  • 7
    Your answer makes it sound like it can answer clear, well written questions that could get answered by humans. My point is: that's only true if the answer is trivial too, since cGPT fails to solve clear and short questions with slightly difficult answers that humans can answer just fine. It even fails to interpret them and changes them to something that it has an answer to.
    – Erik A
    Commented Dec 8, 2022 at 13:01
  • 1
    It can, but there's no guarantee. Nowhere am I claiming that writing a good question guarantees a correct answer. It's just more likely.
    – Cerbrus
    Commented Dec 8, 2022 at 13:16
  • 7
    Piling on examples of CGPT producing garbage, I tried it in this question over on Vi.SE, where CGPT recommended to change font to guifont (which isn't a valid property in that context). When I told it that, it said to use font, and when I told it that doesn't work, right back to guifont. (I've more or less established that particular problem as a bug in Vim though, so it producing anything useful was already extremely unlikely. Was for science, and to see what it would answer, but makes for a great example) Commented Dec 8, 2022 at 13:21
  • 1
    While language models cannot provide all of the things that are demanded of them, they can still be an assistive technology in two ways: by summarizing long text for easier ingestion or by helping to write it in a well-structured manner.
    – Summer-Sky
    Commented Dec 23, 2022 at 16:28
  • Is there some data to measure the question rate? I mean after chatGPT was launched, did SO notice a drop in the question rate? Are there some numbers that compares pre and post chatGPT release having an effect on the question rate per minute here at SO?
    – joesan
    Commented Jan 25, 2023 at 5:32
  • I don't have any numbers, but multiple mods have stated that question range didn't drop at all, @joesan
    – Cerbrus
    Commented Jan 25, 2023 at 8:32
17

I don't see any way to 100% prevent AI-assisted answering.

The #1 most-effective way to prevent the flood of AI-assisted answers on Stack Overflow...

Quickly close all closable questions.

Hammers? Use 'em if you got 'em.


The FGITW answerers were fast before, now they'll be faster.

Perhaps this is a call for offering silver/gold badgers abilities to close pages with greater speed.

Perhaps we should remove any earned rep if a page is closed within n days of being asked. This way askers still get the answers that they need, but there will no longer be a reward for answering questions that should be closed.

I think I'll be a lot happier when AI can accurately assist me in finding good dupe targets (ideally canonicals) before anyone posts an answer.


P.S. Should we mandate that answerers explicitly declare the use of AI assistance? such as <sub>declaration</sub>?

This answer was assisted by artificial intelligence.

15
  • 6
    "Quickly close all duplicate and off-topic questions." doesn't work, sorry. "The FGTIW answerers were fast before, now they'll be faster." quite often a question would get one or more answers by the time it's hammered. Within the span of 2-3 minutes. I'm sitting on newest questions and trying to action them ASAP normally but even then finding a dupe might take longer than writing a simple answer. One question I closed in less than 5 minutes had FIVE ANSWERS by the time I hammered it. Now it's even simpler to write answers.
    – VLAZ
    Commented Dec 7, 2022 at 6:30
  • 11
    CGPT's terms even state that you must state that the content was AI-generated, that you may not pretend it to be your own.
    – Cerbrus
    Commented Dec 7, 2022 at 7:00
  • "This way askers still get the answers that they need" Why is that even needed? That is not the purpose of Stack Overflow, there are other sites for that. But "we should remove any earned rep if a page is closed within n days of being asked" sounds like a good idea!
    – wovano
    Commented Dec 7, 2022 at 7:52
  • Are you asking: Why should askers get the answers that they need on a Q&A site??? @wov Commented Dec 7, 2022 at 8:31
  • 2
    @mickmackusa, no, I'm asking: why should questions that should be closed according to the site rules (because they are off-topic) get an answer anyway to please the (ignorant) asker? Obviously, good on-topic questions (that should not be closed) deserve an answer.
    – wovano
    Commented Dec 7, 2022 at 8:35
  • 1
    What is an FGTIW answerer?
    – Lomtrur
    Commented Dec 7, 2022 at 9:43
  • 4
  • 4
    @wov not all dupes are bad. Stack Exchange does not mind a few unique signposts that point to a single dupe target. Duplicate-ness may be disputable, so allowing answers isn't ridiculous. Commented Dec 7, 2022 at 10:13
  • @Lomtrur Well, it should be FGITW but they went for "Fastest Gun The In West" problem instead.
    – user692942
    Commented Dec 7, 2022 at 10:13
  • @mickmackusa, I agree my remark does not fully apply to questions closed as duplicate (although I personally favor having all answers in one place instead of scathered around over many duplicate questions, but YMMV). However, it does apply to questions closed for other reasons. You state so yourself on your profile: "This is why... I ask others not to post answers to questions that are incomplete, duplicates, or otherwise close-able."
    – wovano
    Commented Dec 7, 2022 at 10:17
  • 2
    I think closing would be much more efficient with the help of active tag regulars. They currently tend to abstain of this because system pushes moderation of unfamiliar tags down their throats. Instead of bothering them with stuff "outside" they should just get opportunity to moderate where they are comfortable doing it, with tag-filtered triage and close voting powers enhanced by tag badges
    – gnat
    Commented Dec 7, 2022 at 11:40
  • 1
    How about a reputation penalty, each time a user is identified using AI-generated answers? Just a thought. Commented Dec 7, 2022 at 11:49
  • @AkshaySehgal many users who post a deluge of CGPT answers are new. They have 1 rep. Or have started posting the answers at 1 rep. There isn't really that much to "punish" them with. It might work for higher rep users who have started posting generated answers, however, I'd lean against that. Simple temporary ban (which is currently being issued) should put the message across. No real need to make a permanent mark.
    – VLAZ
    Commented Dec 7, 2022 at 15:04
  • 5
    Re P.S. Should we mandate that answerers explicitly declare the use of AI assistance? - we already do. Posting an AI generated answer as your own, without attribution is plagiarism. Commented Dec 10, 2022 at 9:04
  • 3
    @chrisneilsen, and (as already pointed out in one of the other answers) OpenAI requires this as well, so not doing it is a violation of both site rules.
    – wovano
    Commented Dec 10, 2022 at 9:15
14

I think unmodified answers from ChatGPT should be banned, but if you use ChatGPT to generate an answer and then independently verify it and correct it to the best of your knowledge as needed, that should be allowed. It can be a useful tool, but simply taking answers from it as-is is often unhelpful.

19
  • 47
    I don't think this needs to be explicitly exempted. It usually takes more time and effort to verify a machine-generated answer than to write one yourself, with less garbage and verbosity included.
    – iBug
    Commented Dec 5, 2022 at 12:14
  • 1
    yeah i guess you're right, it probably has more value for asking about specific details than answering the entire question.
    – user492203
    Commented Dec 5, 2022 at 12:23
  • 3
    "unmodified answers from chatgpt should be banned" But how does one detect if the output from chatgpt was taken unmodified or not? Commented Dec 5, 2022 at 12:55
  • 2
    you can't necessarily tell for sure (well, you can try asking it the same question and see if you get a similar answer), but if it's an obviously low-effort/incorrect answer then it should be deleted. if there are no issues with the answer then it shouldn't really matter if chatgpt was involved in it.
    – user492203
    Commented Dec 5, 2022 at 14:05
  • It is a useful tool to help yourself, not others. I see chatgpt as an alternative to posting a question on Stack Overflow.
    – Gimby
    Commented Dec 5, 2022 at 15:00
  • 23
    If users are taking the time to go through an auto-generated answer and verify its correctness before posting, credit its sources (even if this is actually possible for an auto-generated answer) and avoid plagiarism, etc. – and posting them at a reasonable rate (rather than flooding the site with bad answers), and improving them based on feedback in comments (rather than just dumping them on the site and then abandoning them) – then I would imagine it's harder to tell that they're even auto-generated. But at that point, there's little differentiating it from an answer fully written by a human.
    – V2Blast Staff
    Commented Dec 5, 2022 at 15:47
  • 20
    Relevant XKCD: xkcd.com/810
    – V2Blast Staff
    Commented Dec 5, 2022 at 15:51
  • 8
    @iBug I use ChatGPT every day for my work as a software developer and I disagree. It's a much faster version of StackOverflow basically. Commented Dec 12, 2022 at 22:21
  • 3
    @NikS As others have pointed out multiple times: It's much faster at producing plausible answers that poses a greater problem to Stack Overflow than it seems.
    – iBug
    Commented Dec 13, 2022 at 7:27
  • 3
    if you can verify it, then you should just write the answer yourself. "correct it to the best of your knowledge as needed" and what if you don't have enough knowledge, you will let ChatGPT's mistakes slip in? this is a bad idea
    – symbiont
    Commented Dec 19, 2022 at 0:28
  • 3
    @V2Blast More relevant than that: using ChatGPT could prove self-referential (which is not a good thing at all)
    – Machavity Mod
    Commented Dec 28, 2022 at 13:44
  • 2
    I love how existing scaffolding tools in Visual Studio/code aren't under fire, but using GPT is. This is the whole guns kill people instead of people (with guns) kill people argument. There's a lot of immaturity in this entire post. Furthermore, the arguments being made here act like GPT is spitting out 1000 line methods that no one understands. If you copy paste an answer from anywhere on SO without vetting it that is the source of the problem, not whether it came from GPT, a VS extension or MSDN forums. Commented Jan 1, 2023 at 6:58
  • 2
    @iBug - "... takes more time and effort to verify a machine-generated answer than to write one yourself ..." is probably true if your first language is English. However, for non first language English speakers, writing your own answer could be far more time-consuming than verifying and tweaking an AI-generated one. Commented Jan 17, 2023 at 4:38
  • 1
    Nothing we can do here would inform on how correct it is in a way that would be valuable to it's improvement. @Rahul
    – Kevin B
    Commented Feb 7, 2023 at 22:49
  • 1
    "I think unmodified answers from ChatGPT should be banned" how do you prove this, and there is no point banning something that can't be proven.
    – DubDub
    Commented Apr 3, 2023 at 16:03
14

I think this is the right decision.

The artificial replies can sound authoritative because they may have better grammar than the human contributions. The reply generators are tireless. They could easily overwhelm the human authorities.

The ban is temporary.

All things are temporary. The ban should stay for as long as it is needed.

The artificial replies cannot be trusted.

We are seeing good and bad answers. There are many bad answers just now, but AI progress is rapid. They may be a valuable tool quite soon. Not in a hundred years? They are better than they were last week.

Quality is an issue, but it is not the whole issue. We should not invite AI replies once their quality equals our human contributors. We should exclude them until we understand what their replies are, and the nature of their failings.

We cannot tell whether a reply is from AI or not.

This is true. If we can tell the difference, then the AI can be trained to see the difference and correct for it. Some people may take ChatGPT answers and post them as their own. Others may post replies from their own model to test it, and to gather training data. A simple filter, such as limiting the reply time to a typical typing speed, may stop us getting swamped. But it won't last.

We can ask people not to post AI answers. If we threaten with bans, or talk about our chatbot filtering, some people will take it as a challenge.

Perhaps a longer term solution might be to work with AI. Suppose all dialogues had an option to generate an AI answer using the current popular models. You could see a reply to the question or the whole dialogue, while knowing it had been generated by an AI.

2
  • Agreed on "All bans are temporary", and this is a an important point: it is downright weird (and worrying!) that the temporariness is explicitly mentioned in this particular instance but for nothing else. Commented Jan 5, 2023 at 8:48
  • I like your reply very much. If you don't mind my saying, using ChatGPT to detect ChatGPT is a contradiction.. You wrote "We should not invite AI replies <until> their quality equals our human contributors". What is good quality? There are answers from ChatGPT to questions on StackOverflow that are just as good as human generated answers. There are many wrong answers given by humans. I strongly believe that ChatGPT will outrank the highest ranked member of StackOverflow in the near future, maybe before the decade is over.
    – Catriel
    Commented Jun 5, 2023 at 16:15
14

TL;DR

Even from the point of view of AI researchers, Stack Overflow and other sites with mostly human generated content should ban or force labelling of AI generated content, as otherwise this will cause a circular reasoning catastrophic failure as the newly generated content past year 2022 cannot be fed to train newer AI models anymore since we can't know what was generated by humans or by older AI models.

Longer argument

I would like to provide an alternative perspective, not from the standpoint of Stack Overflow human users, but from Artificial Intelligence researchers.

It's highly likely that GPT-3 and hence ChatGPT was trained on all of Stack Overflow data. This worked because all the inputs at the time was human generated. (PS: Let's put aside the discussion whether it's ethical for AI researchers to use 3rd-party content to train AI models without asking the respective owners - I am here focusing on the fact that it already happened, that this cannot be undone, and the impact on our current and future situation).

Now, if answers from humans are mixed with answers generated by AI, we get a tampered dataset that will be unusable to train future LLM or other language models, because it will cause a hugely flawed circular reasoning loop, as we now feed an AI model data that an older AI model generated, without being able to determine what was generated by humans or by AI.

This means that if we can't ensure that most answers remain generated by humans, this will lead to a catastrophic failure of AI models, as it will simply become impossible to use newer data to make newer models: 2022 will become an "event horizon for AI" , with data generated prior to this year being still usable for training, but any data generated past being mostly unusable because of being tainted potentially in great proportions by AI generated content.

So this issue is not even just specific to Stack Overflow: all websites should either ban the use of AI generated content, or force such content to be labelled as AI generated. But even so, it will only work with compliant users. Since there is no 100% reliable way to detect textual AI generated content, and given we can always expect people to game the system especially when there are incentives to do so, this catastrophic failure seems all but inevitable.

11
  • 7
    Honestly, this sounds like entirely the problem of the researchers. I'm not sure why sites being spammed by these AIs would be sympathetic to the further creation of more of these models.
    – JAD
    Commented Jan 26, 2023 at 9:29
  • 1
    The training data could be filtered to remove AI-generated content (As that content is detectable). Doesn't mean SO should allow AI-generated content, but it's not really SO's responsibility to provide usable training data.
    – Cerbrus
    Commented Jan 26, 2023 at 12:24
  • 1
    @JAD Your opinion sounds very manichean. When not used for spamming, AI has wonderful applications that already changed our current world. In any case, hating a tool is fruitless, just like hating a knife. It can be used for good and bad. I only argue that even from the point of view of AI researchers and users, there is no medium to long-term benefit to accepting AI generated answers, only (catastrophically bad) downsides. So whether you look from the POV of SE users, or from AI users/researchers, banning is one of the few reasonable solutions.
    – gaborous
    Commented Jan 26, 2023 at 20:36
  • @Cerbrus There is no filter that is 100% effective, and it's very easy to workaround these filters with slightly modified models or post-processing, there are already loads of posts like that on social networks by AI users (not even researchers). And I never wrote that it's SO/SE responsibility to provide usable training data, I only provided an alternative viewpoint that also converges towards the same solution. It seems there is a radicalization of "us vs them" mindset, sad to see. Even though my opinion agrees with the ban, it's ditched just for taking the point of view of another party.
    – gaborous
    Commented Jan 26, 2023 at 20:42
  • 1
    i simply don't see how this addresses the policy at all. You aren't contesting the policy, you state you agree with it... but that's what the upvote button is for. Nothing else in the answer is relevant to the policy decision.
    – Kevin B
    Commented Jan 26, 2023 at 20:47
  • 1
    Re "it's very easy to workaround these filters with slightly modified models or post-processing": Yes, but fortunately the worst abusers also have the minimum effort attitude (in other words, they couldn't be bothered). Commented Jan 26, 2023 at 21:18
  • @gaborous The point of view of the other party is, frankly, irrelevant when discussing SO policy.
    – Cerbrus
    Commented Jan 27, 2023 at 8:28
  • 1
    @Cerbrus AI not another party but a technology, just like the (AI-based) recommendation system of SO/SE. Unless we consider SO to be a closed-loop system, which I find implausible since it is a publicly accessible website, I think it is extremely short-sighted to dismiss any non-SO-centric perspective. The perspective I offer is just more systematic, accounting for other factors and the wider technological landscape. I make no claim it should be the primary perspective, I explicitly wrote it is an alternative one, a "food for thought" if you prefer.
    – gaborous
    Commented Jan 28, 2023 at 13:14
  • Its interesting, how researchers label Machine-generated data while implicitly fetching data for training. ?
    – Rahul
    Commented Feb 7, 2023 at 22:48
  • 1
    This topic is commonly known as model collapse Commented Jun 19 at 6:59
  • @WernfriedDomscheit Thank you very much, I did not know there was a name for this concept (despite working in AI!), thank you very much for pointing this out!
    – gaborous
    Commented Jun 30 at 21:16
13

My take on AI-generated answers is this:

Using ChatGPT and similar services is very easy. It's comparable to Google Translate. If you want an AI-generated answer, then you simply use one of those services.

Compare to the language sites. If you ask for a translation, then what you're looking for is something else than what Google Translate produces. If you wanted something from there, then you would use that service instead of asking on a forum.

I think it's good to assume that a user who is asking a question on a forum has tried those easy-to-use services and found them unsatisfying. Note that I'm not saying that it is likely that they have done it. Just that it is good to assume it, in the sense that you should answer a question as if ChatGPT or Google Translate was not enough for the asker.

13

In one line: I want a pure "human" platform, and for this reason I am on Stack Overflow.

In truth, I have learnt coding from SO, and I believe that every line of coding on SO is tested by humans, on a real project with their own hands. (This is mostly true for accepted answers.)

On the other hand, I have used ChatGPT for only for three days and came back. Please don’t ask me about the experience. I do not want to remember that I could lost my coding knowledge. uffffff!

3
  • 15
    "I believe that every line of coding on SO is tested by humans, on a real project with their own hands" - hahahaha.
    – CodeCaster
    Commented Aug 24, 2023 at 15:30
  • @CodeCaster, so AI tested your question and code on his own project? Ha Ha Ha! All are fake. I know my opinions cross worrd and thinkings, but must see that the word I have used is "believe". Try to know the meaning of this word. " will be or has to be or should be" hide behind the word silently. The result is "could be" Commented Aug 25, 2023 at 12:05
  • 3
    At least your motivation is sound "I want a pure human platform". That's a valid preference that doesn't even need to be substantiated with reasonable arguments. However, I don't have that desire, I want a platform that will still be relevant in 10 years and for that to happen the attitude towards AI needs to change drastically.
    – Xartec
    Commented Sep 5, 2023 at 13:42
12

The more I use ChatGPT, the more I agree with this policy. I asked it "Can you explain the difference classes of amateur radio licenses?", and it said this:

There are three classes of amateur radio licenses in the United States: Technician, General, and Extra. Each class of license has its own set of privileges and requirements.

The Technician class license is the entry-level license for amateur radio operators. It allows the licensee to operate on all amateur radio frequencies above 30 megahertz, including the very popular 2-meter and 70-centimeter bands. Technician licensees are not allowed to use the HF (high-frequency) bands, which are the bands used for long-distance communication.

The General class license is the intermediate level license. It allows the licensee to operate on all amateur radio frequencies, including the HF bands. General class licensees are also allowed to use higher-power transmitters and to operate certain types of equipment that are not available to Technician licensees.

The Extra class license is the highest level of amateur radio license. It allows the licensee to operate on all amateur radio frequencies, including the HF bands, and to use the highest power levels allowed by law. Extra class licensees are also allowed to use a wider range of equipment and to operate in certain types of specialized modes, such as digital modes and satellite communications.

To obtain an amateur radio license, an individual must pass a written examination administered by a team of volunteer examiners. The examination covers the rules and regulations governing the operation of amateur radio stations, as well as basic concepts in electronics and radio theory. The level of difficulty of the examination increases with each class of license, with the Extra class license being the most difficult to obtain.

This seemingly very well-written answer is wrong in several ways.

  1. Technician licensees are allowed to use certain small parts of some HF bands, with restrictions.
  2. General licensees can use large portions of all bands, but there are Extra-only frequencies within many of the bands.
  3. On VHF and higher bands, all licensees have exactly the same privileges. General licensees being able to use higher power than Technicians is true on HF bands, but it is not universally true.
  4. Extra licensees have the same power limits General licensees do.
  5. Virtually any transceiver that an Extra can use can also be used by a General licensee.
  6. Even Technicians can use digital modes and satellites. All licensees can use all modes; they're just limited to certain frequencies.

If/when a better AI for answering programming questions is developed, I think it would be helpful to have an authorized way to post AI answers such as my suggestion from a previous answer, but having used ChatGPT more, I've realized that it isn't that AI (yet).

15
  • 2
    @Rubén I'd previously posted an answer arguing against the ban and suggesting a way to allow AI answers, but having used ChatGPT more I decided it's not really a good idea at this time. I don't want to delete my old answer because I still think it will be a good way to handle AI if/when it becomes good enough to be useful for answering SO questions.
    – Someone
    Commented Dec 11, 2022 at 17:38
  • 16
    The accuracy of describing radio licenses doesn't seem hugely relevant for Stack Overflow's topics. Commented Dec 11, 2022 at 17:38
  • 44
    @MisterMiyagi but it's still a good example of how ChatGPT can sound very knowledgeable and confident, but be totally wrong.
    – Someone
    Commented Dec 11, 2022 at 17:39
  • @Someone That's true. It wouldn't have triggered any of my usual red flags. Interestingly enough, it also doesn't trigger any of the things I look out for in ChatGPT answers. Commented Dec 11, 2022 at 17:40
  • 10
    Please bear in mind that this "question" is not a "call for whatever". It might be OK to provide feedback about how the situation was handled previous to the launch of the help article that bans the use of chatGPT, but I not see any reason to polute this question with "ideas" or "opinions"
    – Wicket
    Commented Dec 11, 2022 at 17:43
  • 1
    @Rubén so all of the other answers explaining why people support or oppose the policy are off topic too? I saw that there are quite a few of them and thought it was okay to post another.
    – Someone
    Commented Dec 11, 2022 at 17:44
  • Each "answer" should be directly related to the question, if you want to criticize other answers that usually should be done posting comments direclty to the corresponding answer. If a comment don't provide enough room it might be fine to post another answer but it should include a link to the criticized post, but only if that is directly related to the question... answers should not divert from the question topic.
    – Wicket
    Commented Dec 11, 2022 at 17:48
  • 2
    @Rubén I'm not really criticizing any answer. I'm just giving another piece of information supporting the policy.
    – Someone
    Commented Dec 11, 2022 at 17:49
  • This is not the place for that and regading the temporary ban set by the mods that is late.
    – Wicket
    Commented Dec 11, 2022 at 17:50
  • @Rubén since it's temporary, that means it could end. I would suggest that it not be ended unless/until ChatGPT improves significantly.
    – Someone
    Commented Dec 11, 2022 at 17:53
  • It looks that you missed New help center article and banner on the site about GPT-Generated content
    – Wicket
    Commented Dec 11, 2022 at 17:54
  • The answers by ChatGPT are most of the times very general. Maybe good if you don't know anything about a topic but not good enough to learn much more about a specific problem, not to speak of solving a problem. It's definitely not a problem solver although kind of a search engine giving you some general insight into topics. That's what the current status of ChatGPT seems to be to me. Commented Dec 11, 2022 at 22:33
  • This is an illustration of the fallacy of how the answers are farmed. The Extra class manual contains a section on satellites not found in the other manuals, and it shows up as a topic covered deeply only on that license exam, but it has nothing to do with what's allowed by that license at the exclusion of others. Similarly, a question about Oracle might wind up drifting off into ISO SQL or vice versa. -- K7COI
    – durette
    Commented Dec 13, 2022 at 18:42
  • @MisterMiyagi, I disagree. This is a great illustration of the fallacy of farming answers by the false appearance of textual relevance.
    – durette
    Commented Dec 13, 2022 at 18:44
  • as long as it is an answer that can be used in certain circumstances and no one is around to dispute it, it is a useful answer (conditionally)
    – prusswan
    Commented Jun 17, 2023 at 10:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .