4140

Moderator Note: This post has been locked to prevent comments because people have been using them for protracted debate and discussion (we've deleted over 300 comments on this post alone, not even including its answers).

The comment lock is not meant to suppress discussion or prevent users from expressing their opinions. You are (as always) encouraged to vote on this post to express your agreement/disagreement. If you want to discuss this policy further, or suggest other related changes, please Ask a New Question and use the tag.

This question remains because that is still the best (and only) tool we have to announce this policy site-wide.

All use of generative AI (e.g., ChatGPT1 and other LLMs) is banned when posting content on Stack Overflow.

This includes "asking" the question to an AI generator then copy-pasting its output as well as using an AI generator to "reword" your answers.

Please see the Help Center article: Why posting GPT and ChatGPT generated answers is not currently acceptable

This is a temporary policy intended to slow down the influx of answers and other content created with ChatGPT and other generative AI technologies, typically using Large Language Models (LLM). What the final policy will be regarding the use of these and other similar tools is something that will need to be discussed with Stack Overflow staff and, quite likely, here on Meta Stack Overflow.

Overall, because the average rate of getting correct answers from ChatGPT and other generative AI technologies is too low, the posting of answers created by ChatGPT and other generative AI technologies is substantially harmful to the site and to users who are asking questions and looking for correct answers.

The primary problem is that while the answers which ChatGPT and other generative AI technologies produce have a high rate of being incorrect, they typically look like the answers might be good and the answers are very easy to produce. There are also many people trying out ChatGPT and other generative AI technologies to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with significant subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.

As such, we need to reduce the volume of these posts and we need to be able to deal with the ones which are posted quickly, which means dealing with users, rather than individual posts.

So, for now, the use of ChatGPT or other generative AI technologies to create posts or other content here on Stack Overflow is not permitted. If a user is believed to have used ChatGPT or other generative AI technologies after the posting of this temporary policy, sanctions will be imposed to prevent them from continuing to post such content, even if the posts would otherwise be acceptable.

NOTE: While the above text focuses on answers, because that's where we're experiencing the largest volume of such content, the ban applies to all content on Stack Overflow, except each user's profile content (e.g., your "About me" text).


1. ChatGPT is an Artificial Intelligence based chat bot by OpenAI, which was announced on 2022-11-30. Use of ChatGPT is currently available to the public for free.

1
  • 1
    Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, or in Stack Overflow Chat. Comments continuing discussion may be removed.
    – Samuel Liew Mod
    Feb 26 at 7:28

58 Answers 58

1
2
6

I think unmodified answers from ChatGPT should be banned, but if you use ChatGPT to generate an answer and then independently verify it and correct it to the best of your knowledge as needed, that should be allowed. It can be a useful tool, but simply taking answers from it as-is is often unhelpful.

18
  • 46
    I don't think this needs to be explicitly exempted. It usually takes more time and effort to verify a machine-generated answer than to write one yourself, with less garbage and verbosity included.
    – iBug
    Dec 5, 2022 at 12:14
  • yeah i guess you're right, it probably has more value for asking about specific details than answering the entire question.
    – alexia
    Dec 5, 2022 at 12:23
  • 2
    "unmodified answers from chatgpt should be banned" But how does one detect if the output from chatgpt was taken unmodified or not? Dec 5, 2022 at 12:55
  • 2
    you can't necessarily tell for sure (well, you can try asking it the same question and see if you get a similar answer), but if it's an obviously low-effort/incorrect answer then it should be deleted. if there are no issues with the answer then it shouldn't really matter if chatgpt was involved in it.
    – alexia
    Dec 5, 2022 at 14:05
  • It is a useful tool to help yourself, not others. I see chatgpt as an alternative to posting a question on Stack Overflow.
    – Gimby
    Dec 5, 2022 at 15:00
  • 22
    If users are taking the time to go through an auto-generated answer and verify its correctness before posting, credit its sources (even if this is actually possible for an auto-generated answer) and avoid plagiarism, etc. – and posting them at a reasonable rate (rather than flooding the site with bad answers), and improving them based on feedback in comments (rather than just dumping them on the site and then abandoning them) – then I would imagine it's harder to tell that they're even auto-generated. But at that point, there's little differentiating it from an answer fully written by a human.
    – V2Blast StaffMod
    Dec 5, 2022 at 15:47
  • 20
    Relevant XKCD: xkcd.com/810
    – V2Blast StaffMod
    Dec 5, 2022 at 15:51
  • 7
    @iBug I use ChatGPT every day for my work as a software developer and I disagree. It's a much faster version of StackOverflow basically. Dec 12, 2022 at 22:21
  • 2
    @NikS As others have pointed out multiple times: It's much faster at producing plausible answers that poses a greater problem to Stack Overflow than it seems.
    – iBug
    Dec 13, 2022 at 7:27
  • 2
    if you can verify it, then you should just write the answer yourself. "correct it to the best of your knowledge as needed" and what if you don't have enough knowledge, you will let ChatGPT's mistakes slip in? this is a bad idea
    – symbiont
    Dec 19, 2022 at 0:28
  • 3
    @V2Blast More relevant than that: using ChatGPT could prove self-referential (which is not a good thing at all)
    – Machavity Mod
    Dec 28, 2022 at 13:44
  • 2
    I love how existing scaffolding tools in Visual Studio/code aren't under fire, but using GPT is. This is the whole guns kill people instead of people (with guns) kill people argument. There's a lot of immaturity in this entire post. Furthermore, the arguments being made here act like GPT is spitting out 1000 line methods that no one understands. If you copy paste an answer from anywhere on SO without vetting it that is the source of the problem, not whether it came from GPT, a VS extension or MSDN forums. Jan 1 at 6:58
  • 1
    @iBug - "... takes more time and effort to verify a machine-generated answer than to write one yourself ..." is probably true if your first language is English. However, for non first language English speakers, writing your own answer could be far more time-consuming than verifying and tweaking an AI-generated one. Jan 17 at 4:38
  • I agree with this answer and the comment by @V2Blast. If someone has used ChatGPT to get an answer, taken the time to verify its correctness — the essential human curation step — and then posts it on SO, it could be useful to SO users. I would propose a citation along the lines of "Solution by ChatGPT, verified by me". Ultimately the important thing is answer quality, not where the answer came from. Feb 2 at 11:38
  • 1
    Nothing we can do here would inform on how correct it is in a way that would be valuable to it's improvement. @Rahul
    – Kevin B
    Feb 7 at 22:49
6

My take on AI-generated answers is this:

Using ChatGPT and similar services is very easy. It's comparable to Google Translate. If you want an AI-generated answer, then you simply use one of those services.

Compare to the language sites. If you ask for a translation, then what you're looking for is something else than what Google Translate produces. If you wanted something from there, then you would use that service instead of asking on a forum.

I think it's good to assume that a user who is asking a question on a forum has tried those easy-to-use services and found them unsatisfying. Note that I'm not saying that it is likely that they have done it. Just that it is good to assume it, in the sense that you should answer a question as if ChatGPT or Google Translate was not enough for the asker.

5

What if we fend off AI-generated content with AI-assisted moderation?

The video sharing service I use the most often in China - BiliBili, has an AI-based moderation bot called Avalon, and it monitors for harmful content, makes automatic decisions when harm score is high, and defers to human moderators when it's lacking confidence. It's constantly improving itself based on the evolution of contents and input from human moderators. (Of course, being in China, we also use it for censorship in addition to day-to-day moderation).

This is just my personal opinion, but I think investing in an AI-assisted moderation system is worth it in the long term.

6
  • 8
    The current situation with moderation bots on Quora (with unspecified IQ) is a complete disaster. Though detection of text as images could be a useful addition here. Dec 8, 2022 at 22:57
  • 3
    There was the unfriendly comments detector robot where they used AI-assisted moderation already with moderators being the final decision makers but an AI model was used for automated flagging. The same could probably be done for a "really bad answer" category. However, it's kind of sad to see that this basically results in a technology battle between spammers and cleaners, instead of humans learning how to improve their skills. Dec 9, 2022 at 9:35
  • 2
    This isn't up to SO moderation but rather up to the company behind SO. Good luck convincing them. Dec 10, 2022 at 21:15
  • I like the idea of AI-assisted moderation. Humans still making the final decisions, but automation providing tools to make their job easier/quicker. Still, banning people from posting ChatGPT answers as their own answers seems like an obvious no-brainer to me...
    – mdmay74
    Dec 13, 2022 at 4:01
  • 1
    I don't know. I've been flagged on MSN.com numerous times for stating various things such as: political views; religious views; and, I believe sometimes just simply stating facts. The problem is the algorithm on MSN.com has a notable liberal bias and Microsoft doesn't seem to have any intention of correcting that. This actually goes against Microsoft's core principles of: inclusiveness, fairness, transparency and in some cases even safety...
    – Shawn Eary
    Feb 28 at 20:40
  • This is a contradiction. You want to ban AI with the very same technology that you are banning? This sounds like the old computer science problem of a program that debugs itself - not possible.
    – Catriel
    Jun 5 at 16:02
3

I agree with the ban. Stack Overflow is for questions that the author researched and tried to find an answer for and is still stumped. Questions that require a decent level of expertise to answer appropriately.

If an answerer can paste the question into ChatGPT and get an answer, so can the asker. If a question could have been answered correctly by ChatGPT, then probably the question was poorly researched anyway.

But high quality questions deserve high quality answers. A good asker would have already put their question through ChatGPT and not have gotten a satisfactory answer, and are asking on Stack Overflow for a human expert written answer.

-3

I've given a lot of time before writing this because I think at this point we all have been able to digest and rationalize what is happening around this phenomena.

First of all, I fully agree with the temporary policy, and I am in favor that it becomes a "permanent temporary" one.

The main reason I'm on that opinion is not the fact of the tool being available, but the way so many people were using it: as a copy-pastable BS generator for social networking engagement (in our case, SO reputation). This alone spawns several reasons for why it should be banned, but I don't have to point them out anymore, as it should be common knowledge right now.

I am not in favour of computer-aided code writing as a tool to show examples of how a particular question can be solved. That is a job for GitHub Copilot or other places, not for this website. People can go straight to GPT-3's playground and ask the program to write code for them as an additional reference, but Stack Overflow should remain as human-aided different points of view in a given problem in the form of a question.


On the other hand,

(here comes the downvoting...) xD

I would like to add to the debate "legitimate" use of the technology. Consider some people who are savvy on the topic of the community (in our case, programming), but they are not proficient in the use of English or in a general sense, for whatever reason, haven't developed very good communication abilities.

Would you consider the potential contributions of such a person less valuable than another one who can express themselves "better" (in the sense of getting their point across close to the most optimal possible way)?

Now take the conclusion you've reached by reading the last two paragraphs and let's see what professionals of the marketing area are doing. They're using ChatGPT as a tool for computer-aided writing. Sure, the lazy ones are just copying and pasting whatever the program spits out, but in the hands of a capable professional, ChatGPT is much more powerful and is faster than hours of googling, which is what they were doing until now.

Also, that is what we programmers were doing until now. Of course, our main sources of inspiration are probably Stack Overflow, but there's a lot of forums around. Just like Server Fault users rely mostly on Server Fault, but the ArchWiki is still a reliable curated source of information, among other wikis and forums. But we all became dependent on search engines like Google Search.

Come to think about it, it came as no particular surprise to me that Microsoft decided to integrate ChatGPT into the Bing search engine.

There is a specific ELI5 prompt on the ChatGPT API examples page that try to show how to use the power of synthetic dialectics to further clarify a subject. The job of a communicator is to find the best words in the best order to exchange an idea to a specific target group. And in that field, ChatGPT is not a terrific tool, but it can aid many people as it is right now.


My point is that computer-assisted writing is a beneficial thing. People can use it to write better questions primarily, but also improve their answers' wording. This is specially useful for the handful of people I've described earlier: the ones proficient in programming, but not as much in writing English, or in communicating in general.

The distinction between computer-assisted writing and copy pasting from a BS generator should be obvious.

I could send this whole answer to proof reading humans or ChatGPT and bet excellent feedback from the humans, but reasonable good feedback from the program. It would look less like I'm the author of it, but in both cases, you, the reader, would struggle less to understand what I want to tell you. DISCLAIMER: I did neither. This is 100% my first take on writing the answer, without revision.

Finally, my opinion is that people should somehow feel that it's ok to use computer programs to aid their writing. I'm not sure how the anti GPT policy could be further improved to include this, or if is it even necessary.

I will repeat this paragraph from the beginning, now that the reader has a new perspective:

I am not in favour of computer-aided code writing as a tool to show examples of how a particular question can be solved. That is a job for GitHub Copilot or other places, not for this website. People can go straight to GPT-3's playground and ask the program to write code for them as an additional reference, but Stack Overflow should remain as human-aided different points of view in a given problem in the form of a question.

12
  • 14
    Why use ChatGPT for computer-aided writing? Why not use translators and/or Grammarly and/or other tools that fill this niche? Honestly, the most likely outcomes I can see from using ChatGPT to "aide" writing is a) copy/paste. So pretty much exactly the same as now. But the user can claim "me not good speaker, used ChatGPT to aide". b) ChatGPT fumbles the writing and does not represent the idea the user wanted to put across correctly. There is also a chance that it's OK, however, what are the chances that a user needs tool assist will recognise bad output?
    – VLAZ
    Feb 20 at 18:44
  • @VLAZ looks like ChatGPT agrees with you: platform.openai.com/playground/p/… Feb 20 at 19:12
  • 12
    Code completion isn't a substitute for professional engineering. Using code completion tools doesn't make you an engineer, let alone a good engineer. Code completion tools optimize an expert's workflow, but it's not the workflow unto itself.
    – Makoto
    Feb 20 at 19:21
  • 3
    @Makoto I stated two times the exact same paragraph in my original unedited text arguing against code completion on SO Feb 20 at 19:44
  • 3
    Then you've used a whole lot of words to simply say that you agree with this. There's been plenty of prose on this matter and there's really not a lot of value to keep adding to it unless you're offering a completely unique take on it.
    – Makoto
    Feb 20 at 19:49
  • 6
    I'm sympathetic to the concept of using LLM-based tools (not necessarily ChatGPT) to clean up one's writing. But there's a danger that the tool will say something different to what the writer intends, especially if their English skills aren't strong. That could lead to greater misunderstanding. If I see clear confident English I tend to assume the author knows what they're talking about. If the writing has a few flaws I can tell the author is struggling to express themself, so I might not have the correct interpretation, and I may need to post a comment for clarification.
    – PM 2Ring
    Feb 20 at 19:50
  • 5
    @PM2Ring fun fact - you can interact with ChatGPT in languages other than English. For example, you can instruct it to translate something in English. However, the training it has had other than English is a bit hit and miss. I have no confidence whatsoever it will be able to correctly pick up the meaning in another language. I've seen it very badly misinterpreting rather simple prompts when not in English.
    – VLAZ
    Feb 20 at 19:58
  • 8
    I’m really not a fan for using ChatGPT to translate or significantly brush up technical writing. Since the tool basically makes up stuff, it is absolutely vital that the posting user can verify all of its output; if that output is much more sophisticated or even entirely unintelligible to the posting user, they can’t do that. Feb 20 at 20:29
  • @Makoto agree with what? Your wording makes me think that you think I'm either agreeing or disagreeing with the anti ChatGPT policy, which is not the point of my answer whatsoever. Feb 20 at 23:51
  • @PM2Ring that is a fair point, but in this circumstance people are already struggling with tools like Google Translate and getting worse results. If they include the customary disclaimer that they're not proficient in English and they're being aided by online tools, they'll be fine as always Feb 20 at 23:53
  • @VLAZ I happen to use ChatGPT exclusively on Brazilian Portuguese and I though that was the same output I'd get from English. Feb 20 at 23:55
  • 9
    Even during translation or editing, ChatGPT is prone to introducing falsehoods or misstating facts to the level that they become incorrect. Often this creates the appearance of lying from the author, when the intent was simply to address any grammar or spelling issues. The tool really causes more harm than good, and that is why it is banned.
    – Travis J
    Feb 20 at 23:55
-15

I feel the ban is good and should be permanent because we have to remember that ChatGPT is really just a more interactive search engine and its results, whilst clever, are still the result of scraping existing web pages - just like any other search engine.

By allowing ChatGPT answers on Stack Overflow (and other sites), we'd be creating an echo chamber whereby the answers it generates are simply based on its previous answers, which may not have been right in the first place.

4
  • 16
    Chat GPT is an AI Chatbot that got trained a while ago with large amounts of data from websites/books etc. It isn't a search engine and it doesn't scrape existing websites. It would probably be more accurate if it just copied code from a website. Mar 3 at 12:31
  • 5
    @PhobosFerro You comment contradicts itself. If it was trained with data from websites then it's still dependent on the content of websites and will have to be updated to keep relevant. How it actually gets the new data may not be scraping but it will still be reading its own answers. Mar 3 at 12:52
  • 5
    Do you have first-hand experience using it as an interactive search engine (not a rhetorical question)? Mar 3 at 22:12
  • 1
    @PeterMortensen the branded "new Bing" does that. What it does is: user makes a search query, Microsoft does a regular Bing search, send parsed results as part o a prompt to OpenAI's GPT4, and finally it answers as an interactive chat search. Mar 13 at 14:38
-15

It's just a suggestion. People will keep posting ChatGPT answers anyway. The problem is now that a user can't really differentiate them.

A solution I would suggest which is already mentioned here, but different, is not only to proactively post an answer from ChatGPT (or other models), but intentionally ranks them lower and banner them clearly that this answer is not yet checked by a human. Then a person can confirm, edits, or reject the answer, which will change the rating of the answer itself (this ChatGPT answer have been reviewed by X and rated as correct).

If an open source model is used, then Stack Overflow have even more data to train their own model which will beat all other AI models...

16
  • 14
    I think you're massively underestimating the effort that goes into training this kind of model... Also, why does the generated answer need te be on SO? Why can't a user that wants an AI-generated answer just go to said AI?
    – Cerbrus
    Sep 19 at 13:52
  • 4
    If the AI reliably produces correct answers - why build a library of them at all? If the AI does not reliably produce correct answers - how many and which answers should be posted? Critically, how should volunteers deal with the required massive content volume when we already have too few people for the fewer human generated answers? Sep 19 at 19:52
  • Re "not yet checked by a human": This doesn't (and didn't) even happen with human-generated answers to any significant extend. Why would anyone spend time checking a code dump answer (no explanation whatsoever)? Yes, that is a rhetorical question. A code dump answer may be completely bogus or brilliant. It is difficult to judge unless significant time is spend to actual run and test the code. Sep 20 at 21:03
  • I must admit I do not actually dislike this idea as much as most other suggestions at face-value. I mean it's not what I would like, but if a compromise had to be struck, in the event where we cannot absolutely reliably ban ai generated answers, I would choose a somewhat "site-approved" or even better, "site-trained" ai answer generator. Sep 21 at 0:34
  • Like let's see how good exactly can we train a model exclusively on SO content to answer duplicates. Sep 21 at 0:34
  • @Cerbrus, the second question in your comment isn’t a relevant as it may seem. You ask why something can’, while of course an asker can and is not prevented from doing so regardless. Just as the asker could rtfm or get an answer elsewhere. It’s however about answerers using AI, who do that regardless of what the asker could do.
    – Xartec
    Sep 27 at 12:35
  • 2
    @Xartec Rowanto is arguing that SO should implement GPT to automate answers. That's what my "Why" is asking about. Why do we need to invest time in implementing generated crappy answers, when users can get that nonsense at the source, instead?
    – Cerbrus
    Sep 27 at 13:13
  • @Xartec "It’s however about answerers using AI" To the contrary. This answer is about SO itself using AI, which then - somehow - removes the issue/occurrence of answerers using AI and - somehow - offers a benefit to askers. Sep 27 at 13:24
  • @MisterMiyagi that’s what his proposed solution is about. His answer is a solution adressing the problem as layed out in the second sentence.
    – Xartec
    Sep 27 at 20:57
  • @Cerberus “ Why do we need to invest time in implementing generated crappy answers, when users can get that nonsense at the source, instead?” This, again, relies solely on the false idea AI generated answers are crappy and nonsense. Which is is like building a house on quicksand, as chat gpt would say. The quality of the answers can be heavily improved by prompt engineering which would discourage answerers from using the same ai to produce sub par results. It would be a sensible way to introduce AI, an inevitable situation.
    – Xartec
    Sep 27 at 21:03
  • 2
    @Xartec they've already tried that, it failed spectacularly. Many other sites are similarly trying this, such as quora, and are having similarly questionable results. Why must SO include AI generated answers? What value would that provide? Surely the best person to have a conversation with a chatbot and weed out poor results is the person with the problem being solved, not some rep hunter on SO or SO itself.
    – Kevin B
    Sep 27 at 21:06
  • @KevinB the why has been established, the problem at hand and is repeated in my previous comment. In an ideal world you’d be correct, that would surely be the best person. In reality, people post answers generated by AI regardless of whether SO ‘must’ and no the person with the problem is surely not automatically the most capable of asking the question. That assumes everyone masters prompt engineering which just isn’t true. We can disagree on opinions, not facts. An imo, a more controlled AI answer would likely reduce the urge to create a poor Ai generated answer.
    – Xartec
    Sep 27 at 21:16
  • 1
    @Xartec Yes, and we've been routinely deleting said answers for nearly a year now. Yes, we haven't gotten all of them, but we've gotten enough, and enough accounts are actioned against to keep the problem at bay. Your assertion that we cannot discern whether or not content was written by AI is quite provably false.
    – Kevin B
    Sep 27 at 21:19
  • 2
    There is no magic prompt engineering masterpiece that will suddenly make any current or near release version of gpt capable of producing answers to new questions. At best it can summarize an existing answer to an already answered question, however even that becomes more likely to hallucinate than to provide an accurate answer the more you allow it to modify the content it is summarizing to fit the person looking for an answer. The user would be better off just getting the existing answer.
    – Kevin B
    Sep 27 at 21:34
-16

I, personally, think that there's not much we, as a community, can do to stop AI-generated content from being used on the site in the long term. Furthermore, I think that the long-term is what we should be focusing on.

The Future of AI in a Different Timeline

The current banhammer stopgap may work for now, but it's a simple solution requiring manpower, and it won't be viable forever. Who here remembers when Dragon Naturally Speaking came out on CD in the 90s? I was 8 or 9 when my grandfather got us a copy. We had an old beige microphone, and my friends and family were absolutely amazed at the speech recognition capability. They could dictate documents directly into Microsoft Works, and I could.... Well... Do what 9-year-olds do best: See how many curse words it could recognize... Even if you had to enunciate them.

Fast forward a couple of years and the same software could recognize multiple individuals in a single conversation and create a transcript including respective speakers. Fast forward a few more years and Microsoft has added "Train your computer to better understand you"1, so enunciation is no longer an obstacle.

That 10-year timespan I just covered would be easy to react to as a community. The problem is AI doesn't take 10 years, it takes 10 hours2. It's easy to spot right now (question rephrased to statement followed by bullet points for your vanilla ChatGPT) but what happens when we get the option to "Train your computer to better type like you"? The Insider Build of Windows 11 (Dev Channel) currently has a Copilot preview that can access your active tab3. I'm certain Microsoft Office access will be next, so It's only a few steps away at most in my opinion.

My Opinion Moving Forward

My thought is this... The banhammer on AI-generated content that isn't cited should continue. However, we should move towards a system that accepts it, so long as it's clearly marked/cited as machine generated[4] and there should rarely, if ever, be a top answer marked that is wholly AI-Generated. As we move towards that system, the 30-day bans need to turn into 90-day bans. If we incorporate automated detection at some point, it should be human-reviewed, and 90 days should turn into 180 days. I do not see this as too harsh of a punishment so long as users have been explicitly warned.

I believe that if history (As a whole) has shown us anything, it's that we cannot simply ignore a problem, sweep it under the run, and expect it to go away. Bandaids are temporary, AI is not. Humans have suppressed so many different things over the years... Catholics persecuted Presbyterians as heretics in the Middle Ages, Americans in the land of the free bought and sold black slaves to tend their households and farmlands, women were expected to quietly obey their husbands without a voice to vote until the 1900s, and Jews... Well, Jews have gotten the shaft since the dawn of time.

I'm not saying that AI deserves citizenship or human rights[5]. But as widespread as AI will inevitably become, and as integrated into our lives as it will we need to treat it that way.

TLDR:

We should standardize a boilerplate for AI content and require anyone incorporating it into their content to use it. We should continue the 30-day ban stopgap until a point where it's no longer needed. We should work towards a way of automating the recognition of AI-generated content that isn't cited, always require human review of this automation's output to reduce false positives and implement very steep consequences for those who violate the rules.

Again, all of this is just my opinion. I've been a lurker for 7 years, but as you can see from my reputation, I haven't been active until this past week (I've started to enjoy teaching and tutoring in IT). So, in the grand scheme of things, my 2 cents is literally just that, 2 cents. But I hope that at least a few of you will share somewhat similar opinions. Thanks for your time :)

Clarification on Long-term viability and manpower

I do not necessarily think there will be a need to patrol AI-generated content forever. At least, I hope there isn't... However, dumping everything into implementing 30-day bans as stopgaps can't be the answer. Because if we're relying on humans to recognize AI-generated content, we will fail no matter what. I'm not necessarily saying we should absolutely do anything specific. I'm only saying that we need to look much further ahead than much of the discussion going on here.

I do not know what technology drives Stack Exchange on the backend. But the technology has to move forward to account for AI Content. Whether you call this a forum, a wiki, an image board, a social media network, or anything else... The technology underneath needs to move forward. To stress this point again if we're relying on humans to recognize AI-generated content, we're going to fail no matter what

What's Stopping Users from Simply not Adding the Boilerplate?

As Stack Overflow is built on a foundation of user trust, there's nothing that can be done to prevent this except moderating content. Though, in my opinion, adding the Boilerplate at least lets the userbase know it's expected. I found out via a comment warning when skimming through questions.

But AI Content Can't Reliably be Detected?

I'm not asking for someone to look into this or share information related to site analytics or any other privileged information. However, I would postulate that greater than 65% of users who have been banned for posting AI Generated Content are newer users trying to boost their reputation. Furthermore, I would estimate that greater than 80% of those are using a vanilla ChatGPT based AI, of which greater than 95% are "non-precise" style (This high percentage is based on the difficulty of getting a reliable output to open ended questions).

I want to point out an assumption in my argument, I'm looking at this from the point of view that if you know enough about AI to use anything outside of what's available on the mainstream channels, you likely have the experience to answer the questions without using AI content or to examine the content for accuracy before posting it. As mentioned in the previous section, I am basing this assumption off the foundation of trust.

If those numbers are remotely close, then the additional review queue should not require much additional overhead to patrol a large portion of violations. I can quite easily imagine a natural language string analysis algorithm combined with a user event timing algorithm that could pick out a relatively high percentage of violations. I'm sure there are many users way more talented than myself who could imagine the same in a far more optimized and efficient way.

An Alternative

If patrolling content and review queues are too far-fetched, then maybe instead of targeting users, we should target posts (I thought this was mentioned in one of the comment threads but couldn't find it on a second look). Automated boilerplate addition to user posts if they are flagged as AI Generated by an algorithm. A certain reputation level allows an individual to remove the boilerplate. A higher reputation level will automatically bypass the check on their posts.


If policing content isn't the answer, then decriminalization and regulation is.


[1] There was a step in between, where Microsoft let you correct its understanding with input rather than via prompt; See This MiPad Research Article

[2] Keep in mind that AI can't just pick up any task and do it quicker and better. Take for instance the following study in which it took an AI 924 hours to learn a game that would take humans less than 30 minutes:

  • Lake, Brenden & Ullman, Tomer & Tenenbaum, Joshua & Gershman, Samuel. (2016). Building Machines That Learn and Think Like People. CBMM Memo No. 046.

[3] Verified the knowledge is public before posting, See This Public Blog Post on Windows.com

[4] We should probably require citing the exact AI that generated it. I foresee that as an upcoming IP battle in the near future. Plus, as always, there are going to be biases on AI sources that could affect the community as a whole if we don't.

[5] Yet... But I can't convince myself to rule that out at some point in the future, the human brain is only so complex, and technology is surely, albeit slowly, getting there.

8
  • 6
    "We should standardize a boilerplate for AI content and require anyone incorporating it into their content to use it." and...if somebody doesn't? We'd still need the manpower that the start of your post claims is not viable forever. So...you want AI generated output to be embraced but we should still pour the same amount of effort into detecting "illegitimate" usage of AI generated content?
    – VLAZ
    Aug 27 at 12:11
  • 13
    As with many other proposals to integrate/accommodate GenAI answers, I’m still missing the "why". What’s the point building a repository of things we can just generate? If GenAI reaches a point at which it can reliably generate answers then SO is obsolete and we - the users - can do better things with our time than nanny some algorithm that far outscales us anyway. Aug 27 at 15:14
  • @VLAZ Edited for clarification of my stance. Aug 27 at 15:25
  • 6
    Nothing you suggest is feasible. AI content can't reliably be detected, and yet another review queue isn't a solution either. Also, what's to prevent users from just not adding the "This is AI-generated" boilerplate?
    – Cerbrus
    Aug 27 at 18:19
  • @Cerbrus Updated answer to address your concerns, including an alternative to policing content. Aug 28 at 0:13
  • Also, thank you to both of you who edited my answer. I am terrible with grammar and consistency in voice. I also didn't know that blockquotes were considered abusive when used for formatting. I always read through edits and try to learn from them, I really appreciate it. Aug 28 at 0:15
  • 1
    I don't think your edits answer the questions above :(
    – M--
    Aug 28 at 0:43
  • 2
    Your edits don't answer my concerns... At all...
    – Cerbrus
    Aug 28 at 7:50
-18

The more tricky question is: should paraphrases of GPT-whatsit-generated verbiage be banned?

2
  • 16
    Yes, just like with plagiarism, direct paraphrasing of AI generated content isn't permitted. We've detected and appropriately handled a substantial number of posts and users where there was considerable effort made to hide/obfuscate that the post content was AI generated. Is detection of such posts perfect? Of course, not. However, moderators are likely going to be less lenient for a user that has clearly demonstrated, by trying to hide/obfuscate, that they know they are doing something that's against the rules and at the same time caused moderators and users doing curation to spend more time.
    – Makyen Mod
    Mar 29 at 17:43
  • 4
    whatsit: "(chiefly UK) A thing (used in a vague way to refer to something whose name one cannot recall, or that one is embarrassed to say)" Mar 29 at 18:14
-25

I think Peter's answer has the right spirit, but the wrong implementation. We might need to get comfortable with this technology being around in the long term, and one way to stop the abuse it might generate is to build it into the system with the necessary precautions and abuse prevention mechanisms.

Make no mistake: the genie is out of the bottle. You can't put it back in. You can't wish it away. This is going to be a thing going forward, and it doesn't even have to be a problem.

Basically, you create some kind of system user that posts an AI-generated answer to ~every question. (Maybe ask the bot if this looks like the sort of question that belongs on Stack Overflow first, so you don't start automatically answering obviously-off-topic questions like "How is babby formed?", although that can be gamed and that's a concern.) You mark it as a bot. You surround the answer with the necessary warnings. Its answers are all community wiki, meaning the bot never gains any reputation and users are encouraged to edit the answer if it's only slightly wrong. The bot automatically deletes its own answers if they get a low enough score. Other people trying to run a ChatGPT Stack Overflow gold rush for Internet brownie points would find themselves unable to keep up with this system user, and would stop trying.

You run this for a while, determine if it's successful or not, and... if I was your CFO, I'd better hope it's not, because as other comments show, this might not be a very cheap model to run (even if you had a license from the creators to run it for $0). I can't imagine Stack Overflow staying sustainable as a commercial entity if it has to pay an AI tax on every single question on the website.

The good news is that, if this does become too expensive for Stack Overflow, it's gonna also be too expensive for random Joe's trying to gain cheap reputation this way — and that probably also means the end of the gold rush of people who just want to write "make number go up with AI" blog posts.

48
  • 11
    I'd like to point out that OpenAI probably pays big time right now for the server, GPU and memory and electricity it uses, not even considering the training costs (these typically range in the lower single-digit millions for complex language models). An OpenAI-paid-for ChatGPT answer is going to be really expensive if someone actually needs to pay for it. (We're talking spacefaring carbuilding lunatic billionaire-sized venture capital being burnt here.) Dec 5, 2022 at 11:37
  • 3
    Yes, I addressed that point towards the end. It's possible that, even if successful, this trial can't work for financial reasons. The good news is that, if this does become too expensive for Stack Overflow, it's gonna also be too expensive for random Joe's trying to gain cheap reputation this way. My understanding is that the problem, for now, is the gold rush of random Joe's trying to get a number to go up.
    – badp
    Dec 5, 2022 at 11:40
  • 5
    ChatGPT does not give a canonical answer to a certain question. If you ask the same question multiple times, you will get multiple different answers that sometimes even make conflicting statements. There seems to be a random element used when the AI generates its answer.
    – NineBerry
    Dec 5, 2022 at 11:55
  • 4
    So an automated bot dumps crap answers on questions... You're missing one glaring problem: ChatGPT doesn't return the same answer for the exact same question twice, so any user that probably also doesn't use the exact same input will absolutely get a different answer... This solution can't solve anything, other than a overflowing wallet.
    – Cerbrus
    Dec 5, 2022 at 11:55
  • 11
    "if this does become too expensive for Stack Overflow, it's gonna also be too expensive for random Joe's trying to gain cheap reputation this way" A thousand users with a thousand free / trial accounts can posts tens of thousands of generated answers... SE, a single entity, will have to fork over cash for every single answer they automatically generate. You can't compare corporate usage with individual usage.
    – Cerbrus
    Dec 5, 2022 at 12:01
  • 9
    This would defeat the purpose of Stack Overflow. It actually sounds like a completely different service. GH Copilot is doing something like that already, doesn't it?
    – Dharman Mod
    Dec 5, 2022 at 12:25
  • 9
    @badp No, I am saying that the purpose of Stack Overflow is to provide quality answers to common problems. These answers can then be found by humans searching for the same issues. A bot answering every question on Stack Overflow would go against the purpose of the site. We already have this issue with some users who add a code-only answer or repeat the same solution on multiple questions.
    – Dharman Mod
    Dec 5, 2022 at 12:32
  • 4
    @badp Because they looked for it already on Google. Asking a question on Stack Overflow is the last resort. It means that a question like that hasn't been asked by anyone else yet.
    – Dharman Mod
    Dec 5, 2022 at 12:39
  • 3
    @Dharman that's desired user behaviour, not actual user behaviour. If things were that easy, we'd never need to close questions as duplicate.
    – badp
    Dec 5, 2022 at 12:40
  • 21
    The problem this ban is meant to solve is that ChatGPT can produce answers in seconds which require minutes of multiple people's time to verify if they are worth having on the site or not, and that is a waste of time when a large proportion of such answers are not worth having on the site. If every question automatically received an answer like that, it would make the problem worse, not better, by requiring more people to spend more time verifying and voting on those answers.
    – kaya3
    Dec 5, 2022 at 13:09
  • 4
    There are many questions on Stack Overflow which the person asking cannot try out first hand and see if they are correct. Consider a question like "will a linked list be more efficient than an array in this case?" with an answer like "a linked list will be more efficient because most of the operations are at the start of the list". If the OP was able to try it out and see which was more efficient then they wouldn't have asked the question, and the answer doesn't tell them how to try it out. "What is the time complexity of this algorithm?" is another class of such questions, which ...
    – kaya3
    Dec 5, 2022 at 14:05
  • 5
    ... Stack Overflow already has a lot of users posting (non-AI-written) answers that "seem right", that the OP may be satisfied with, but are wrong in ways which cannot be demonstrated by "trying them out" (and this often makes it hard to convince the person who wrote the answer that they are wrong, too). Bad answers like that waste a lot of people's time, the site needs less of them, not more. As for "voting is integral to the site", of course it is, but the site is better off if experts spend their time writing good answers, instead of checking and downvoting rubbish answers.
    – kaya3
    Dec 5, 2022 at 14:08
  • 5
    That presupposes that there are rubbish answers which need to be downvoted in order to be removed from sight. But what you're proposing is to create those rubbish answers, automatically, en masse.
    – kaya3
    Dec 5, 2022 at 14:37
  • 4
    @badp If the answers weren't rubbish, this wouldn't have been a problem in the first place. Duh. The whole entire friggin issue here is users posting AI-generated content en masse, without checking the answers.
    – Cerbrus
    Dec 5, 2022 at 14:43
  • 5
    A super power people crave is being able to express themselves better. I think you're right in that this isn't going away, and if implemented as assistive instead of prescriptive, could result in people writing way better questions. I also see the potential to help people write answers too, but perhaps as a coach and not a co-author. I think it could do what guided "wizards" just don't do very well.
    – Tim Post
    Dec 5, 2022 at 18:41
-27

I honestly believe ChatGPT is a powerful tool, but in reality it doesn't give exactly what we want. All the answers posted are from real legends who put their hands on the code and tried the solution. I would say it would be disrespectful to mix AI answers with human answers because the AI is trained from human answers.

1
  • 16
    ChatGPT is insanely great and is a powerful tool, but it will also produce answers that are totally wrong and/or made up. It can not be trusted. Mar 30 at 13:33
-27

Instead of banning ChatGPT, Stack Overflow could consider adding a new button "Show AI Generated Answer(s)" to each question page. Such answers should be hidden by default, but if anybody is curious about what ChatGPT (or any other future AI tool of the day) has to say about it, they can check it out.

Also, such AI-generated answers could be compared to the "human"-generated answers with another AI tool to compute a similarity score, and if a "human"-generated answer is too similar to an AI-generated one, then such an answer should be banned (since the probability that such "human" answer has actually been produced using an AI tool would be high) and the user who posted it could be given some penalty.

So, this approach would kill two birds with one stone - people would be much less likely to post AI-generated answers since such answers would already be there automatically, and Stack Overflow would get an automated tool for detecting AI-generated answers and for punishing users who abuse the "no AI-generated answers" policy.

6
  • 13
    No? People aren't going to go through the rigor of ensuring that the answer they see or read is comprehensible or useful for their use case. We already have cases of people copying code from Stack Overflow and putting it into production. This doesn't make that better; it actually makes it orders of magnitude worse.
    – Makoto
    Feb 1 at 21:47
  • 11
    People already barely follow instructions like How to Ask and How to Answer. The problem we see with lots of AI-generated answers is that the people who post them do so blindly, without verifying that it answers the question or testing for bugs. I doubt asking these low-effort posters to put in more effort to self-identify (even though it's barely any effort to check a box) is going to have the effect you desire. Until such time as AI-generated answers are reliably correct, if people are interested in what the AI has to say, they can go talk to it at the AI's website. Feb 2 at 0:08
  • I don't think this is a bad idea. You cannot stop anyone from looking for AI-generated answers. The point is not to be vindictive of AI generate responses but to let others decide if those are accurate or even helpful. In my experience apart from answering basic questions, pre-2021, ChatGTP has not provided any relevance. This data can also be useful in future to train better models.
    – Rahul
    Feb 7 at 22:37
  • 7
    if people want AI generated answers, there's a place for that. (it isn't here)
    – Kevin B
    Feb 7 at 22:52
  • Not sure why this was downvoted but i can understand why when everyone are so angry and ban-happy already... This is the least-effort and also most realistic way of fixing the problem now and forever. It will always work too, because chatgpt will always produce grammatically correct answers(even if completely wrong) and a similarity algo should be feasible to implement(ie: not too complex). It is also the only answer where it saves SO users from cleaning up and saves SO from becoming more authoritarian(it already is too authoritarian for many users...)
    – n00p
    Feb 24 at 7:30
  • This is a great suggestion. I wrote an answer similar to yours but with the added usage of a tag to help StackOverflow filter out or correct AI generated answers. Unfortunately, the senior members of this forum do not encourage any type of interaction with known AI answers. I understand their reasoning. The big question is how will they be able to tell the difference otherwise? More AI detection? Is OK to use AI to "detect" AI generated answers, but it's not OK to submit AI generated answers? Seems like a paradox to me.
    – Catriel
    Jun 5 at 15:03
-30

Yes, it should be banned. To the question of "How do we identify those posts?", it should be considered that this problem is not new or unique to Stack Overflow. Plagiarism is a concern that spans broadly. An answer found in academia is to copy/paste answers back into ChatGPT and see if it responds to it as a continuation of a conversation; if it does, then flag it as AI-generated.

This is a potential technique, and one that scales. Multiple suggestions here imply that "you can tell it by looking at it", which isn't all that helpful because we can't expect people to reliably keep up with the potentially exponential flow of spam answers.

8
  • 21
    No, that's not at all accurate. ChatGPT just pretends. It can't recognize its own output.
    – Cerbrus
    Jan 18 at 1:07
  • 14
    The conversation chains in ChatGPT are separate. Every time you start a new chat, it keeps that session for that session only. I've seen nothing to imply that it persists any information cross-sessions. In fact, what I do for amusement is ask ChatGPT where my hometown is located and it gives me a new, amusing, and mostly wrong information every time. You can try to correct it but if I start a new session it goes back to the nonsensical information again.
    – VLAZ
    Jan 18 at 7:38
  • Re "...is to copy/paste answers back into ChatGPT and it will respond to it as a continuation of the conversation": Interesting. Do you have some examples and/or references for that? Jan 18 at 18:55
  • @PeterMortensen openai-openai-detector.hf.space
    – n8.
    Jan 19 at 16:40
  • Also this: nypost.com/2022/12/26/…
    – n8.
    Jan 19 at 16:42
  • My main point is that SO isn't the only one suffering from this issue, and to try and concoct a solution internally is wacky. Smart people are already very far down this road, we have search engines to identify who they are.
    – n8.
    Jan 19 at 16:43
  • 7
    this just isn't really relevant to the conversation, tbh. We aren't expecting users to go out and look for chatgpt answers to report. That's not what this is about. Mods and other involved users are already on the case and already know how to find such answers and deal with them. Obviously, report things you think are chatgpt answers, but we don't need this to be a witch hunt.
    – Kevin B
    Jan 19 at 16:51
  • 7
    @n8. If that is your main point, you might want to edit your answer to actually say so. Right now the prominent focus is on a single suggestion that doesn’t actually work as described. Jan 19 at 18:35
-32

Reading through the answers and comments, I can't help but detect a lot of bias, seemingly out of fear for the unknown or potential competitor.

This line in the OP is telling:

in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.

Why set out to determine it 'is actually bad' instead of good. In my experience, it's usually correct (because I ask the right questions). In the cases it's not, it's useful to discuss with ChatGPT where the mistake lies. With some frequency I ask it to reread its reply and whether it is sure that's correct.

Similar in many comments, where people clearly show bias without supporting or even convincing arguments. Comments like "it is stupid" and "it's a good joke generator". The main argument seems to be "it's not always correct". Yes, neither are all (or even most?) human answers, but that aside, if that's your main argument, what will you do in 6 months or 2 years?

Personally I think ChatGPT is hands down the most productive assistant / near-coworker I ever had (in 30+ years IT and coding) and anyone not adapting it ASAP to gain at least experience with it is contributing to their own demise.

Important to understand is that it's an assistant, a tool, not a substitute. AI won't replace developers; developers who use AI will replace developers. Pick a side that suits you and your family. Sticking your head in the sand isn't a fruitful approach to AI, embrace it, control it, use it to increase productivity.

Posting answers or questions written by ChatGPT straight to SO is like copying and pasting from another site, but banning questions and answers ChatGPT assisted in writing just seems wrong. It's almost like banning a spell/grammar checker.

I learned never to complain without offering alternatives. People posting answers should be held accountable for bad answers. That way they'll put in the extra effort to make sure the AI-assistant answer is useful to the one asking the question. Whether it's text, questions and answers, or code, everything an AI produces should be considered a draft. Perhaps a test section limited to certain topics, or show the ChatGPT-assisted answers (allow answerers to mark them as such) at the bottom of the answer list, collapsed and hidden till the reader opens them. Anything that doesn't involve throwing the baby out with the bathwater.

31
  • 14
    "seemingly out of fear for the unknown or potential competitor." Wrong. This bias is based on a understanding of how LLMs work, and what their limitations are.
    – Cerbrus
    Sep 5 at 14:38
  • 12
    "Actually bad" vs "Actually good" doesn't matter. The same level of effort is required to validate it.
    – Cerbrus
    Sep 5 at 14:39
  • 6
    "what will you do in 6 months or 2 years." We'll cross that bridge when we get to it.
    – Cerbrus
    Sep 5 at 14:39
  • 21
    You're missing the point that GPT was causing a flood of low-effort generated copy-pasted content. There was no way to accurately moderate all of it. Your last paragraph assumes users are honest. They're not. They're just dumping AI-generated text on the site and seeing what sticks.
    – Cerbrus
    Sep 5 at 14:41
  • 10
    Seems like you're missing the point here. We're not banning chatgpt due to fear of being replaced... banning it in that case would have no effect on the outcome anyway. Instead, it's banned for the reasons outlined in the question: the success rate is too low. Yes, you using it yourself can poke and prod chatgpt enough to end up at a valid answer, however, that doesn't work for generating long-term useful content, particularly when answerers use it as a fire and forget tool for farming reputation rather than for producing high quality content.
    – Kevin B
    Sep 5 at 14:44
  • 9
    "Important to understand is that it's an assistant, a tool, not a substitute." That's why it is banned as a substitute for manually writing answers, not as an assistant, a tool. This answers seems to be missing what the ban is about: People are still free to use ChatGPT themselves. As many (all?) of the positives mentioned here require interacting with ChatGPT, it is not suitable for a Q&A format where answers are fixed and discussion is intentionally kept to a minimum. Sep 5 at 14:58
  • 3
    @Xartec Clearly the latter, however the former is a mixed bag. It's difficult to ban the one without also banning the other.
    – Kevin B
    Sep 5 at 15:05
  • 7
    Let me put it this way. Users who properly use chatgpt to assist creating their answer are creating answers that are indistinguishable from answers that aren't assisted by GPT at all. if they're indistinguishable we clearly can't do anything about them.
    – Kevin B
    Sep 5 at 15:08
  • 5
    If users need to be told how to use chatgpt to write their answers they're clearly using it for the wrong reasons. I don't think a guide would help, given the existing guidance we have for writing questions and answers is largely ignored anyway.
    – Kevin B
    Sep 5 at 15:16
  • 5
    I mean, you're missing the point, as i expected, ;) the correct use of chatgpt is as a research assistant or a last resort at getting ideas, not a code-writing service or debugging tool. It shouldn't be used to explain what code does or why it was written in the way it is without heavy work from the user in improving the output, given most of the time the output is full of useless or irrelevant information such as "how an if statement works".
    – Kevin B
    Sep 5 at 15:23
  • 5
    @Xartec If someone treats ChatGPT as a draft and then improves on that draft manually their answer is not the output of ChatGPT rewording it and not covered by the ban. The point of treating any verbatim output of ChatGPT as banned is that one cannot efficiently tell the difference between "small rewrite" and "complete rewrite" (or supervised/unsupervised, or whatever you want to call responsible versus irresponsible use) since ChatGPT by its nature always rewrites. Sep 5 at 15:26
  • 2
    @Xartec fair, however, it's still a solution that doesn't solve the problem at hand. (this ban, on the other hand, does.)
    – Kevin B
    Sep 5 at 15:38
  • 5
    To put a counterpoint into context: You don't become an expert at programming by using autocomplete or an IDE. Those who contextualize this can do OK with AI-derived tools since they know that it's not a panacea. The problem is that around the world, there are a lot of terrible engineers that treat AI as the solution. This is why it has to be banned; a lot of people who copy-paste from this site don't get that they still have to validate what it is they're doing.
    – Makoto
    Sep 5 at 15:56
  • 5
    @Xartec - ". In which case it's my opinion ..." - It's not our responsibility as a community to teach users how to effectively use ChatGPT as a tool. In fact, Stack Overflow is NOT a learning resource, or more specifically not a replacement for adequate learning from other resources on the user's part. For every "good" output you have been able to be generated with ChatGPT I can show you 30 outputs that appear right but were actually factually incorrect. They appeared to an individual with zero domain knowledge to be correct but in reality, were factually and technically incorrect. Sep 5 at 17:38
  • 5
    @Xartec - Users already don't follow those guidelines. Given the amount of inaccuracy with regards to ChatGPT I don't believe it's worth the squeeze. Sep 5 at 19:52
-33

I have questions from ChatGPT and some of the answers were 100% accurate. Now Stack Overflow should allow accurate and acceptable answers from AI. It can save a lot of time.

It has come to experience that the logic, queries (MySQL, mongo dB) can take up to 12 hours. ChatGPT has answered and created queries like that in just seconds. (I have pro ChatGPT.) I have created an API that has multiple if-else and multiple queries with more than 500 lines of code (2000 ms response time), but with the help of ChatGPT, I have done that API with just 20 lines of code, with an average response time of 500 ms.

Now is the time to use ChatGPT and such platforms to speed up the development process. ChatGPT is really helpful to newcomers and for developing small-scale logic and functions.

15
  • 6
    You should really use a spelling checker. "sorry for any gramitical miskates." does not excuse a lack of effort.
    – Cerbrus
    Jul 17 at 10:52
  • 12
    That aside, just because ChatGPT sometimes generates correct output, doesn't mean it's a valuable addition to SE. Users can get that from the AI itself, no need to host it here.
    – Cerbrus
    Jul 17 at 10:53
  • I think about 80% it generate correct answers. @Cerbrus Jul 17 at 10:58
  • @Cerbrus some developers who native language is not english find it difficult to use AI tools for programming. they just search for error in SO and like in the ChatGPT so ChatGPT can't answer them . that why it well be helpful to copy-paste and answer and write a human explaination of that. Jul 17 at 11:02
  • 19
    So you lack the experience to be productive in creating software, and you outsource that work to GPT. You're happy with what it produces all the time, 80% of the time. Yet you fail to see why others don't value your assessment as much. It's like you skip going to the doctor with your headache, because ChatGPT said it was probably nothing. Your claim is then that ChatGPT is way cheaper than a doctor but equally useful, and your proof is that you're still alive while not paying as much as for a visit to a professional. The world has become a worse place thanks to GPT.
    – CodeCaster
    Jul 17 at 11:17
  • 6
    this seems to merely repeat points already made in several prior answers here
    – gnat
    Jul 17 at 11:20
  • @CodeCaster i think if the result is 100% accurate then we have to accept that. it not just about the copy paste. ask the chatgpt get the answer , test it and if it is correct then write you answer in SO to help others. becuase sometime ChatGPT correct answer while sometime it answer may be wrong. Jul 17 at 11:23
  • 9
    Related: meta.stackoverflow.com/questions/422392/… See this answer. That the LLM is fast and seemingly accurate enough for your use case is not a sufficient condition for letting people mass-dump AI generated answers.
    – E_net4
    Jul 17 at 11:26
  • 10
    If you are qualified to test chatgpt, just write an answer from scratch. If you can't write an answer that can be edited to suffient quality without chatgpt, you probably don't know enough to know if a chat gpt answer is correct or subtly but significantly wrong. Jul 17 at 11:27
  • 13
    "i think if the result is 100% accurate" And that's the problem. It's not. Not even close.
    – Cerbrus
    Jul 17 at 11:29
  • Please read this ; meta.stackoverflow.com/a/423112/9570734 Jul 17 at 11:35
  • 4
    After you read all the answers here explaining why we don't need it on SO.
    – Cerbrus
    Jul 17 at 11:42
  • 4
    To be honest, your specific case is the exact reason I rail very hard against AI in code. Someone who doesn't really understand what they're doing and can't really independently verify what the actual output of the LLM is would fare no better in a practical situation than someone who can copy and paste from Stack Overflow (ironically).
    – Makoto
    Jul 17 at 15:28
  • 7
    @Engr.AftabUfaq my point is that I don't trust you to validate an answer given by ChatGPT. What does "100% accurate" even mean? That it compiles/lints and runs without errors? What bar is that?
    – CodeCaster
    Jul 17 at 17:20
  • @CodeCaster Over-treatment is a very serious issue. Don’t go to the doctor because of simple headaches. If ChatGPT tells you that your headache is probably nothing, and it makes you cancel your plans to see the doctor, ChatGPT has made the world a better place. Calm down, rest, stop unnecessary medication, and stop wasting your doctor’s time. Jul 23 at 5:41
-34

I was reading this post extensively and I'm really worried by the reactions.

The first thing I remembered was cab driver's reaction when Uber came to my town. They reacted extremely angrily. They got together, persecuted Uber drivers. They used lawfare and political connections against Uber drivers, and they got to physical fights with Uber drivers to the point Uber drivers initially couldn't reveal themselves when they picked up a client because cab drivers where constantly looking for them and picking fights with them and their clients.

This scenery lasted a couple of months until they realized the inevitability of their fate. Some cab corporations even tried to educate cab drivers to give candies and treat client the best way possible. Nothing could resist Uber and today there are very few cab drivers resisting in my town.

Now when I read this thread I notice some very worrying trends:

First the level of ChatGPT answers on this matter (most upvoted answers) just shows how advanced it is. The sarcastic answer was terrifying.

Second, I saw that most people see the ban as the correct option, without having a reasonable way of distinguishing AI-generated answers from human answers. I think there are only two possible ways: letting users decide if it's an AI generated answer or having direct help from OpenAI itself. But I really don't think that humans will have the ability to tell one from the other. That leaves us with the only option of asking OpenAI for help. Has anyone contacted them yet?

Then the level of harshness with those who advocated ChatGPT integration (most downvoted answers) only reinforced the memory of cab drivers reaction. This worries me the most because disruptive technologies have to be embraced from the start or things will only get worse.

Adding to this is a very compelling pro-AI factor: the fact that some users are really fed up with aggressive answers from humans in SO and would much rather prefer to interact with a AI that treated them good. This is getting so critical that some people left SO altogether. I live in Brazil and I don't use the Portuguese SO because of extreme rudeness I got there several times. English SO is less bad, but I can assure you that if there were any other options people would embrace them in a heartbeat.

Just like I and many other people were fed up by cab drivers unethical attitudes like trying to figure out if the users knew the town so they could make a longer paths to the destination, not giving correct change, rudeness and many other things. That made Uber irresistible. As soon as it was available I never ever used a cab again. Since the first day it was on town.

Finally, remember that ChatGPT is learning and its answers will only get better and better. What are wrong or bad answers now will probably be the best answers in the future.

My advice (which will make my answer quickly get to most downvoted): If we can't get OpenAI to help, integration with ChatGPT is the only possible option. Create a clearly labeled automated answer for each post from ChatGPT and let users downvote it if it's bad as with any normal user.

This way users will have an immediate answer they know was AI generated and they will know there is a greater risk of being wrong, just like we know with automated translations of text.

I know this will be unpopular because it will make more difficult to build reputation points, especially if ChatGPT improves its answers. It's still better than losing all SO or making fruitless attempts to differentiate AI answers from human answers.

Any other option will not stand this test.

Maybe this means that some time in the near future SO will be no longer relevant because you can just ask an AI what the problem is with no need for human interaction. Well, if that is the case SO is already doomed and needs to rethink its business model from the scratch. If that's the case, it's better to embrace it as soon as possible. Humans can always help with comments and corrections at least while it still generates wrong or bad answers. But if it gets really good at it, there is no possible future for SO.

If you can't beat them, join them - a popular proverb.

Resistance is futile, you will be assimilated - Borgs.

20
  • 26
    "remember that ChatGPT is learning and it's answers will only get better and better. What are wrong or bad answers now will probably be the best answers in the future." then why don't we discuss this in the future, rather than the present. We act on what we have right now. And right now ChatGPT can generate content that is very wrong and potentially dangerous. Which is a big part of the reason why it was banned.
    – VLAZ
    Mar 16 at 17:14
  • 8
    Instead of comparing people to physically violent folks, I recommend to actually acknowledge and address the points that have been brought up for the umpteenth time already. Mar 16 at 17:15
  • 32
    Uber vs cab drivers is a flawed analogy. Using an Uber, you get to your destination, just like a cab. If you have an "answer" provided by ChatGPT or other AI generation (at the current level of capability), you don't have an actual answer. You have "eloquent bullshit" that sounds like an answer. It is, sometimes, an answer, but it's quite likely to be hilariously wrong, self-contradictory, and/or insidiously wrong such that it takes a subject matter expert to see that it's incorrect. So, it's not actually an answer and is likely to substantially mislead readers. That's not a replacement.
    – Makyen Mod
    Mar 16 at 17:21
  • 17
    I'll put it more laconically. You can ask ChatGPT whatever you want. It might even work for you. But you shouldn't be posting it here and representing it as your work. Worse, you shouldn't really look to use it in your code and represent it as your work, since depending on what you're working on, you could get bit hard by licensing.
    – Makoto
    Mar 16 at 17:22
  • 16
    @Makyen I guess it would be like hiring a cab and having the cab driver confidently take you to some other destination and then drop you off. Eventually you might realise you're in the wrong part of town, or even the wrong town altogether. Mar 16 at 17:57
  • 14
    After ~30 answers that all state that AI is the future (I agree with that) and that ChatGPT is a big step forward (also agree), there is not a single post here that states a reason why ChatGPT answers should be posted on SO. Or how we deal with the fallout of people copy-pasting AI generated answers faster than they can be reviewed without any checking.
    – BDL
    Mar 16 at 18:45
  • 6
    "But if it gets really good at it, there is no possible future for SO." Stack Overflow did not ban that AI. It banned the AI that's really lousy at it, but makes it really easy for someone who has no clue what they're talking about to post a dozen plausible-sounding, but wrong, answers a day.
    – beaker
    Mar 16 at 18:45
  • 12
    There are so many misconceptions in here... And none of them are not yet discussed in the answers here.
    – Cerbrus
    Mar 16 at 19:17
  • 5
    "I saw that most people see the ban as the correct option, without having a reasonable way of distinguishing AI-generated answers from human answers <...> But I really don't think that humans will have the ability to tell one from the other." Blatantly incorrect. There's detection software out there that's plenty accurate to detect what is and isn't AI-generated.
    – Cerbrus
    Mar 16 at 19:19
  • 9
    "the fact that some users are really fed up with aggressive answers from humans in SO and would much rather prefer to interact with a AI that treated them good." Why does that interaction need to be on SO?
    – Cerbrus
    Mar 16 at 19:21
  • 16
    "remember that ChatGPT is learning and its answers will only get better and better." Blatantly incorrect. The "P" in "GPT" stands for Pre-trained. It's not learning, and it's not getting significantly smarter. Certainly not smart enough to provide answers with any measure of consistent technical accuracy.
    – Cerbrus
    Mar 16 at 19:23
  • 18
    "If we can't get OpenAI to help, integration with ChatGPT is the only possible option." Again, why does it need to be on SO? Who is going to pay for that? What benefit is there to having SO embed the mediocre output, over users just going to ChatGPT if they want to?
    – Cerbrus
    Mar 16 at 19:24
  • 12
    "I know this will be unpopular because it will make more difficult to build reputation points" That has absolutely nothing to do with this...
    – Cerbrus
    Mar 16 at 19:25
  • 7
    "If you can't beat them, join them - a popular proverb." We aren't setting out to do the same thing as a chatbot. we are not a chatbot. See also The future role of Stack Exchange vs. emerging AIs and Could ChatGPT be a viable way to answer people's questions?- both of which I have written answers to.
    – starball
    Mar 16 at 22:41
  • 15
    Why are all the ChatGPT supporting answers analogies with things that have nothing to do with the subject matter?
    – David
    Mar 21 at 10:30
-35

Banning all ChatGPT answers is a good temporary move, as it opens a time window where it could be discussed thoroughly, but it, IMO, shouldn't be permanent, as it could also help if used according to its capabilities and by acknowledging its limitations.

Consider the following situation:

Someone sees a question they know the answer to.

They don't want to worry about the structure of the answer, so they use ChatGPT with a prompt that directs it towards the correct answer of the problem (for example, if the question is "How do I remove and get the last element of an array?", a possible prompt might be "Using the array.pop method, write a Stack Overflow answer to the question ..."

They then check and verify the answer to see if ChatGPT has done any mistakes, and either direct ChatGPT to correct the answer, or correct it themselves.

They then post the answer.

Is this helpful to SO? I would assume it is as the user who asked their question gets an answer that works, and the answerer spends less time formulating and explaining the answer and more time worrying about the correctness of the answer. As ChatGPT is a language model, here it would have been used correctly according to its capabilities (language and not programming - the programming knowledge comes from the answerer).

Should this be banned permanently? Permanently banning all ChatGPT answers means this should be banned as well, even though it actually is helpful to the Q&A format we have going on here.

Bad ChatGPT answers are just bad answers, and I don't think we should have another rule specifically for ChatGPT. Spamming good-looking but bad answers with AI tools and abandoning them to "see the numbers go up" should be the behavior that is banned.

11
  • 9
    The problem is that users have been proven to be untrustworthy, and have been going for quantity over quality when using the AI to generate answers. Besides, answers that are simple enough for the AI to consistently answer correctly, generally have excellent due targets on SO... TL;DR: There's nothing of value to SO, generated from CGPT.
    – Cerbrus
    Dec 13, 2022 at 16:44
  • 3
    The use-case described by this answer would be a great way to use ChatGPT. Unfortunately, it's not how it's currently used. On the long run, we have to find a way how responsible use of AI can be allowed while still having methods to prevent flooding the site with garbage.
    – BDL
    Dec 13, 2022 at 17:41
  • 1
    I had a similar thought: What if you have the skills and willingness to validate the AI-generated answer, even if you didn't come up with it? Such good-faith use of a chat AI by an actual expert would be completely indistinguishable from a human answer, and such use would be unenforceable. We're being asked here to self-enforce this ban on ourselves. If you don't want to self-enforce this ban on yourself, others may never notice, but you're crossing a different line here by publicly advocating that others not self-enforce themselves.
    – durette
    Dec 13, 2022 at 18:36
  • 11
    There really isn't much value in carving out an exception for cases where the tool is used in a way such that it won't be recognizable as chatgpt and is actually producing valuable content, people using it have already proven that they're overwhelmingly unwilling to take those steps. People who are takin those steps aren't spamming the site with nonsense posts, but they're still potentially posting plagiarized content.
    – Kevin B
    Dec 13, 2022 at 18:45
  • 1
    "Is this helpful to SO?" To answer the question in the answer: Yes it is helpful. Caveat: only a minority will actually do it like this. But otherwhise it's a valid idea. Dec 13, 2022 at 19:48
  • 2
    The main problem though, with this suggestion of augmented answering, is that you never know when the chat bot will randomly introduce fake or false information to support its possibly correct larger point. These nuanced false points might seem harmless if for example we are talking about resolving the argument of who won the last World Cup. The chat bot would perhaps make a comment like "Argentina won the 2022 World Cup. It was their third time matching up against France in the finals, and only the second time Messi won". That last part, unless you are truly informed, may seem real.
    – Travis J
    Dec 24, 2022 at 7:28
  • 4
    In code, it can be dangerous to use those extra parts, especially since nuance is often very important (for example a wrong conversion to let's say, oh I don't know, feet to meters #mars). Perhaps with things of a more lenient nature, these white lies don't particularly harm anything, but when people's savings, kids, or lives are at stake, we can't just lightly gloss over such glaring inaccuracy.
    – Travis J
    Dec 24, 2022 at 7:28
  • @TravisJ of course I am not suggesting people to blindly copy and paste those answers directly from ChatGPT, rather here, the tool would be used just as a glorified sentence generator, and the output it generates should then be left to the human operator to check. Dec 27, 2022 at 22:25
  • 4
    @kahveciderin - Unfortunately a very large sample size of the minority have proven they cannot be trusted to use ChartGPT to help them submit an answer. They have proven they do not have the knowledge to confirm the answer content is (correct, accurate, ect.). That sample size has been extrapolated across multiple communities. Dec 28, 2022 at 12:31
  • Main Problem with this Scenario I think, ... is that 'ChatGPT' doesn't check for DUPLICATES, oops...!
    – chivracq
    Jan 15 at 3:17
  • I am against permanent ban, but not for the reasons given here. Everyone should be allowed to learn from their mistakes. It is as simple as that. Banning for a long time - sure. It needs to hurt to sink in. Permanent banning is needlessly extreme.
    – Gimby
    Mar 3 at 14:20
-37

Why not have all new questions include an automated answer by an official Stack Overflow ChatGPT account, with a clear indication that this is the ChatGPT response? Maybe even show the user the ChatGPT answer before the question is posted, to reduce duplicate/low-quality questions.

This way, it just gets the ChatGPT controversy out of the way... ironically, by embracing it. If the answer works, then great. If it doesn't work, well now at least there is a Stack Overflow sanctioned answer written by ChatGPT to compare new answers against. But if there's already a ChatGPT answer, why would anyone answer it with another ChatGPT answer?

If the problem is users abusing questions with quick, low quality answers... well that's a different problem; those users will always exist.

14
  • 8
    Because, to quote the question post: Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers. Mar 30 at 0:50
  • 25
    I don't get where you people are getting "the ChatGPT answer". There are multiple requests for this but they all fail at that point because ChatGPT doesn't give one answer. It generates statistically plausible text which can be different answers. Even if we assume that SO will show a ChatGPT answer at asking time and/or after posting a question, that answer need not be the same as you'd get from generating again. The claim that it would discourage others from posting a ChatGPT generated answer is basically unbacked by anything as it's not just the same text spewed out.
    – VLAZ
    Mar 30 at 4:44
  • 12
    For instance, here are two different ChatGPT responses to your suggestion. They're similar-ish, but about 20 seconds of editing could eliminate the most obvious parts. Also, an answer being posted already definitely doesn't stop people from posting the same general idea again.
    – Ryan M Mod
    Mar 30 at 4:52
  • 13
    Self-contradictory (and wrong) within the same sentence: "Is the correct spelling "StackOverflow"?". Response: "Yes, that is correct. "Stack Overflow" is a popular question and answer website for programmers, and the correct spelling is "StackOverflow."" Mar 30 at 13:40
  • 3
    And it only wants to please you (right or wrong): "Are you sure about the spelling?". Response: "I apologize for the confusion. While "StackOverflow" is a common way of writing the name of the website, the correct spelling is actually "Stack Overflow" with a space between the two words. Thank you for bringing this to my attention." Mar 30 at 14:03
  • 1
    I'd support this. Seeing a GPT generated "here is an AI's best attempt" next to user created content would be good for the site. May 3 at 19:56
  • 1
    @Yakk-AdamNevraumont Good, how? Users asking questions generally don't have the best understanding of the subject, and a convincingly written wrong answer would do more harm that good.
    – Cerbrus
    Jun 5 at 14:10
  • @Cerbrus 1. It sometimes isn't wrong, 2. It competes with non-official AI answers, 3. It can provide a basis for someone's correct answer. Also, labeling it as "AI's best guess" also gives a comparison between what humans answer and what AIs answer. Jun 5 at 15:56
  • 1
    @Yakk-AdamNevraumont if one wants AI's best guess, they can go to a resource that already provides that. It makes no sense to add a limited version of that to SO.
    – Kevin B
    Jun 5 at 15:59
  • I like this idea. Having an AI user who would answer questions would be a fine way of training a model. The model could be refined by using up/down votes and commenting on its answers. Users could allow or ban the AI account from providing answers.
    – Catriel
    Jun 5 at 16:40
  • @KevinB Except, why not keep the eyeballs here? Jun 5 at 17:03
  • The solution existing here would be half baked compared to a tool made for such use
    – Kevin B
    Jun 5 at 17:51
  • 2
    "It sometimes isn't wrong" Well isn't that great! Dude, that's not even close to being good enough.
    – Cerbrus
    Jun 6 at 7:22
  • 1
    I would support an experiment to provide a ChatGPT-generated answer on the site as an option for the asker, but that doesn't mean it should be posted as an actual answer for future visitors to the question. When it works, it would probably help reduce the number of trivial and duplicate questions. (Flabbergastingly, that's apparently not what the company wants, probably because that would reduce advertising revenue; but it would help maintain the quality of the site, and reduce the need for manual administration.)
    – tripleee
    Jun 13 at 6:13
-37

TL;DR: assimilate, don't exterminate!

I would like to see a separate section for AI-generated answers, i.e. why not just embrace it by retaining AI-generated answers but keeping them separate from human answers?

That serves two purposes:

  1. AI can distinguish AI-generated answers so that it doesn't feed them back into itself when they no doubt use ordered site content like this to generate answers.
  2. AI answers can still be viewed and voted on, and maybe some will even become the accepted answer.
7
  • 13
    Why create more work, which will in turn create more work? A feature like what you suggest requires time to be put into implementing it, and then once the feature is made, it will require even more time by curators to confirm the content is not invalid. What is the benefit?
    – Daedalus
    Feb 17 at 3:11
  • 20
    Specifically, following up on @Daedalus's comment, what is the benefit of integrating an AI service like ChatGPT directly into SO? People who want AI-generated responses can just ask the AI. That service already exists. People who want answers written by human experts can come to SO. We already provide that service. Why mix them? Beyond that, the reason we don't have a section for AI generated answers is the same reason we don't have a section for answers written by monkeys with typewriters: those answers are terrible. They don't meet our minimum quality standards. Feb 17 at 6:00
  • 3
    Who's gonna pay for that? At the rate SO is getting questions, this would get very expensive very quickly.
    – Cerbrus
    Feb 17 at 13:00
  • Who said the answers needed to be curated? And who said ChatGPT is the only AI? Come on people, this technology is only going to get better. Where's your imagination?
    – racitup
    Feb 18 at 4:36
  • 18
    We don't create policies based on our imagination. We create them based on the reality that is in front of us, that we're dealing with right now. (As for who said the answers needed to be curated: that's the whole design principle/goal of this site.) Feb 18 at 5:45
  • Okay, give it a year or two and let's see how SO is doing with your policy...
    – racitup
    Mar 6 at 18:56
  • 7
    And why do you care about how SO will do in a year or two, with this or any other policy? just use whatever tool that works for you. that was true in the past, it's true now, and will be true in The Future (tm). if SO dies because something else replaces it... so be it. I don't get all these answers worried about "SO should adapt, or it will be replaced by something else!!!". Mar 21 at 15:30
-38

It might make sense to consider integrating ChatGPT into the site engine itself.

In this way, an answer received from it should be displayed in the least annoying way with a note that this answer is not related to SF, was received programmatically and is most likely wrong, and the possibility of drowning it in minuses.

This will reduce the motivation to post similar answers, as well as create data for training neural networks (including in the minds of site users) that recognize the generated answers.

5
  • 11
    This has already been suggested in this post, multiple times. Dec 5, 2022 at 23:58
  • Hm. In comments? Understood, please understand and forgive. :( I haven't seen similar answers. Dec 6, 2022 at 0:01
  • 2
    What is "SF"? It would normally be Server Fault here. Do you mean Stack Overflow? Dec 6, 2022 at 0:14
  • 9
    You don't exactly have to look hard for it. They're all on this page. The answers are: one, two, three,
    – Zoe Mod
    Dec 6, 2022 at 10:57
  • 1
    @ZoestandswithUkraine: There's one difference this answer has from the others that suggest basically the same thing: this one would frame the AI answer as being likely wrong, and warning people not to post it or an answer like it. The others all suggested that the AI answer might actually help people solve their problems, and would get the same "benefit" without people having to post such answers. That said, I don't think this is actually a good idea. but it is different; I nearly upvoted. (The dollars and electricity to AI-generate answers from the ask interface make this a non-starter). Dec 13, 2022 at 4:29
-38

A Solution?

I agree to most other answers, except the "but there is no solution" part. Also, I believe not all posters here understand that we're just at the beginning.

Hence, my proposal would be to attack, instead of defend.

Why not enable a feature that sends all questions to ChatGPT right after posting and display the result alongside the answer? It should be marked as the ChatGPT answer and users could opt to not display it.

  • This would immediately stop people from abusing ChatGPT to farm reputation. The similarity would be too obvious, at least for the case where the question is just copy-pasted. If ChatGPT users enhance the question to improve the response, they already added some value and would not be in rapid fire mode anymore.

  • It would give the benefit of the doubt that an AI answer might actually be valuable. By rating those answers the same way as rating human answers, we can see how they rank with others.

  • Humans who write answers can refer to it and agree or disagree, if that makes any sense. They can point out whether there is only a minor mistake in the AI answer or whether the answer is based on a misunderstanding or predominant misconception on the internet (as the source of information).

I think this solution would scale for some time to come, but I am not sure, how feasible that is. Will Stack Overflow be charged, or can Stack Overflow sell this to OpenAI as a marketing hack? I don't know.

9
  • 21
    No. This has been suggested plenty of times already. Look at the other answers here as to why this can't work.
    – Cerbrus
    Mar 3 at 17:07
  • would you care to point me at it, because I didn't see it.
    – Ingo
    Mar 3 at 17:10
  • @Cerbrus, just found it on the next side, sorry for not being thorough in the first place. It seems indeed, that the solution could be difficult, but I am not entirely on your side. In the long run, ChatGPT will become less expensive and in the short term, it may wish to run this as advertisment the same way I can go there and ask questions to it.
    – Ingo
    Mar 3 at 17:20
  • For the point with repeated questions yield different answers: I would assume that the different answer will be somewhat suffering from the same problem. Might be difficult to compare word by word, but might be good enough to discourage abuse.
    – Ingo
    Mar 3 at 17:22
  • 20
    ChatGPT can offer completely different answers to the same question, including absolute contradictions of what it said mere seconds ago. The similarities between pairs of ChatGPT answers are structural in nature, not content based; there is no use in having a "reference answer" to spot other generated answers for the same question. Mar 3 at 17:39
  • 8
    "… in the short term, it may wish to run this as advertisment the same way I can go there and ask questions to it." That would be a rather poor advertisement. ChatGPT isn’t made nor meant for the kind of questions SO is made for. Expecting experts to waste their time trying to curate a stream of technical nonsense isn’t a winning story… (This is in essence something this very meta-question already said - there is just no capacity to manually vet all the content that ChatGPT has generated, let alone could generate, for SO.) Mar 3 at 17:46
  • 2
    gotcha, and to be honest, I didn't expect the answers would be 'contradicting' and that sounds like a general flaw to me. It would at least make sense, if ChatGPT would enhance itself based on the content it receives, but I was not able to observe any valuable learning, based on my feedback. Even in the most stupid way.
    – Ingo
    Mar 6 at 16:50
  • 11
    @Ingo ChatGPT doesn't "remember" the conversations it has. As I said here "ChatGPT generates plausible text, consistent with its training data and the prompt, but it doesn't know what it's talking about, and it has no way of representing or evaluating the truth of its utterances. Yes, it can say true things, but it can also say complete nonsense, and it can't tell the difference". It's designed to manipulate syntax, not semantics. Stephen Wolfram gives a good outline of how it works in the first of his articles linked in my answer.
    – PM 2Ring
    Mar 8 at 14:20
  • See also now the train wreck at meta.stackoverflow.com/questions/425162/…
    – tripleee
    Jun 19 at 3:43
-41

It seems like a slippery slope here. Am I banned from using ChatGPT for doing my own research? Certainly not. So if I gain education by way of ChatGPT, am I then banned from conveying that knowledge by way of answering a SO question? I wouldn't think so, as how I came to know something should be irrelevant.

So then, I suppose the question is "If I use ChatGPT to research a topic solely so that I can answer a question on SO, is that wrong?" I can't think why it would be, so long as I'm properly curating the answer from my own knowledge.

And if that's ok, then the question becomes "How much does my answer have to differ from the ChatGPT answer that I used to inform myself so that I could answer the SO question?"

8
  • 15
    "…if I gain education by way of chatGPT…" Ah, I see this is a pure hypothetical, so we don't have to worry about the answer to it, because that's not going to happen. Feb 15 at 6:51
  • 15
    "so long as I'm properly curating the answer from my own knowledge" The fact that most people weren't doing this is why we're in this mess in the first place.
    – E_net4
    Feb 15 at 9:28
  • 2
    How you learn isn't going to turn into you write answers in a form that will 99% of the time be wrong but very well written.
    – Kevin B
    Feb 15 at 15:28
  • 1
    Wow, 14 dislikes. Guess I hit a nerve. Funny how a dissenting opinion amidst a sea of minds that are already made up leads to downvotes with no real mention of why. I'd love to know where the flaw in my logic lies. I've already proven to myself that chatGPT makes a great research assistant. So this response came from an informed position. I could prove that statement if held to the fire. But there's no interest here in evolving opinions here it seems.
    – CryptoFool
    Feb 15 at 16:32
  • 17
    Yea, i mean, funny how people express agreement on meta with votes, and how unpopular opinions meet a lot of disagreement. Almost as if the system is working exactly as designed. Funny! ¯\_(ツ)_/¯
    – Kevin B
    Feb 15 at 16:52
  • 18
    So where exactly is the slope and why is it slippery? The announcement makes it pretty clear where the line is drawn and why it is exactly where it is. If you manually write about your own, verified knowledge then no one cares where that comes from. Feb 15 at 17:27
  • @MisterMiyagi - If the metric is "copying/pasting from the chatGPT site", then you're right, it's not a slippery slope. And by reading the notice of the ban, that's the way I read it. The slippery slope would be if one wanted to take it any further than that...to say that one can't "paraphrase chatGPT output".
    – CryptoFool
    Feb 15 at 23:34
  • 8
    The problem is with people blindly copy-pasting content in bulk, without validating the contents... If a user were to take the effort to paraphrase the content (manually, not with some kind of AI), I'd presume they'd at least check if it's correct.
    – Cerbrus
    Feb 17 at 13:02
-42

I have an idea for allowing the productive use of ChatGPT:

  1. Users who post AI-generated content must check a box indicating that. Perhaps it could go next to the "Community wiki" checkbox. A mockup screenshot
  2. Only users with at least 1,000 reputation (or some other value) are allowed to post AI answers. New users must write answers themselves.
  3. Downvotes on AI-generated answers take 10 rep, not 2.
  4. AI-generated answers with 5 or more downvotes are automatically deleted, regardless of the number of upvotes.
  5. Users who post x five-downvote AI answers in y days are not allowed to post more AI answers for z days. Not sure what the best values for x, y, and z are.
  6. Reputation lost from downvotes on AI answers is not regained on deletion of the answer, regardless of why the answer is deleted. Reputation gained from them is lost, though.
  7. Users who break Rule 1 are blocked from answering for 30 days.

Overall, this allows experienced users to post useful AI-generated answers, while preventing inexperienced users who may not know how to evaluate the output from doing so. It also makes it "expensive" in terms of reputation to post bad AI answers. Violations of rule 1 and 2 would be caught the same way violations of this policy currently are. (Really, because the option to mark an answer as AI-generated wouldn't be shown to low-rep users, any "violation" of Rule 2 would also be a violation of Rule 1, so there's only one rule people can actually violate.)

8
  • 26
    This all depends on the user being honest, which is exactly the root of this whole issue.
    – Cerbrus
    Dec 10, 2022 at 17:47
  • @Cerbrus how are AI answers currently being caught? The same methods would work for catching users who violate Rule 1.
    – Someone
    Dec 10, 2022 at 18:07
  • 4
    How does this fix any of the problems? The issue are people who would lie about step 1 and we already apply step 5 to them. What is gained by the added complexity? Dec 10, 2022 at 18:51
  • 1
    @MisterMiyagi this allows for the posting of useful AI-generated content.
    – Someone
    Dec 10, 2022 at 18:55
  • 7
    Alright, let’s play along for a bit. How would such answers look to others? Is there going to be some indication or does it just affect voting? How does this avoid the problem of seemingly good answers - as in, how realistic is it to get -5? In my experience that only happens to absolute crap. The vote based throttling seems to be too slow for the rapid posters we see, who posts dozens of answers at once. Is there some rate limit involved regardless of votes? How do we protect against people deleting their answers at -4? Dec 10, 2022 at 19:06
  • 1
    @MisterMiyagi there would be a banner at the top of the post saying it was AI-generated. The issue of seemingly good answers could be solved by deleting at five downvotes, not a score of negative five; that way people who know about the topic and see that the answer is problematic can cause it to be deleted without their votes being cancelled by upvotes from people who think it's a good answer. If someone deletes the answer at 4 downvotes, then that's a good thing; it means the problematic content was removed sooner. Reputation lost from downvotes would not be recovered when posts are deleted.
    – Someone
    Dec 10, 2022 at 19:10
  • 10
    @Someone but there really isn't any useful generated AI content, it's all well written rubbish. Dec 10, 2022 at 22:07
  • 2
    Recommended reading on this topic: ietf.org/rfc/rfc3514.txt Dec 16, 2022 at 1:36
-47

I just wanted to suggest that any AI-generated answers not be deleted altogether but just moved to a separate, still-accessible page of "Suspected AI-generated content" for consultation. This way, in case they DO contain something useful, which is going to happen more and more as it quickly improves, the poster is still helped and no time is wasted. After all, that is the main and, arguably, sole purpose of the website as it is the service it provides and that which generates visits and, therefore, revenue. More importantly there is absolute NO HARM WHATSOEVER in doing this, nor does it cost any resources.

A just compensation mechanism could then be thought of for the people who actually took the time from their busy schedules to give meaningful answers based on their hard-earned knowledge. Maybe any rewards could be split among all genuine contributors or anyone who contributed gets an increased reward for their next accepted answers. I'm sure you guys could think of something fair and fitting that could not be abused.

The inescapable reality is that there will be a drastically decreasing need for human help in the coming years not just here but everywhere. This is unavoidable and should NOT be seen as something bad at all. It's just technology replacing manual labor again, this time at a much grander scale. This is a great thing, an incredible thing, and needs to be accepted and used for good, certainly not resisted in any way.

I must say that I would find it very funny if Stack Overflow, one of the world's greatest bastions of programming, machine learning and all things computer-related, were to treat arguably the greatest fruit of its subject matter with contempt and prejudice instead of teaching the world by example as how to incorporate AI into existing solutions without there being any adverse effects for the humans that already participate in them.

12
  • 6
    The social impact of ML will be massive, and perhaps the subject matter at hand is a taste of what it is to come. But people should be allowed to be cynical about these developments without their views being labelled as "contempt and prejudice". This isn't an honest way to argue.
    – halfer
    Dec 27, 2022 at 21:12
  • They are so long as there is no direct impact on the quality of the service being offered, as far as I am concerned. For instance, I would like the right to go through AI-generated content once I've tried everything else, particularly those generated by others since I am not an expert and my prompts are of lower quality than those generated by someone who is. I don't care who typed the answer or whose knowledge it is, so long as it helps me I'm happy. If it's faster and easier for whoever did it, that's all the better. Dec 27, 2022 at 21:16
  • 8
    @PythonMillionaire: If you want answers from an AI, then why not copy-paste your question to the AI site? If the answer helped you, you are highly encouraged to self-answer your problem, as long as you reformulate the answer of the AI. As of now, most answers that get deleted are generated by exactly that way, copying the question and posting it to the AI without any editing. Then they copy back the answer, also without checking for correctness or changing anything.
    – BDL
    Dec 27, 2022 at 21:35
  • 1
    I certainly understand and agree that mere copy pasting in and of itself is pretty problematic. However, my reasons are that my prompts aren't as good and I can't fact check. AI plus intermediate-level human knowledge is far, far more useful and accurate than just AI plus beginner-level knowledge. I don't know what information is relevant and worth sharing with the AI and a lot of other things. I agree that mere copy pasting has an incomparably higher chance of producing garbage answers but humans can also do that and we can at least expect AIs to constantly improve, not so humans Dec 27, 2022 at 21:47
  • 3
    @PythonMillionaire Users aren't fact checking. They aren't applying intermediate level knowledge. You can't do that for an answer in under 5 minutes. If you do do those things, then this policy probably won't trigger because no one will notice. Its about having a response for the bad actors. Until we have a manageable process for preventing those who exploit the system, then the policies have to be broad to minimize volunteer moderator effort. Dec 28, 2022 at 2:32
  • 1
    I perfectly understand and agree! Just put them ALL in the page I mentioned. No need to delete Dec 28, 2022 at 2:40
  • 9
    “More importantly there is absolute NO HARM WHATSOEVER in doing this, nor does it cost any resources.” - I strongly disagree that there is no harm caused by users using any tool similar to ChartGPT to post answers, low quality trash generated by ChartGPT, takes very limited valuable review time away from reviewing actual potential high quality answers. Horrible low quality content generated by ChartGPT or any similar tool has absolutely no place on any Stack Exchange. Dec 28, 2022 at 12:16
  • 1
    Hi Security Hound. That's not what I said, though. I said there is no harm in moving such replies to a separate thread where they are appropriately labeled as AI content as opposed to deleting it all outright Dec 28, 2022 at 13:38
  • 7
    We don't get tools like that. In fact, that would take some serious development work to make happen. Deletion is still the best tool for the job right now. Remember, deletion is not necessarily permanent
    – Machavity Mod
    Dec 28, 2022 at 13:45
  • That's too bad. Hopefully deletion won't be permanent as we really, really cannot escape the fact that everything will drastically change very soon and we need to incorporate these wonderful developments into our society as a whole Dec 28, 2022 at 15:19
  • 5
    “I said there is no harm in moving such replies to a separate thread where they are appropriately labeled as AI content as opposed to deleting it all outright” - Except Se developer resources which are limited. I would rather see improved review tools than a feature to deal with the absolute trash content generated by ChartGPT. I detected a user who submitted 12 answers and every single one, was incomplete or incorrect. A 100% track record proves that ChartGPT generated content is 100% unreliable. ChartGPT isn’t an AI. Until such time it’s 110% inaccurate CGPT content should remain banned Dec 29, 2022 at 21:31
  • 2
    SO's temporary ban & the explanation make sense. This answer raises some good points. I understand downvotes indicate disagreement, but giving reasons would be more helpful. AI is here to stay. Skilled programmers & learners, of all people, know better than to dismiss it. Many colleges ban Wikipedia use (incl. correctly cited) in student essays on grounds often similar to people's objection to content from ChatGPT. Like Wikipedia, ChatGPT can be a great learning tool, a good starting point. Who else can figure out how to harness ChatGPT's power for teaching & learning than people here on SO?
    – YCode
    Dec 30, 2022 at 20:57
-47

As GPT-4 is available for $20/month, shouldn't you rewrite your policy?


ChatGPT is actually training. Given the fact that it gives wrong code (sometimes), it is normal to not accept answers exclusively written by the AI.

But this is only the beginning.

You are at the same point that chess players were when computer engines entered the board and started to play chess at a human level.

Chess adapted to engines, and Stack Overflow should adapt to AI coding systems, perhaps develop its own AI system (they have the site content to train its project; they could open it as an account subject to human democracy).

A few decades later, the chess engines have an Elo rating of 3300, while Magnus Carlsen is rated 2900, so there will come a day when users will prefer to ask their code problems from an AI rather than a human (maybe soon). This will rid the site of bad questions that are not well-received and not answered. It will also help the site to evaluate its own content to detect errors (perhaps adding a warning: "The AI has detected a problem in this answer").

There still are chess players who would prefer that there weren't any engines, but by now chess engines have become friends of most chess players.

The AI coding systems should also become a friend of the community of programmers.

You are complaining about a first attempt, ChatGPT, the first project open to the public. It is important that AI is available for everybody, including the programming community.

45
  • 39
    How does this actually relate to the ban? It reads mostly like a vague outlook into an entirely hypothetical future which isn't there yet and frankly quite overoptimistic. Mar 14 at 14:51
  • 25
    We don't dabble in speculation here. Come back when AI has gotten batter.
    – Cerbrus
    Mar 14 at 14:57
  • 6
    We can discuss that when that "later" actually comes and we know what those "later AI" actually are. From what has been discussed countless times already, I would wager no: If there is an AI that can just generate high-quality answers, there is no need to maintain a repository of those answers. Mar 14 at 14:57
  • 21
    Chess is a whole different problem than natural language is. Solving questions it yet another different problem. Just because AI has gotten good at chess, doesn't mean it's gonna get just as good, just as fast, at solving problems. You're way overestimating the power of a language model.
    – Cerbrus
    Mar 14 at 15:09
  • 18
    Why are we even discussing this again? All of the points in this answer have been covered already in older answers and discussions here.
    – Cerbrus
    Mar 14 at 15:10
  • 31
    "You are complaining about a first attempt..." No, definitely not, we've made a policy about ChatGPT and AI generated content as it exists today, which is nearly 60 years after Eliza bot was released. While ChatGPT is a substantial improvement, it still lacks entire categories of functionality that would be necessary for it to consistently answer questions correctly, or even without being hilariously wrong and self-contradictory. The hurtles which AI still needs to surmount in order to be truly useful for answering questions are hard and will take substantial time.
    – Makyen Mod
    Mar 14 at 15:11
  • 6
    A Spanish paywalled article? Yea, no way I'm ever gonna read that.
    – Cerbrus
    Mar 14 at 15:14
  • 14
    The current ChatGPT isn't OpenAI's first release. They were incorporated in 2015. So, look at where they were then and where they are now. From that, with hopefully some acceleration factor, you could estimate how much longer it will take, but even an estimate based on that would be very optimistic as to the time-frame for improvement. Will there be improvements? Of course there will. There nearly always are. The issue is that it's gotten to the point where it's a problem for us and still has a long way to go until it's useful for what SO does. We're dealing with it as it is now.
    – Makyen Mod
    Mar 14 at 15:21
  • 17
    @Universal_learner I recommend to read up on how such an AI works, then re-annecdotes the examples you have just given. Those are quite frankly terrifying. Mar 14 at 15:35
  • 18
    I mean, that's part of the problem. You look at it and see what you think are good answers/solutions... but you don't necessarily know what a good answer/solution is. New devs using this tool are running with answers from an AI that isn't built to provide accurate/good coding advice.
    – Kevin B
    Mar 14 at 16:20
  • 12
    @Universal_learner Better than 54% means being totally average. Average for people that actually bother to do such exercises, i.e. many, many beginners. Exercises made specifically to be self-contained bites of challenges. Challenges with a huge body of available solutions. None of these match what Stack Overflow is striving for. Mar 21 at 9:09
  • 6
    @Universal_learner the ban on ChatGPT is not analogous to a ban on Stockfish being used as a tool to write and review the lines contained in chess books. It's equivalent to a ban on AI-generated chess books, which aren't banned, but would be of no use to anyone even at a task where computers are much better than at writing code. "Solve me this tactics" doesn't really have an equivalent programming question type, but if it did, the difference would be that Stockfish actually gets the right answer pretty much all the time, while ChatGPT does not.
    – David
    Mar 21 at 10:10
  • 12
    @Universal_learner good for you. Just keep in mind that obviously wrong code is better than code that tricks you into thinking it's right but actually isn't.
    – David
    Mar 21 at 10:30
  • 14
    Why do you care about this at all? if gpt or whatever does it for you, just use that? no point trying this site to be anything else. Mar 21 at 15:24
  • 10
    "ChatGPT is actually training" to be clear, ChatGPT 3 doesn't use chats you have with it as its training data. You can't "teach" it something new and have it be able to use that new knowledge in separate conversations. Probably not an important point at this point but I thought I'd point it out. Mar 22 at 9:52
-48

Prediction: SO will eventually incorporate AI trained on correct answers to clear questions.

================================================

Well, after all the down-votes and comments as if I had set a date for the end of the world, I noticed today, 15/09/2023, SO is now selectively offering its alpha OverflowAI search feature.

I expect that SO's use of AI will soon get into the answers side of things.

================================================

If it doesn't, the AI of Bing, Google or Baidu will make SO obsolete due to it being too slow to get answers to questions.

Of course, what is 'correct' in the training data needs to be truly correct, and not just what the OP decided among the early answers, as later answers may be more comprehensive or nuanced.

Raw SO answers would be poor training for AI because most answers are just not good enough or misleading, let alone the masses of unhelpful comments. The problem with AI is that it needs an eclectic and massive tranche of correct and accurate data for its training so that it is not going to be polluted too much by subsequent real-life data. SO is in a very good position to have its own AI, as it would not be sweeping up irrelevant or incorrect data that search engine AI would be incorporating now.

The role of moderators would shift to being more about checking the training data and AI answers than policing question quality, as language AI would easily pick up ambiguous phrasing and respond with suggestions, correct them, or flag for human attention.

Also, the increase in SO moderators aggressively closing newbie questions or commenters making snarky comments too often may just force more to go to AI answers. Popularity is relative to the help offered. Not providing help but insults instead will hurt SO when it is up against AI that is far more forgiving regarding the 'quality' of questions. See The Tyrannical Mods of Stack Overflow for some examples, including some where even popularity didn't save them.

I know tech experts and mods on SO don't want to be customer service reps, but that really needs to be part of the role, otherwise the more forgiving AI will eat all SO's meals before it gets to see them.

We have so many free programming tools these days with documentation that is often unhelpful. Not everyone is building Apollo guidance systems, so do not need to know all the ins and outs of their language, but just want to get the task at hand done. Stifling a poor question instead of helping them work through their issue is going to kill SO for them. But that is a labour-intensive process, which is where AI could significantly alleviate the load by interactively converging on a result, or at least clarify the question.

SO has the opportunity to step up and meet the challenge of AI, or close off, and become irrelevant. The window of opportunity is rapidly closing!

Response to comments

  1. Aggressively enforcing rules without taking the time to educate new users is what alienates new users.

  2. Competent AI is NOT way in the future. ChatGPT was only released less than a year ago, and few were expecting it to do what it does, so I would expect capacity and competence to increase rapidly now that such AI is being put front and centre in search engines.

  3. 'Why would we even have persistent AI answers at a future point when they can just be generated on demand as good as expert answers?' - that is exactly the existential question for SO posed by AI. The competency of AI can increase very rapidly with the right training, so unless SO adapts to the disruption posed by AI, it will be irrelevant, just because the current model is so heavily reliant upon competent human attention and a huge portfolio of loosely-structured Q&A decided by a popularity contest among those whose own competency is decided by a popularity contest. Populist 'truth' does not make for good AI processing!

  4. Yes, some user and mods may not want to have to be like customer service reps, and so might be happy to have ChatGPT et al be the first point of call, but the big question is whether there will be enough users after that for SO to be viable enough to satisfy what Prosus expected of it when they bought it for $1.8b in 2021. Too few users and they will just have to write it off like many formerly popular sites before it.

When AI is given facts to prioritise upon (like BNF statements for programming languages) and training effort is made to avoid irrelevancies, the current SO paradigm will seem slow and archaic by comparison.

I don't say AI is going to be a universal panacea, but in the realm of sifting knowledge given an intended goal, its speed will trounce everything else that purports to be authoritative but is still essentially based upon hearsay, SO included.

SO will always be behind the 8-ball in getting its data up to standard given the amount of ad-hoc queries and its maxim that popularity is a reliable indicator of truth and accuracy. There is not enough unbiased humans to do that job.

Generative AI will not tend to find novel solutions to questions, but at least it does catch a lot of obvious coding errors and can make suggestions based on previous solutions. SO has been losing queries to the likes of ChatGPT, so ignoring its advantages would be folly on SO's part. They are using AI now, and are in a unique position to optimise what results are used to train their own AI. If AI helps take a load off of moderators, in a productive way, they should go for it. For some mods, that may be a challenge, but after all, they are volunteers with no voting rights when it comes to using AI.

Some have decried the incidence of wrong answers from AI, or that code may be insecure. Well, SO is based solely upon what the OP decides is the best answer, and everything else is a popularity contest. Nowhere is accuracy or security guaranteed by SO, because it is not feasible to do so, just because using people to do it doesn't scale. However, using AI to find the obvious errors will, like most computer-assisted technologies, up the lower end of answers, perhaps providing feedback as they are typed.

37
  • 15
    Newbie questions should not be closed here. No question is too "simple" for SO. If you see questions closed because they're "newbie", please raise a flag or otherwise let us know, because this is unacceptable. There are, however, plenty of questions that are unsuitable for this site, for example because they are off-topic, are unclear, are too broad or too opinion-based to fit into our Q&A format, etc. Also, the rate of questions getting closed hasn't increased. We've always aggressively closed off-topic questions. Feb 17 at 5:56
  • 6
    As far as the snarky comments, I'm of two minds about that. In one sense, the borderline snarky comments are part of the "human" factor and part of what makes it fun and interesting to participate here. AI isn't going to be able to do that, but I'm not sure that's a feature so much as a limitation. As a nod to this, ChatGPT even has a feature where it can try to emulate different writing styles if you ask it to do so (e.g., "write a snarky reply"). For snarky comments that are over the line or otherwise unappreciated, they should be flagged for removal. Mods remove lots of these every day. Feb 17 at 5:58
  • 20
    What does this prediction of some far away future have to do with the problems and ban we have right now? Why would people on a volunteer platform even be interested in "checking the training data and AI answers"? Why would we even have persistent AI answers at a future point when they can just be generated on demand as good as expert answers? Feb 17 at 7:39
  • 10
    Language models don't understand data. They don't comprehend factual correctness. That level of comprehension is just not something current technology can even come close to. I don't see that happening any time soon.
    – Cerbrus
    Feb 17 at 12:59
  • 14
    "Competent AI is NOT way in the future. ChatGPT was only released less than a year ago." yet is not competent.
    – VLAZ
    Feb 18 at 8:25
  • 7
    People made the same prediction about en.wikipedia.org/wiki/ELIZA. When it comes to understanding the underlying concepts as opposed to the language used to communicate it, chatGPT has not fundamentally advanced beyond eliza. Feb 18 at 21:01
  • 6
    If only people would flag these inappropriate comments instead of making YouTube videos about them... Feb 19 at 7:18
  • 5
    @Patanjali That basically means that ELIZA wasn’t trained automatically but "by hand". The basic idea of choosing the most likely response based on weighting the input words/grammar/patterns is surprisingly similar - the key part is that neither system actually understands its input and output based on the same semantics as a human expert or technical program like a compiler would. They’re just statistical models; ChatGPT is a more sophisticated model, but it still has very similar flaws. Feb 19 at 7:18
  • 6
    @Patanjali the problem with SO is that people come here and expect hand holding and tutoring, which is not what SO is about. Users don't have to come here for training. They can visit ChatGPT, if they wish, or many other places.
    – VLAZ
    Feb 19 at 7:39
  • 11
    I've watched the video before, yes. No, a more forgiving question policy would not solve the comment problem. If anything, it would lead to more snarky/rude comments because people have no other means of dispensing with off-topic, unclear, or otherwise inappropriate questions. This site has standards, and those standards are key to our having quality content. Like Wikipedia, you can't create an article about just anything, and it can't contain whatever you want to write in it. However, SO's quality standards are also enforced by humans, who sometimes get it wrong and/or are sometimes rude. Feb 19 at 8:05
  • 18
    OK, so SO becomes a resource that people only consult when the AI gorilla can't solve it for them? Sounds good to me. Feb 19 at 8:30
  • 12
    @Patanjali I don't think you really get it. SO isn't and doesn't need to be a tutoring service. If it takes ChatGPT to finally stop the tutoring questions coming in then that's a win for SO. Not a loss.
    – VLAZ
    Feb 19 at 8:41
  • 7
    @Patanjali you aren't hired as a consultant for how to make SO profitable. You have no insight into what the company does internally, nor what it plans to do. Your suggestions are thus 1. unneeded 2. not even relevant to the company. So, feel free to stop your financial strategy plans. Or just directly apply to SE Inc. for a position which would allow you to enact them.
    – VLAZ
    Feb 19 at 8:59
  • 10
    @Patanjali Because there isn’t such a magical future AI yet, and as far as I can tell not in the next decade. Calling for a change that is both unnecessary and premature is subverting I disagree with strongly. That we shouldn’t care about making changes doesn’t mean that we shouldn’t care about not making changes. Feb 19 at 9:20
  • 7
    @Patanjali: As long as SO isn't using their AI to post answers, they are fully compatible with the ban on AI generated answers.
    – BDL
    Sep 16 at 23:11
-56

If this issue gets too far out of hand, one possible way to mitigate this might be to integrate ChatGPT and show the user a possible answer before the question is even posted. Something like this:

Screenshot of user asking a question about removing an element from a vector in Rust, and the correct answer given through ChatGPT

That would beat the answerers hungry for quick-and-easy rep at their own game. If you can't beat them, join them.

58
  • 9
    Aside from that, it doesn't make the thing with wrong/incomplete/potentially dangerous (?) answers better.
    – dan1st
    Dec 5, 2022 at 7:30
  • 72
    The problem with this is that it provides answers which look correct but aren't necessarily actually correct.
    – forest
    Dec 5, 2022 at 7:34
  • 5
    Does this answer your question? en.wikipedia.org/wiki/Luddite Dec 5, 2022 at 7:51
  • 74
    It's all fun and games until system("sudo rm -rf /") appears and the user blindly runs it
    – Zoe Mod
    Dec 5, 2022 at 7:52
  • 23
    If someone wants this, they should probably set up their own clone of SO and run it there. It's not like we don't have enough human generated crap to drown in already. | "If you can't beat them, join them." -- If we can't beat this, I'll go spend my time on something else.
    – Dan Mašek
    Dec 5, 2022 at 8:07
  • 5
    @bad_coder that's what this post is saying, and it has been heavily downvoted and explained why that's a bad idea. were you going for a joke on your name "bad_coder"? Dec 5, 2022 at 8:44
  • 8
    Even better, for the FGITW: "Is this your answer?"
    – bad_coder
    Dec 5, 2022 at 9:07
  • 11
    I'd rather see that money go to plagiarism checking rather than creating a questionable answer that may or may not be correct (but often isn't)
    – Zoe Mod
    Dec 5, 2022 at 10:20
  • 11
    @Spidy did you miss the part where AI writes crap answers?
    – Cerbrus
    Dec 5, 2022 at 11:48
  • 16
    In addition to the possibility of these answers being wrong, I assume the question author would, in many cases, be the least equipped to understand whether the auto-generated answer is correct – and whether it's recommending something dangerous/destructive (see Zoe's example). Subject matter experts who are posting answers and reviewing them might be able to judge the quality of the answer, but the question author usually doesn't know what the right answer is – that's usually why they're asking it in the first place.
    – V2Blast StaffMod
    Dec 5, 2022 at 15:54
  • 8
    No, @mattmc. This just add the crappy generated answers in a even more official format, without any human oversight. Also, who's gonna pay for that?
    – Cerbrus
    Dec 5, 2022 at 23:17
  • 5
    @Sgdva I have no idea what you're trying to say. We don't need to wait for a future AI, we already have tons of plagiarism detection software, and it exists as numerous SaaS systems. SO hasn't integrated it because that's how they roll. If you're proposing implementing CGPT, my resistance to that remains that, unlike plagiarism checking, a CGPT can go unfiltered to a user who assumes the answer is good without anything dangerous in it, while a plagiarism detection bot, realistically, is half automatic and half manual review, and neither of those risks giving users dangerous code
    – Zoe Mod
    Dec 6, 2022 at 15:50
  • 8
    Already now, people believe anything GPT says is gold, true, and 100% correct. As has been demonstrated countless times, that's wrong. A direct CGPT integration offering answers increases the risk it's perceived as accurate, and yes, at potential risk to unsuspecting users. Additionally, since CGPT exists, why would you need an SO integration? If it's so good, just go directly to CGPT and ask it rather than taking the roundabout way of going via Stack Overflow, when SO doesn't own nor develop OpenAI nor CGPT.
    – Zoe Mod
    Dec 6, 2022 at 15:52
  • 14
    Guys, as much as you disagree with this answer, that's not a reason to delete-vote it! Don't abuse your privileges like that.
    – Cerbrus
    Dec 8, 2022 at 8:42
  • 4
    Other commenters have covered the (likely) case that the AI is wrong, but this would also be harmful if the AI manages to get the answer right. The goal of Stack Overflow is to have questions and answers that are useful to everyone, not just the question asker. If the AI manages to get the answer right, this feature might discourage people from asking questions that will be useful to others. Dec 10, 2022 at 18:05
-71

I get the point, but if you'll allow a lurker's five cents: I believe that ChatGPT has more to contribute to the platform than to hinder it. How about implementing the bot natively on the platform? Let it answer the questions and, if you want, put an alert saying "this is an automatic response and may contain errors". ChatGPT is helping me a lot, it's fast and practical. It may (yet) not be the right one, but it's enough to help get to the answer.

14
  • 13
    We've been over this twice already.
    – Zoe Mod
    Dec 5, 2022 at 12:03
  • 37
    You clearly didn't read the other answers here: ChatGPT writes bad answers, contradicts itself in the answers, and is extremely costly to implement on a scale SE would require.
    – Cerbrus
    Dec 5, 2022 at 12:03
  • 1
    “How about implementing the bot natively on the platform?” - No; These CGPT answers are absolutely horrible and useless. Dec 5, 2022 at 12:11
  • 4
    Aggressive responses are one of the things that discourage people from posting here, another advantage of ChatGPT. By the way, have you tried to tell him that the answer is wrong or bad? He usually fixes it. I won't insist, it's just my opinion. :) Dec 5, 2022 at 12:11
  • 29
    [1/2] The problem isn't ChatGPT itself. Feel absolutely free to use it to solve your own problems. You may even use it during your research for writing an answer here. The real problem are users who copy-paste ChatGPT answer to SO without even checking if they are correct at a high rate. We had a user yesterday who posted 20 answers in a little bit over an hour, were at least a third of the answers didn't even match the programming language of the question or were outright wrong.
    – BDL
    Dec 5, 2022 at 12:14
  • 7
    And who is going to tell the bot the answer is bad, if it's just automatically showing the author of the question (that doesn't know the answer) whatever it generated? Who's to stop the bot from giving a incorrect, or even dangerous answer? And who on earth is gonna pay for the bot?
    – Cerbrus
    Dec 5, 2022 at 12:15
  • 11
    [2/2] Unless you find a way that the person who copies the answer to SO makes sure that it's a good answer and tells the bot when he is wrong, this isn't going to scale. You can't rely on volunteers here to vote on these answers to get the signal. That's not going to scale on the size of SO.
    – BDL
    Dec 5, 2022 at 12:18
  • 3
    “ChatGPT is helping me a lot, it's fast and practical.” - But the user’s based on output generated by CGPT are absolute trash answers. Low quality answers generated by CGPT are beyond unhelpful. Feel free to use it, just Don’t Post its Output, and experienced users in the community can tell when an answer is based on useless CGPT output. Dec 5, 2022 at 19:56
  • 1
    If ChatGPT helps you, there's nothing preventing you from using it. But in doing so, you're fully aware that the answers you're getting are coming from ChatGPT, and you probably know enough to at least take them with a grain of salt. The issue for SO is people expecting relatively high quality, moderated SO answers could be getting low-quality ChatGPT answers, usually without knowing it, and that's not a benefit to anybody.
    – Caleb
    Dec 7, 2022 at 4:23
  • It's funny, but I think this is involuntarily giving the right answer. Sure, it's pricey and won't be added, but chatGPT answer as first answer would not be the answer, but a reference for any other answer to say: hey, this is an AI answer, if you're answering with this, you're an AI, and your answer will go straight to moderation (or, deleted) - (WOW, I've used the word "answer" more than anyone else here :D)
    – nnsense
    Mar 9 at 21:14
  • @nnsense "a reference for any other answer to say: hey, this is an AI answer, if you're answering with this, you're an AI" why do you think there is the AI answer here? ChatGPT can generate different answers based on how you've asked and/or based on your existing chat history in the session. Each can claim either A or B if there are two options available. It's not like any and all ChatGPT answers always choose A, for example. Yet again, the only thing ChatGPT does is generate plausible text. It doesn't take decisions on questions.
    – VLAZ
    Mar 10 at 2:04
  • The topic here is: chatGPT answers are banned. Fine, so you need something to understand that an answer is indeed a taken from chatGPT, the only way I see is to have a reference, to compare. ChatGPT answers are more or less similar when the question is exactly the same, if someone used it just as reference to answer it's fine, the point is to avoid those which are copy/pasting from it. Can't think of any other way, filtering out chatGPT answers can't be just "guessed", that would create a lot of false positive.
    – nnsense
    Mar 10 at 19:13
  • The methods in use to detect chatgpt answers isn't producing a lot of false poisitves.
    – Kevin B
    Mar 10 at 19:16
  • @BDL With respect to "You may even use it during your research for writing an answer here." - the replies & comments I got to meta.stackoverflow.com/questions/425211/… seemed to imply otherwise. Doing exactly that is what made moderators delete the original answer I had posted here stackoverflow.com/questions/48119360/…. Shows rules are really not clear... Jun 17 at 12:30
1
2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .