114

Following up from the bustling discussion as a consequence of the temporary ChatGPT ban, I guess the big gaping question is how can one determine if an answer used ChatGPT?

As an example, @AKX suggests that there are some tells: answers that start with "It looks like the issue...", or "To fix this...", or those that ends with "I hope this helps" are heuristics.

But we can't confirm that the answer is indeed AI generated, as many legit SO users can and do begin their answers like this.

Another suggestion by @drewcassidy is that we can use an output detector like https://huggingface.co/openai-detector/ - using the model against itself - to determine if the answer is real or fake.

42
  • 4
    In addition these phrases are pretty common in use, it's not like these are strong indicators. You might supposed to make an account and check it. No seriously, don't have a clue how this rule is supposed to be enforced. Commented Dec 6, 2022 at 9:26
  • 10
    @snakecharmerb your skipping the point of the whole question. How do we gather evidence in the first place? Or should we just flag every answer that does not look human ? Commented Dec 6, 2022 at 9:29
  • 3
    I don't really think you can do it based on rules, it'll involve human intuition. Judge the overall style of the post. No spelling mistakes will probably be a big tell :)
    – Gimby
    Commented Dec 6, 2022 at 9:29
  • 11
    Do we actually need a 100% reliable discriminator? If a user manually posts content indistinguishable from a mindless bot without subject matter expertise, misidentifying them as doing it with less effort doesn't seem that harmful. Commented Dec 6, 2022 at 9:31
  • 5
    You can usually tell from a user's history on the site whether they are likely to have produced an AI-like answer by themselves. Commented Dec 6, 2022 at 9:42
  • 11
    I also use the users history as a indicator. A new user posting 5 answers with just a little more than 3 minutes inbetween is a good hint. If you then find an answer that contradicts itself (last example: your problem is that the object is not extendable, solution: use object.Freeze to make the object not extendable), then you have a very good idea of what is happening.
    – BDL
    Commented Dec 6, 2022 at 9:55
  • 5
    @Barkermn01 lol i love that the user is called GPT-3 😭
    – hongsy
    Commented Dec 6, 2022 at 10:37
  • 4
    @Sgdva If you can’t tell whether it was tested and actually solves the problem, do you care? Commented Dec 6, 2022 at 15:46
  • 5
    @Sgdva, IMO if we allow these bot's at all they will start spamming the site, how long is it till some one makes a bot that starts asking questions it can't find and bots answering it, that's a recipe for destroying the site.
    – Barkermn01
    Commented Dec 6, 2022 at 17:27
  • 2
    @Barkermn01 It's the same argument "if we allow people to google, how are we going to make sure their facts are right?" crude answer, we don't: we need to be analytical. Going a little off topic: if you want to think that drinking chlorine will cure you from COVID because you googled it or a pseudo doctor told you so (I know for a fact people with credentials suggested to do so), it's the same as thinking that the bot will replace your work. These are tools, users are the ones to be held responsable, we cannot forbid knives just because you can kill people with them.
    – Sgdva
    Commented Dec 6, 2022 at 18:31
  • 7
    @Thingamabobs: Being a native speaker is not a guarantee. They still need to actively learn to write. 52% of Americans falls **** below **** literacy level 3. "Level 3 is considered a suitable minimum for coping with the demands of everyday life in a complex advanced society." Commented Dec 6, 2022 at 23:56
  • 10
    @Sklivvz gnerally, ChatGPT answers do not look like junk, they look like great answers despite the fact that they are junk. not deleting them damages the site.
    – Kevin B
    Commented Dec 7, 2022 at 21:55
  • 7
    In a fit of irony, I posted this question into ChatGPT and it returned an "internal server error" Commented Dec 8, 2022 at 19:20
  • 5
    I hope the solution is not that we try to start writing really bad English and cross our fingers that AI's down come down to our level. Commented Dec 11, 2022 at 17:51
  • 3
    @JohnFire because for now, due to how often they don't answer the question, it is banned. People are free to use it themselves, just not to try "helping" others by copy pasting it here. If people want to use chatgpt to get an answer, they can.
    – Kevin B
    Commented Jan 25, 2023 at 20:27

9 Answers 9

118

The general question of how to detect these posts is something we've been avoiding publicly answering, because publicly describing how we detect ChatGPT generated content provides those people using ChatGPT with specific information which they could use to make it more difficult for us to detect ChatGPT generated content. So, while I/we understand your desire to know, this isn't something we're officially answering publicly at this time.

For moderators, there is a significant amount of information available about detection of these posts in the Moderator Team and other locations mentioned in there. Moderators can also get information through the Teachers' Lounge chat room.

11
  • So, the answer is flagged instantly, I mean will OP know in time not to use the answer?
    – Ed_Gravy
    Commented Dec 6, 2022 at 16:00
  • 8
    At this time, there can be significant delay between flagging and a moderator taking a look at it. While we're trying to stay on top of these, there are literally thousands of such posts, and that's over and above all the other stuff we normally handle. We all are, of course, volunteers, so how long it will take to get to each flag is indeterminate. So, overall, the answer to your question is that it's likely the OP, who gets directly notified of any new answer (and who is often specifically watching for answers to their question), will see the answer prior to a mod looking at it.
    – Makyen Mod
    Commented Dec 6, 2022 at 16:27
  • 2
    Well, modGPT turns out to be a dead end... (pasted from chatgpt) "It is not possible for me to determine whether a given programming solution was produced by me or not. This is because chatgpt is a large language model trained by....", etc.
    – danh
    Commented Dec 10, 2022 at 16:16
  • 1
    "...there is a significant amount of information available about detection of these posts..." Is there? Without details it's difficult to judge how high the success rate of detection really is. Could be anything really. Commented Dec 11, 2022 at 17:57
  • 5
    @Trilarion The "success rate" of detection is something that is really only possible to guess at in aggregate. In order to know the success rate, you'd have to identify everything that was posted that originated from ChatGPT, even the things which were not detected as such, which is, obviously, not really possible, if for no other reason than it's, at best, difficult to prove that you've found everything. But, yes, members of the public not being able to make such judgements about the efficacy of detection is one of the trade-offs of not making all information public, at this time.
    – Makyen Mod
    Commented Dec 11, 2022 at 18:10
  • 2
    "yeah we can't really detect ChatGPT with any sort of accuracy but we don't want to admit it" Commented Jan 10, 2023 at 23:38
  • 18
    @ScottishTapWater Your statement is completely unjustified based on the available public and private information. It's just slinging mud around. If you take the time to identify a notable quantity of A) posts which are ChatGPT generated that have not been identified, and/or B) posts which have been identified as generated by ChatGPT, but were not actually generated by ChatGPT, then the first part of your statement might have some basis. Currently, it's entirely baseless. As to "but we don't want to admit it", that's just presuming motivation based on your own, erroneous, assumptions.
    – Makyen Mod
    Commented Jan 11, 2023 at 19:37
  • 4
    While there have been some of (B), it's a low number and those are typically reevaluated when the author indicates an objection, either by flagging their deleted post or responding to the moderator message. As to (A): of course detection methods are both imperfect and continuously improving. So, of course, there will be some ChatGPT generated posts which are not, yet, identified. So, if you can identify a significant quantity (proportional to the thousands of posts which have been identified as ChatGPT and deleted), then the first part of your statement might have some basis.
    – Makyen Mod
    Commented Jan 11, 2023 at 19:37
  • 2
    An interesting metric, which you could make public without revealing anything, would be how many answers written before ChatGPT (e.g. before 2021 to be sure) would get flagged as written by ChatGPT.
    – JiK
    Commented May 5, 2023 at 8:40
  • @JiK While there are semi-automated processes bringing some posts to our attention, actual flagging is done by humans and the choice to act is done by moderators (most of whom at least claim to be human :) ). Thus, to actually get such statistics would be to perform a double-blind study involving human flaggers and moderators. That would be quite time intensive, so is unlikely to be something that happens. An additional complication is that it would require using the people who are actually flagging and handling the flags, as the experience of the people makes a significant difference.
    – Makyen Mod
    Commented May 5, 2023 at 17:28
  • 2
    More details about how moderators determined if content was AI generated and a short discussion of false positives in What has happened to lead moderators to consider striking? by Machavity. Commented Jun 4, 2023 at 21:40
32

I don't think there's going to be any value in having you, a mere mortal, try to determine if an answer or question was generated by ChatGPT.

The reason is super, super simple: we don't want this to turn into a witch hunt.

There are things you can do even today to help with curation of answers; if an answer is wrong or incomplete, you can downvote it. If you happen to notice a trend of similarly bad answers from an account, you can flag for moderator attention with the series of answers you've got as evidence.

This shouldn't be a "those bad answers are more moderatable than others" situation - just vote on answers as you always would, and let the moderators deal with the heavier lifting on this circumstance.

10
  • 15
    Makoto makes a valuable point here that I think hasn't surfaced quite enough in discussions like this: witch hunts on the internet are far too common and (especially with anything that requires a subjective opinion) the results are very rarely correct. I'm glad to see this come into the discussiion.
    – Philippe StaffMod
    Commented Dec 11, 2022 at 17:19
  • I second that, and also, we should focus on what the real issue is exactly, is it that we don't like the Idea of people abusing the system and using chatGPT to effortlessly answer questions and get upvotes or is it about the value and correctness of answers. And what if chatGPT is able to give valuable and useful answers that respects stackoverflow's guidelines what do you do about that? or is someone was to copy/paste an answer from another forum to SO, would that also be a problem ?
    – Xsmael
    Commented Jan 5, 2023 at 14:09
  • 21
    To be very clear about something: The massive effort and hours that the community curators/flaggers and elected moderators spent together over the last 6 months very successfully stemming the tide of the ChatGPT copy-paste flood and shielding our community’s users from having to see all that junk never in any way shape or form came anywhere close to actually resembling anything like a “witch hunt”. We had been very accurately identifying the ChatGPT posts with an extremely low number of false positives. And I say that as someone who handled a very very large number of the flags.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:17
  • 12
    Here are some personal numbers: Conservatively speaking, for the last 6 months (since the beginning of December 2022) every single day I have been spending 40 minutes a day just on moderating ChatGPT flags. I think that works out to 120 hours in total that I have personally spent on them so far. And in those hours week after week, I have looked at multiple thousands of ChatGPT-flagged posts: I think it must be on the order of 10000 or even 15000 at this point.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:27
  • 12
    I do assert that in the many hours I have spent actually looking at the ChatGPT-flagged posts, and from the many thousands of those posts that I have actually closely and carefully scrutinized in detail, I have in fact learned to recognize the ChatGPT copy-paste cases (including many of the ones whose contest the posters had edited before pasting in, to intentionally obscure the provenance) and I can in fact with a very high level of confidence and a very very low level of false positives effectively identify answers whose provenance is ChatGPT or other AI tools.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:33
  • 10
    To anyone who claims the elected moderators have been getting it wrong for the last 6 months, with an unacceptably high rate of false positives, and who wants to challenge my personal assertions about how accurately I can identify the ChatGPT cases: I invite you to work together with me over several days looking at least some several dozens of ChatGPT flags as they come in, and scrutinizing together the content of the flagged posts together, and actually honestly talking together in good faith about which posts we think we can agree are highly likely not the poster’s own original work.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:41
  • 8
    Alternatively, rather than looking at new flags as they come in, I invite anyone who wants to challenge my personal assertions about how accurately I can identify the ChatGPT cases to go together through the logs of my moderation activities, and take some significantly-sized random sample at least of the thousands of ChatGPT flags I have handled, and scrutinizing together the content of the flagged posts together, and actually honestly talking together in good faith about which posts we think we can agree are highly likely not the poster’s own original work.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:44
  • 8
    Until anybody who questions how well we have been handling the ChatGPT flags actually does something concrete like I have described in my other comments, waving vague suggestions about possible “witch hunts” under the noses of the elected moderators and community curators/flaggers is completely irresponsible at best — but in reality in combination with the other indirect accusations and aggressions that the key company reps driving the communication around this have slung at the elected moderators, it is completely unconscionable and completely unprincipled.
    – sideshowbarker Mod
    Commented Jun 4, 2023 at 23:52
  • 3
    @sideshowbarker: Don't think that this was written as a slight to the work of moderators or people who actually are taking the time to do this the right way. This was mostly to stop people from brigading together, thinking that they were doing something constructive rather than not. Rest assured, I value and appreciate all the things that the moderation team have done to this point to stop the excessive flood of ChatGPT-generated content to come on this site.
    – Makoto
    Commented Jun 5, 2023 at 3:33
  • 3
    @Makoto Yes, I understand, and I never thought otherwise of you personally. And to be very clear: My comments here weren’t directed at all to you. And I’ll also take the opportunity now to tell you: While I don’t follow the meta sites closely or read nearly as much of them as I should, I’m an avid follower of your meta postings. To cite just one example among many: At the time when I was trying to understand the Collectives feature, I read (and re-read) all your meta postings about it, and they were a huge help to me in making an evaluation of Collectives that I had needed to make at the time.
    – sideshowbarker Mod
    Commented Jun 5, 2023 at 7:19
17

how can one determine if an answer used ChatGPT?

Answers by ChatGPT will likely have perfect grammar and punctuation (unless this bot learns to dumb itself down in that regard), a certain length maybe (3-4 paragraphs, typically no links nor code/emphasis formatting but a certain fondness for bullets/numbered lists) and most importantly be kind of useless, missing the point or stopping short of the solution. Unfortunately detecting that would require domain knowledge. Fortunately we have this knowledge but we have only limited time.

Therefore it seems apt to use AI in order to detect AI. Kind of what a GAN does. ChatGPT is the generator and what we would need to build would be a discriminator and then see where that leads us.

In the end, nobody knows if you are a dog on the Internet. I could be one (or an AI).

The really crucial thing is being an expert. That is much harder to fake. People will realize at some point if I just sound well or if there is real substance behind my words. At least for those topics with definite answers like programming.

An important thing would also be to assess the accuracy (false positives/negatives) of such a detection. I guess if one takes all posts before December 2022 as negatives and all self created answers by ChatGPT as positives, it would be possible to estimate these error rates and judge how well that detection system can work. That's really important to know. See for example Why was my answer deleted for using ChatGPT even when I didn't? for what can happen if you start your answer with "Here's an example of how you could use.." among other things.

Finally, we could also include a stricter trust model to recognize experts in their fields. In the past, we mostly emphasized that the content should stand on its own, but exploiting the link of every answer to a user account could result in additional information about whom to trust and whom not to trust.


P.S.: As Cerbrus and ConceptRat mention in another question, a bit of help from OpenAI could make the detection much easier, i.e. if they would have an API that lets you check if a certain part of content was generated by them lately. That might alleviate the problem greatly (unless people start transforming the output before posting here).


P.S.2: As a corollary to the difficulty it seems to make to determine if some content was AI generated, it's now also difficult to determine if content was human generated. That's a bit sad, because for example, why bothering typing all that courtesies in emails yourself if the other side will probably just think that an AI created them. That seems to reduce the value of human generated content. That's really one of the major problems I have with AI right now (much more than the allegation of AI being cold and calculating, humans are that too quite often). I don't want to be seen simply as a provider of training material for an AI. But I'm digressing somewhat.


P.S.3: Fast forward to May 2024 after a moderator strike in June 2023 on the issue and a large backlog of AI generated content flags that seems to be hard to catch up, a user was reported to have generated over a thousand posts relying on AI generated content over a year without detection or handling of the detections. This surely highlights how difficult it is to differentiate between AI and humans. The Touring test has largely been passed. That may also promote the use or help of AI generated content, which may slowly become more and more accepted. Kind of: if we cannot prevent it effectively and battling it is too much work, we can as well embrace it.

6
  • 5
    Many thanks to Peter Mortensen for providing context and improvements. (constantly and over many years) Commented Dec 11, 2022 at 21:13
  • 1
    "if they would have an API that lets you check if a certain part of content was generated by them lately." The problem is that they don't store output. When the session closes, the data is gone.
    – Cerbrus
    Commented Dec 12, 2022 at 10:26
  • /me starts working on script to add grammatical errors and mess up code a little so it looks human generated :)
    – Yepher
    Commented Dec 12, 2022 at 17:42
  • 2
    In regards to "typically no links or formatting", the AI does seem fond of bullet/numbered lists, though.
    – Cerbrus
    Commented Dec 14, 2022 at 9:02
  • I don't think this -> `Answers by ChatGPT will likely have perfect grammar and punctuation is right, for me I use Grammarly to avoid such things and correct any mistakes but the post/answer is still written by me @Trilarion
    – Ahmad
    Commented Apr 3, 2023 at 11:08
  • 1
    @Ahmad I see it from a statistical point of view. It doesn't really matter then what a single individual does. I only speak about the average user. Commented Apr 3, 2023 at 16:01
11

I have flagged several suspicious answers with in need of moderator intervention and a comment about answer contents and user behavior (usually there are a lot of low quality answers in succession in different topics) but probably it would be great to add a separate flag/closure reason/deletion reason specifically for this use case.

P.S.

Also it seems there are some AI-generated questions =)

4
  • 19
    AI generated questions? If it means no typos and proper markdown, I'm all for it
    – Phil
    Commented Dec 8, 2022 at 0:58
  • 1
    "I have flagged several suspicious answers..." To be an answer to the question here you would probably have to say a few words about what raised your suspicion in these cases. Otherwise, how does it answer the question? Commented Dec 11, 2022 at 17:52
  • 1
    @phil: Uh, no. Can't use AI for answers? Can't use it for questions either. Commented Dec 12, 2022 at 12:50
  • 4
    @RobertHarvey was mostly tongue in cheek but perhaps it could be useful if the prompt was something like "reformat this question to be suitable for Stack Overflow"
    – Phil
    Commented Dec 12, 2022 at 18:01
9

It's easy to detect ChatGPT answers. Just look at 4 or 5 ChatGPT answers and you'll notice them almost immediately thereafter using your newly-trained basic human pattern matching abilities.

Let's practice! Here's a thread: What is the best practice solution to embed videos from different sources in a ReactNative app? [screenshot]

To make it more interesting, here's (only!) the first paragraph of each of the 6 answers, verbatim:

To achieve the requirements you listed, I would suggest using the expo-video library instead of WebView. Here's a sample code snippet to get you started:

To achieve the requirements you listed, I would suggest using a third-party library for video playback instead of relying solely on WebView or Expo's built-in Video component.

There are a few different approaches you can take to embed videos from different sources in a ReactNative app, depending on your specific requirements and use case. Here are a few best practice solutions:

There are a few challenges that need to be addressed in order to implement a seamless video experience that meets your requirements. Some of these challenges include handling device orientation changes, ensuring consistent playback across multiple video sources, and implementing a player that works well on both Android and iOS.

You can use Amazon IVS Player for react native: [https://github.com/aws/amazon-ivs-react-native-player][1]. Its a very minimalist video player out of the box, so all the basic functionality you require would have to be implemented by the you, like rotation, autoplay, pause/continue etc. But on flip side, it can be programmed to do anything you need it to do. Plus its not a webview.

If you want a smooth video playback that can handle rotation and multiple sources, go for the react-native-video library instead of using WebView. Here's a quick example of how to implement it:

Now test yourself: which are AI and which are human? If you're not sure based on first paragraphs, go into easy mode and view the thread or screenshot above for full context.

Great job! 5 of the answers were ChatGPT and one was human. You're ready to start flagging ChatGPT posts! (but wait, you're on strike, right?)

See this meta SE post for more threads you can practice on.

4
  • 2
    Perfect spelling isn't an accurate indication, though... My spellign is awlays prefect.]
    – Cerbrus
    Commented Jun 21, 2023 at 18:57
  • No, I just used that as an easy means of identification rather than detection (although spelling can be part of the detection procedure--there's no need to itemize all of the parameters since they're intuitive, and other answers attempt that). Check out the full thread if you're unable to detect based on this and I can assure you, it'll be trivial. If you imagine the thread without the answers deleted, that's what the site will look like if SO keeps blocking mods.
    – ggorlen
    Commented Jun 21, 2023 at 18:58
  • 1
    Nooooo you can't just use your human intuition. You must be using some sort of tool, and tools (apart from ChatGPT of course) have flaws. Stop doing this. /s
    – CodeCaster
    Commented Jun 21, 2023 at 19:04
  • 8
    the comical part is the OP begging not to use ChatGPT :)
    – pierpy
    Commented Jun 21, 2023 at 19:13
4

There are a few tell-tale signs that can provide a general indicator that answers and accompanying example code were pasted from ChatGPT.

Once you've seen them, it's hard not to spot them when they pop up in the wild. And while not a guarantee that the answer is AI-written, they are strong indicators that are often worth further investigation.

As for code correctness, if perfect-looking example code has a glaring error typically only caught by intellisense, something's probably wrong. ChatGPT tends to prefer explicitly naming variable types over inference, so it's rare (in my experience) to see var foo = something() (C#) or auto bar = another() (C++).

If reviewing an answer to determine if it is human or not, it's probably worth your time to paste it into an IDE, if you don't already.

Presumptive Type Classification

Many property classes in C# provide a getter function, named the same as the class name, prefixed with Get. Likewise for methods returning a collection, GetAll. This is usually by design, but it is not always the case, especially when dealing with code introduced very early on before adopting this naming convention.

using System.Net.NetworkInformation;

IPGlobalProperties properties = IPGlobalProperties.GetIPGlobalProperties(); // truth
//...
NetworkInterface[] interfaces = NetworkInterface.GetAllNetworkInterfaces(); // truth
foreach (var ni in interfaces) {
  IPProperties ipProperties = ni.GetIPProperties(); // fiction
  //...
}

Just because the GetIPGlobalProperties() and GetAllNetworkInterfaces() methods use their class names verbatim, does not guarantee that a GetIPProperties() method is part of the IPProperties class. Indeed, such a class does not exist in the System.Net.NetworkInformation namespace. GetIPProperties() is actually part of the IPInterfaceProperties class.

Similarly, the existence of an enum specifically describing common interface types (NetworkInterfaceType), does not mean every possible connection type in existence is included.

using System.Net.NetworkInformation;

if (ni.NetworkInterfaceType == NetworkInterfaceType.Ethernet // truth
  || ni.NetworkInterfaceType == NetworkInterfaceType.Wireless80211 // truth
  || ni.NetworkInterfaceType == NetworkInterfaceType.Bridge) // fiction
{
  //...
}

Both of the above errors are easily apparent when pasted into an IDE, but ultimately would go unnoticed except to the most astute programmer with significant first-hand experience using these specific .NET classes, as the following unaltered snippet from a ChatGPT response shows:

AI-generated code excerpt with errors

Symmetric Sentences

Example code is often introduced and summarized by a single sentence, using a consistently recognizable sentence structure with very little - if any - deviation from the following:

Standalone introduction sentences (meaning a single-sentence paragraph) begin with:

  • "To get/display ..."
  • "For example, ..."

Multi-sentence introductory paragraphs typically use the modified form "To do this, ..." in the last sentence of the paragraph.

Summary sentences will begin with one of the following, with very few exceptions:

  • "This will display/return ..."
  • "In this example, ..."

These tend to be paired, meaning an example introduced with "For example..." will invariably be followed by a summary beginning with "In this example...". Introductions beginning with "To display..." are followed by summaries beginning with "This will display...".

7
  • 6
    It is mildly distressing that my normal way of writing answers tends to match what you've described here in the "Symmetric Sentences" section. (And my normal answers also often include typos in the code blocks, so maybe I am a bot after all!)
    – Cody Gray Mod
    Commented Jan 7, 2023 at 8:59
  • 1
    Re "ultimately would go unnoticed except to the most astute programmer": I suspect this for many (most?) of the "try this" answers on Stack Overflow (code dumps without any explanation) by humans (copied from somewhere?), especially late answers. Though I could be completely wrong. Commented Jan 7, 2023 at 13:04
  • 1
    @Cody "so maybe I am a bot after all!" - typical bot speak.
    – CodeCaster
    Commented May 24, 2023 at 8:25
  • 1
    While I appreciate your desire to answer the question, keep in mind that the accepted answer is very clear that "publicly describing how we detect ChatGPT generated content provides those people using ChatGPT with specific information which they could use to make it more difficult for us to detect ChatGPT generated content" I, at least, would appreciate it if you would delete your post. I understand that security-through-obscurity isn't a great strategy, but neither is providing information on how to circumvent detection ;-). Commented May 26, 2023 at 16:57
  • Might be worth noting that AI advancements in the weeks following my originally posting this, have long since made these observations obsolete. Commented May 26, 2023 at 19:58
  • it hasn't, though, tbh. it's still fairly easy to recognize gpt generated content just by looking at it.
    – Kevin B
    Commented May 26, 2023 at 20:00
  • @KevinB to the keen reader, yes. But you won't see the patterns I mentioned consistently anymore. More training data = more versatility in writing style = lower concentration of the nuances I mentioned. At this point it generally boils down to being grammatically "too perfect", or (except for cases when generated with a prompted tone and writing style) an absence of self identity in the writing (the "third person" persona - ie. an article written for programmers, explaining programming patterns, while never once including itself as part of that group as a programmer.) Commented May 27, 2023 at 3:19
0

As the state-of-the-art AI model evolves, it may eventually be indistinguishable from human-written text. So instead of judging by arbitrary "noticeable traits", I rely on my "sense of language" to identify answers that don't "feel natural" to me. Then I check the user to see if they have just posted multiple answers in quick succession, plus other heuristics, like being new users or low-rep users. If it's a single occurrence I'd just flag accordingly (usually NAA, sometimes VLQ), and if there are multiple low-quality answers, I double-check and then raise a mod flag explaining my concern.

4
  • The very AI that writes text can be used to determine if it's likely to have written a certain text. Detection isn't a technical problem.
    – Cerbrus
    Commented Dec 16, 2022 at 7:56
  • AFAIK, NAA and VLQ are not the right tools for this. When I suspect an AI-generated answer, I do the same checks you do and then flag them for moderator attention with a message like “Possible ChatGPT-generated answer.”
    – Julia
    Commented Jan 10, 2023 at 23:07
  • @Cerbrus I'm not sure what you mean by that. Are you referring to OpenAI's AI Text Classifier?
    – Ryan M Mod
    Commented May 27, 2023 at 3:28
  • @RyanM more in general. The technology that generates this text is what's used to determine if something is likely to be generated.
    – Cerbrus
    Commented May 27, 2023 at 20:42
-26

We can't, and as such we shouldn't waste our time trying.

ChatGPT is merely the latest and greatest iteration of GPT. As GPT continues to improve, so will the answers it generates, which in turn will make such answers more difficult to detect. The end result is an arms race between the curators of Stack Overflow and the developers of GPT - a race that we the former, with our limited personnel and resources, cannot ever hope to win.

But we curators are smarter than an algorithm that is nowhere near being artificially intelligent. And we can win, but to do so we have to think deeper than merely curating answers, and instead turn to prevention. Or in short, move the goalposts by ensuring that all users on Stack Overflow are real human beings, not bots.

This may sounds ridiculous and unworkable but it truly is the only way to have some sort of protection from more and more sophisticated bots based on more and more sophisticated language models. As more and more of these bots flood the Internet and begin to crowd out human-generated content with computer-generated noise, the only way for human communities where accuracy is paramount (such as Stack Overflow) to continue to function will be via identity verification.

For Stack Overflow, this sort of verification is unlikely to cause much turmoil, as many of the most trusted users already use their real names for posting the content they produce. Whether Stack Exchange Inc. is forward-thinking enough to care is the most important concern.

Of course, identity verification doesn't preclude verified users from using GPT to generate answers, but simple numbers dictate that the numbers of such users will be low, and as such ordinary curation efforts should be able to manage the detection and removal of such content and users.

12
  • 1
    "We can't" That's simply not true. The very tool used to generate text can be used to detect how likely it is the text is AI-generated. There are plenty of services out there that can. Even if all users are humans, that doesn't stop them from using some AI to generate content. The whole problem here is human users using AI-generated text.
    – Cerbrus
    Commented Dec 14, 2022 at 8:29
  • 1
    "simple numbers dictate that the numbers of such users will be low, and as such ordinary curation efforts should be able to manage the detection and removal of such content and users." That's the entire problem here. The number of users isn't low. The content they post is creating significant work for moderators. You're completely missing the point/reason of this ChatGPT blanket ban.
    – Cerbrus
    Commented Dec 14, 2022 at 8:30
  • @Cerbrus How many of those users are new?
    – Ian Kemp
    Commented Dec 14, 2022 at 9:48
  • Why does that matter?
    – Cerbrus
    Commented Dec 14, 2022 at 11:12
  • 1
    Because new users who intend to sign up only for the purpose of spamming, are unlikely to want (or be able) to go through the effort of proving their identity. So they won't sign up at all - leading to less spam, ChatGPT or other.
    – Ian Kemp
    Commented Dec 14, 2022 at 12:14
  • 3
    And existing users will leave the platform en-masse, because plenty of users don't trust SO with their data. This has been an issue in the past with something as comparatively trivial as usernames.
    – Cerbrus
    Commented Dec 14, 2022 at 12:16
  • So the worst case of trying to fight the low-quality answers is we end up with high-quality answers? xkcd.com/810 Commented Dec 15, 2022 at 23:07
  • 2
    @JohnMontgomery If you believe that a machine learning model with zero comprehension of what abstract concepts like "programming" are or entail, can ever provide high-quality answers about such abstract concepts except by accident, then I have nothing to say to you.
    – Ian Kemp
    Commented Dec 15, 2022 at 23:24
  • @IanKemp Your first paragraph seemed to imply that you think it could, or else why wouldn't we able to detect it? Commented Dec 15, 2022 at 23:41
  • 5
    @JohnMontgomery GPT will generate more and more answers that appear correct, but won't actually be correct. As its sophistication increases the proportion of each answer that is incorrect will continue to shrink, but that doesn't matter: one syntax error is the same to a compiler as one thousand. However, as each answer becomes proportionally more correct, it also becomes proportionally more difficult for a human to detect the fundamental incorrectness that invalidates said answer in its entirety.
    – Ian Kemp
    Commented Dec 15, 2022 at 23:47
  • 2
    This is uninformed. OpenAI themselves have released tools to detect if something was generated from ChatGPT or not. They have no interested in disguising the output from the bot as something else, quite the contrary. So detecting it is rather a race between OpenAI and OpenAI... and in that race I'm fairly certain that OpenAI can hope to win. SO just have to use their tool intended for this purpose.
    – Lundin
    Commented Feb 2, 2023 at 11:51
  • Re "turn to prevention": Yes, after the fact is just band-aid. But it doesn't have to be identity. As most of the post spammers are trying to do the least amount of work (which is why ChatGPT has replaced regular plagiarism), only a tiny amount of friction needs to be added to prevent most such posts. I don't know what form it would take; the most obvious ones are probably easy to circumvent. Commented Jun 6, 2023 at 12:30
-38

You can detect the explanations, but not the code because the code could come from anywhere or ChatGPT might just be reading content from a programming language's official site.

Why would you ban code generated by ChatGPT when other users will post the same exact code from either Python's official documentation site, Java, or C#? So just let it go; don't even try to detect as some other moderators advised. Detect the code explanations or code that does not compile posted from ChatGPT, otherwise just stand down.

5
  • 4
    ChatGPT doesn't answer with only code. This whole ban only really affect those that blindly copy-paste the output, any way, so your whole "don't detect the code" is completely irrelevant.
    – Cerbrus
    Commented Jan 8, 2023 at 14:27
  • You guys will struggle and won't be able to completely quarantine SO from GPT code and content
    – user16612111
    Commented Jan 8, 2023 at 15:40
  • 11
    ChatGPT doesn't copy content, including code. It generates content using a large language model. While it's possible the generated code is the same as some code in the material used to train ChatGPT, it's unlikely that someone will be able to find exactly the same code from somewhere else. If ChatGPT copied code from somewhere, the code would probably be better, because the copied code would likely have been tested, rather than generated by a model that just puts "words" together in ways it thinks "fit", based on its model with no actual understanding nor functional checking.
    – Makyen Mod
    Commented Jan 8, 2023 at 18:06
  • 7
    "You guys will struggle and won't be able to completely quarantine SO from GPT code and content" These blanket statements are just nonsense... Of course some posts will slip through. This whole rule exists so mods have somewhere to point users they're suspending for ChatGPT use. No arguments, no ifs or buts.
    – Cerbrus
    Commented Jan 9, 2023 at 8:28
  • 6
    "You can detect the explanations, but not the code", well, this not true (at least in the small Tag I follow/answer ('iMacros' for Web-Automation))... // iMacros Code generated by 'ChatGPT' is hilarious and fairly easy to spot because 'ChatGPT' creates Commands that simply don't exist in that Prog-Language by borrowing Commands/Methods from "popular" other Languages (usually JS/PHP/C++) and trying to apply them in iMacros Scripts with some very approx Syntax, ... that of course doesn't even compile. And such Code is very easy to detect for any Advanced User in that Language.
    – chivracq
    Commented May 24, 2023 at 13:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .