50

Question(s) first: how should situations like these be dealt with? How does a certain stance affect the community? How does it affect the corporation's ability to make money?

I've recently encountered a user who clearly uses GenAI, probably straight-up ChatGPT, to author answers and even comments. The user admits this, when asked, but there are no disclaimers anywhere.

One example of a comment (the subject was image processing):

The context of the images to be processed is crucial in determining the best technical approach to adopt.

(which was posted after a regular gave a to-the-point comment that basically solved the issue, unless OP had some undisclosed constraints to add)

The user elaborates when their use of GenAI questioned:

[...] These models serve not only to translate between languages but also as a compensatory tool in cases of language disorders. [...]

The writing style is that of ChatGPT: low on content/meaning/substance, high on pomp and truisms, sounding authoritative, passive voice, impersonal, abstract rather than concrete, prone to repeating parts of the prompt. The kind that pings techies' bullshit meters, which have been finely calibrated on GenAI content over the past year or so.

Further, I think this user's pre-ChatGPT writing was just fine, albeit strongly reading like English-as-a-second-language (ESL), with strong signs of the native language shining through. Maybe it's dyslexia or a manual handicap, but I don't think so. Supposing there is a disorder of any kind, and supposing it warranted compensation by technical means, I don't think it warrants subjecting us to current-generation GenAI. At the very least, the model needs to be beaten into shape so it stops acting like... it does.

I think SO needs to get ahead of this claim. If it were to be accepted, a whole lot of rep hunters would deploy this claim as an excuse to keep flooding us with GenAI-excreted "content" that we have to sift through and combat.

19
  • 60
    As the current policy states all use of generative AI is banned when posting content on Stack Overflow. Just flag the user's post if you believe they are using generative AI. Commented Mar 24 at 12:04
  • 53
    If I completely lose my vision, that is no excuse to put myself in the passenger seat, and my 3 years old child in the driver's seat, and continue on like before... Commented Mar 24 at 12:04
  • 20
    I'm asking for a discussion because disability claims usually win over TOS, if you make enough of a stink over it. Commented Mar 24 at 12:16
  • 15
    I feel like we actually would need the user to explain how the LLM "compensates for a disability". I admit I lack a disability, but I'm struggling to see how an LLM would provide said compensation, or what it offers them that other tools designed for those with disabilities do not.
    – Thom A
    Commented Mar 24 at 12:58
  • 3
    Is it possible that the GenAI was only used as a translation tool rather than a generator? Or does it not taint translations with its "style"? Commented Mar 24 at 15:55
  • 12
    Sure, disability, but the quality of their contributions will still fall below what’s required since ChatGPT responses fall below the quality threshold. So while they could use it for disability reasons they will quickly be answer banned. I don’t fall for the ESL claims, there are plenty of tools that write grammatical perfect English but at not LLM like ChatGPT. Disability or no disability, if the quality of the contribution doesn’t meet a minimum level of quality, it will be downvoted. I use extensions to correct my own grammar, i have never once been accused of using ChatGPT. Commented Mar 24 at 22:07
  • 1) what does this have to do at all with "the corporation's ability to make money"? 2) what disability did that user say they have?
    – starball
    Commented Mar 24 at 23:09
  • 3
    @AndrewMorton: That's something I could actually get behind if they're willing to post the untranslated and the AI translated text.
    – Joshua
    Commented Mar 24 at 23:45
  • 25
    If ESL is a disability, I'm disabled in a lot of languages!
    – Mentalist
    Commented Mar 25 at 7:41
  • 11
    Lying to farm rep from LLM-generated content is not a disability. Except, perhaps, of the soul.
    – Ian Kemp
    Commented Mar 25 at 9:28
  • 6
    @ChristophRackwitz SE Inc. has made it clear that they don't care about quality, just user and click count. The curators are the ones that pushed for the ban on LLM-generated content, not the company, and moderators have been effectively banned from removing such content.
    – Ian Kemp
    Commented Mar 25 at 10:44
  • 13
    Writing answers with poor grammar/spelling due to dyslexia or poor English skills is acceptable - other human SO users can help edit that into shape. But posting AI-generated content is not. And well, there are AI:ish tools more suitable than ChatGPT, that can correct grammar and spelling without changing the meaning of the text. These tools aren't typically "GenAI" as far as I can tell, but much more basic. It's hard to draw the line on what's AI and what isn't.
    – Lundin
    Commented Mar 25 at 11:45
  • 8
    Next steps: asking a GenAI to generate a doctor prescription for allowing using GenAI for contributing on StackOverflow, and to generate a lawyer defense that this prescription wins over the Terms of Service of StackOverflow.
    – Cœur
    Commented Mar 25 at 12:03
  • 2
    interesting discussion there, @starball. thanks for linking to it. personally, I don't object to use of AI. I object to using it for cheating, spamming, rep farming, ... and annoying the rest of us with its tone of voice, its utter lack of care for honesty and truth. Commented Mar 26 at 8:58
  • 3
    Everyone wants to use A.I. to generate content, yet no one wants to read content generated by A.I. Commented Mar 26 at 10:16

5 Answers 5

92

Answers generated by artificial intelligence tools are not allowed on Stack Overflow. Learn more
-- Banner on the answer writing box

Such usage is already covered by the current policy: It is not allowed.

The entire reason why SO introduced a blanket ban on AI content - not on wrong AI content, or misleading AI content, or this-triggers-my-inner-luddite AI content - is that it is not feasible to differentiate "good" from "bad" AI content.

There are all sorts of pros and cons about using AI as an equalizer - that it enables people to do things, whether they are able to verify what it enables them to do, yadda yadda - but ultimately none of those matter:

SO introduced a blanket ban on AI content. And there is currently no way it could have any other kind of ban.

8
  • 1
    We might conclude that SE needs its own translation service, to resolve the conflict between its "English only" policy and its "no generative AI content" policy.
    – Mentalist
    Commented Mar 25 at 7:50
  • 50
    @Mentalist There is no such conflict. Translation was absolutely possible for years without genAI and it still is. Commented Mar 25 at 8:11
  • 2
    Usually, in real life, you would try to be flexible about what is banned if it can actually help (e.g. dogs are forbidden, but assistance dogs are usually still allowed, unless the company applies policy way too strictly). In this case though... I don't know, without knowing the disability, I don't see how it's relevant.
    – Clockwork
    Commented Mar 25 at 8:33
  • 18
    @Mentalist I don't see how either of those things are linked in any way shape or form.
    – Ian Kemp
    Commented Mar 25 at 9:29
  • 1
    ... unless they are generated with SO AI. Am I the only one seeing the contradiction here? Commented Mar 26 at 10:29
  • 5
    @SilentCloud OverflowAI is subject to the exact same rules as any other AI: content generated by it may not be posted from individual accounts in response to questions. This is not a contradiction.
    – Ryan M Mod
    Commented Mar 26 at 10:37
  • 1
    @IanKemp My assumption is that because translation tools are usually backed by some AI model, they're seeing that as being AI-generated. In my book, that's not AI-generated content, that's AI-translated content. The original content was still generated by a human.
    – Clonkex
    Commented Mar 26 at 23:55
  • @IanKemp Maybe I should have clarified - I mean that people who don't speak English cannot contribute content without either including a lot of mistakes or relying on AI-powered translation tools. Every auto-translated post will be categorized as "gen AI content", right? Then those people have no way to contribute unless SE offers a translation service to act as an exception to the rule that would otherwise ban all AI-translated posts. Does that explanation make my reasoning more clear? (Sorry I didn't notice your comment a week ago.)
    – Mentalist
    Commented Apr 1 at 5:13
51

I do not really consider ESL a disability, but either way, being unable or unwilling to adequately answer questions is generally not considered an excuse to 'cheat,' i.e., plagiarize, from AI or otherwise. This precedent predates AI, so why would AI be different, especially after being explicitly disallowed?

As long as the answers are understandable, and the user knows what they are talking about, there are many other users who are willing to edit their answers to correct English mistakes or make them flow more naturally.

AI on the other hand, does not know what they are talking about. AI will produce confident/sophisticated-seeming answers that are dead wrong.

2
  • 16
    The biggest fault with ChatGPT responses is context, often the LLM will focus on the wrong noun or subject, and the response while perfectly accurate doesn’t actually answer the question that was presented. What ends up happening is a factually correct answer is submitted which doesn’t answer the question, new users of course have a problem understanding the reason a factual accurate answer would be downvoted, without understanding that their answer is not helpful to either the author or the community. Commented Mar 24 at 23:58
  • 1
    True that. As has been said elsewhere, "Context is king" Commented Mar 26 at 1:09
26

While I'd be the first to advocate making reasonable accommodations for a disability, allowing someone to copy and paste answers that are entirely generated by AI isn't a reasonable accommodation. That's effectively not writing answers at all. Unfortunately, if the only accommodation that will allow them to write answers is having AI write the entire thing for them, that means that they're not capable of writing answers at all.

Also, ESL isn't a disability. I speak very mediocre Spanish, but I'm not going to the Spanish-language site and posting AI plagiarism answers based on that.

There are, of course, edge cases in this (for example, using tools like Grammarly). As another example of an edge case, I also know someone who will sometimes write documentation by feeding ChatGPT the exact points he wants to cover, having it generate the paragraphs for him, and then tweaking the paragraphs to his liking. Both of those are arguably reasonable accommodations, but that's not what's going on here: the user in question literally wants to be allowed to copy and paste something written entirely by AI "as is".

12
  • 7
    I used Grammarly daily to generate most content I submit daily on various SE communities. I have compared what ChatGPT generates and what Grammarly through the same detection tools. Grammarly corrected content has never been mistaken for a ChatGPT response. This is of course due to the fact the original content was created by a human. I have said this several times, ChatGPT generated content is it easy to detect, even if a human modifies it. A human can have a writing style but ChatGPT uses a writing template. Commented Mar 25 at 16:30
  • 3
    Totally agree. There are plenty of good reasonable accommodations, and GenAI is probably still in the category of unreasonable ones. Commented Mar 26 at 1:11
  • 1
    @JonathanDavidArndt Very limited use (like I describe above) might be reasonable under certain circumstances, but copying and pasting an entire answer that AI generated seems intrinsically unreasonable to me regardless of how good GenAI is. If the only way that someone can write an answer is to have AI write the whole thing for them, they can't write answers. That's like saying that Real Madrid should hire me with the accommodation that Lionel Messi should play all of my games for me - the fact that he's really good at soccer doesn't change the fact that I can't play at all. Commented Mar 26 at 14:11
  • 3
    @JonathanDavidArndt By analogy, I had an ACL reconstruction on my knee a few years ago, so if I wanted to play for Real Madrid, asking them to give me extra time with the athletic trainers or special knee braces would be reasonable accommodations. However, asking them to have someone else play all of my games for me is intrinsically unreasonable (because if I need that then I'm clearly completely unable to play). They're not under any legal or ethical obligation to be that accommodating - you still need to be able to perform the essential job duties for accommodation requirements to apply. Commented Mar 26 at 14:23
  • @EJoshuaS-StandwithUkraine "If the only way that someone can write an answer is to have AI write the whole thing for them, they can't write answers" Unless the AI is used as a speech-to-text which doesn't seem to be the case there.
    – Clockwork
    Commented Mar 26 at 18:44
  • @SecurityHound "I used Grammarly daily to generate..." "Grammarly corrected content..." So, are you letting grammarly generate or merely correct content? Generating is clearly forbidden. Correction, even though it's got some AI in there, is fine...
    – Cerbrus
    Commented Mar 28 at 8:06
  • @Cerbrus - I obviously mean, I write the content, and then correct any mistakes with Grammarly. My point is nothing Granmarly has ever “generated” as ever been misinterpreted as being generated by a LLM. Commented Mar 28 at 11:34
  • @SecurityHound because Grammarly is not generating. It's applying grammar and spelling rules to what's written and suggests changes. These changes are so minor that they don't really change the input in any significant manner. A LLM generator will generate walls of text from very little input. Grammarly will never output significantly more than the input. Its in-and output are very closely related, contrary to banned LLM output.
    – Cerbrus
    Commented Mar 28 at 12:00
  • So, yea, Grammarly uses some AI to help out, but it really can not be compared with LLMs generating text. They're different tools with completely different operating parameters.
    – Cerbrus
    Commented Mar 28 at 12:01
  • 1
    I clearly used the wrong word. My point proper tools to correct grammar will not be mistaken for LLM responses. Commented Mar 28 at 12:09
  • Naturally, because they don't generate new content :D
    – Cerbrus
    Commented Mar 28 at 13:08
  • @SecurityHound RE Cerbrus replies, your first comment was perfectly fine, I understood exactly what you meant by: Grammarly corrected content has never been mistaken for a ChatGPT response. but it just goes to show how even one word, "generated" that was probably written in haste, can be picked apart by someone on SO. Imagine what an ESL user faces when their English skills are lower. Grammarly doesn't translate, so this non-native speaker translates their post with a different AI tool believing the translation to be equally idiomatic and also grammatical when that's not always the case.
    – Mari-Lou A
    Commented Apr 13 at 12:01
8

Interesting to note that a similar, but theoretical, question came up during the election in the comments on a candidate's (not mine) nomination post.

Under the Americans with Disabilities Act (and likely corresponding laws in other countries), if the user has a true "language disorder" (i.e., not ESL), and believes that they need a "reasonable accommodation" that is not normally afforded other users, then they will likely need to:

  • Contact Stack Exchange directly
  • Provide a doctor's certification of the disability
  • SE staff would likely attempt to contact the doctor's office to verify the veracity of the certification
  • If validated, Stack Exchange staff would work with the user to determine whether an accommodation would be allowed or not, and if so, to what extent.

Until and unless such time that the Mod staff is notified by SE Staff that an exception is being made, no exception should be made to the existing policy that applies to all users.

Given that it seems the user was able to navigate the site sufficiently before using GenAI, it seems unlikely that an accommodation would be warranted or granted in this case.

5
  • 3
    Thanks for entertaining this train of thought. Perhaps it shouldn't be about navigating the site but about "participating in the community". Stephen Hawking's mechanical voice might not have been to everyone's liking either but it did enable him to converse with people. That's a very flawed comparison but the best I can think of right now. Commented Mar 26 at 16:13
  • 3
    @ChristophRackwitz You're absolutely right - "participate in the community" is better wording than "navigate the site". It sounds like they were able to do both adequately pre-ChatGPT, though. If a user wasn't able to, due to a disability, then perhaps they would have a reason to ask for special accommodation. SE could still decline it, on the basis that it disrupts their business or other grounds. Regardless, though, it doesn't feel like it's up to to us as users, or even the Mods. Commented Mar 26 at 18:12
  • 1
    The ADA does not concern itself with accessibility in the sense of being able to write answers. As far as websites go, they have accessibility guidelines for visual impairments. That's basically it. Mental/intellectual impairments are simply not part of their web guidelines.
    – Cerbrus
    Commented Mar 28 at 8:01
  • @Cerbrus "The ADA does not concern itself with accessibility in the sense of being able to write answers. [...] Mental/intellectual impairments are simply not part of their web guidelines." There's also some physical disabilities, when writing an answer requires assisting tools because you are not physically able to use a keyboard. Some people may use AI speech-to-text tools when their voice is recognisable, and some may be tempted to use tools to generate a huge part of the text for them (discouraged). And some people just ask for help from their closed ones when they can (I'm a helper).
    – Clockwork
    Commented Mar 28 at 8:35
  • 3
    @Clockwork those are "client-side" tools that the website isn't aware of. There are no rules in the ADA that require websites to allow speech-to-text. Also, text generated from speech-to-text isn't "AI" in the sense we're discussing here. Input and output are very closely linked. As such, speech to text is perfectly fine to use on SO.
    – Cerbrus
    Commented Mar 28 at 8:40
-22

I don't see what the problem is.

You don't link to the post in question but I will go by what you have posted here, assuming everything is accurate.

low on content/meaning/substance, high on pomp and truisms, sounding authoritative, passive voice, impersonal, abstract rather than concrete

If the answer is clearly unhelpful (or blatantly wrong), edit it if you think you can salvage it, or flag for deletion if not. That has been true before ChatGPT was even invented. There is no reason to make every issue an argument about AI use.

16
  • 1
    note that mods are not expected to evaluate technical correctness, and will likely decline a custom mod flag requesting action on technical inaccuracy of an answer post.
    – starball
    Commented Mar 26 at 2:03
  • @starball then those who understand that the post is wrong or superficial can downvote it or vote to delete. That's how I understand the system here is setup to work. Not everything needs a resolution with a custom mod flag. I don't expect any mod to know some or even any of the technical stuff posted here. Thats what the up and downvote buttons are for
    – user13267
    Commented Mar 26 at 2:05
  • 1
    yes, but your answer post did not mention downvoting, and instead mentioned flagging for deletion. did you perhaps mean to say "voting for deletion" (which is a different thing)?
    – starball
    Commented Mar 26 at 2:06
  • if "voting for deletion" is how you understand it then take it that way
    – user13267
    Commented Mar 26 at 2:08
  • 7
    what does this have to do with how I understand it? they are two different things. stackoverflow.com/help/privileges/trusted-user stackoverflow.com/help/privileges/flag-posts. or are you saying that you refuse to clarify your meaning, and that your readers should interpret what you said however they feel like it?
    – starball
    Commented Mar 26 at 2:12
  • the main point is marking an answer as not helpful, which is the whole point of this site existing, finding correct, helpful answers. Do it by voting to delete, pressing up/down vote buttons, flag for attention to be deleted, whatever. That is the main issue here, not whether my answer mentions "some particular feature the site provides for marking post as useful/useless". If a site feature is not present to someone due to reputation, then thats a limitation of the site software, not anything to do with AI
    – user13267
    Commented Mar 26 at 2:18
  • 5
    "There is no reason to make every issue an argument about AI use." I don’t get why this answer tries to treat things so generally when there is a specific situation. AI use on this issue seems like an adequate reason to make this issue an argument about AI use. Commented Mar 26 at 6:53
  • @MisterMiyagi and the issue, according to op, is "low on content/meaning/substance, high on pomp and truisms, sounding authoritative, passive voice, impersonal, abstract rather than concrete", which is a problem irregardless of whether it was copied and posted from an AI, or someone manually made it up
    – user13267
    Commented Mar 26 at 7:05
  • 3
    @user13267 The meta-question contains a lot more details, most of which do directly call out the use of GenAI. It certainly does not present that quote as the issue, but as part of a larger issue which among other things include clearly matching GenAI content (this being subject to the ban), admitting to purposely use GenAI (again, see the ban), and other parts that clearly relate the issue to GenAI usage. The entire reason why the ban exists is that the normal mechanisms don’t work anymore with GenAI, so content cannot be treated irregardless of being generated or not. Commented Mar 26 at 7:15
  • 8
    I want to emphasize that I only started this discussion because that user claims to use GenAI to "compensate for disability" (translation and "compensatory tool in cases of language disorders"). That is the point of it, the subject of debate. I wanted to see where the community stands on that claim to circumventing the GenAI ban. I picked the example comment only to illustrate the quality of their AI-generated content, not to make it the subject of debate. Commented Mar 26 at 7:25
  • @ChristophRackwitz thats still focusing on the wrong issue, if someone is posting incorrect answers, it should be dealt with for that reason, not for where he copy/pasted it from. So far it has been acceptable to copy/paste information from some blog if it answers a users question. There will be a time in the future where such blogs will also be possibly generated by AI. If someone then refers those blogs in their answer (after verifying its correct) and links to them as a reference source, are we supposed to say its not allowed owing to some people's notion that all ai should be banned?
    – user13267
    Commented Mar 26 at 8:25
  • you cant expect that answerer who referenced the blog to also find out whether the blog was ai generated or not, we can just expect him to verify what he posts is correct. that is what should be the point of focus here, that people are held responsible for what they post, not whether they copied it from somewhere or typed out each letter manually
    – user13267
    Commented Mar 26 at 8:25
  • 6
    @user13267 If you want to challenge the GenAI ban itself, please feel free to do so in a separate discussion (or at the very least do so explicitly in your answer). But right here right now it is in effect and does apply. Commented Mar 26 at 8:32
  • 8
    @user13267 I think you are too focused on the "style" of question that the user mentions. That's not the problem here. The problem is that they suspect the answers of being AIGC, and the user seems to at least try to justify GenAI-use in response. The part about "low on content/..." could be edited out of that question without changing its core meaning. The focus here is on how to handle a claim that GenAI is acceptable when used to compensate for a "disability" (if one exists). Commented Mar 26 at 14:14
  • 5
    Sorry, you seem to have completely missed the point of the question.
    – Dan Mašek
    Commented Mar 26 at 22:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .