68

Background: I have ~1,600 "helpful" flags for AI content, with 27 declined (and some marked helpful but not acted on), and another ~300 pending. In addition (without detailing specifics), I've been one of the most active contributors in the behind-the-scenes AI efforts on SE. Apart from the posts I've had the time to flag, I've captured around 10k suspected AI posts in a database for further analysis and future flagging. I think it's fair to say that I've spent entirely too much time thinking about AI here on SO and SE ;-). That's not to say that I'm always right about all-things-AI, but I do hope you'll take my experience in this subject into account when considering the following.

Even though the current temporary AI ban disallows "all use.", some comments on the question and answers here appear supportive of some level of use of AI on SE, and I'm (perhaps surprisingly, to some, after flagging nearly 2k posts) in favor of some responsible use of AI here on Stack Overflow.

If the current policy were to be updated to allow responsible use of AI, what might that look like? Would the community support a trial period of a modified policy?

For any answer, I'd like to consider whether the proposed changes might make things better or worse than they are today.

48
  • 28
    This seems like a good and important question to me. If you agree that the answer is "I don't think there is any", upvote that answer. Downvoting the question ultimately reduces the visibility of that answer. Commented Oct 24, 2023 at 16:28
  • 6
    @NotTheDr01ds The community being heavily against an answer on meta isn't really a good reason to delete it. The answer still serves a purpose - showing that the community is against the stance taken in the answer.
    – cigien
    Commented Oct 24, 2023 at 16:53
  • 4
    @cigien If I felt that the community was voting on the answer based on any particular points it made, then I'd be okay with that, but it seems that the community is just heavily opposed to any use of AI here, at least for now. The upvotes on that answer are enough to make that point ;-) Commented Oct 24, 2023 at 18:01
  • 4
    Your answer definitely defines what responsible use of AI should be. The problem with your answer is not the answer itself, but its applicability on Stack Overflow or other sites in the network. We cannot allow responsible use of AI because vast majority of users will not use it with care. Without blanket ban, we will not be able to distinguish between responsible or irresponsible use and even worse, we will lose the only way we can fight the flood of bad AI posts.
    – Dalija Prasnikar Mod
    Commented Oct 24, 2023 at 18:43
  • 2
    "Without blanket ban, we will not be able to distinguish between responsible or irresponsible use and even worse, we will lose the only way we can fight the flood of bad AI posts." - I disagree -- We would still have all of the same tools available to us that we have now, only for some subset of the answers, we would at least have fair warning that the post was AI-assisted. That still seems like an improvement over what we have now. Commented Oct 24, 2023 at 19:17
  • 5
    @NotTheDr01ds We can now remove answers if we detect they are AI generated and we don't have to dwell whether they are accurate or not. If we allow responsible use of AI what we will get is people will just post the same garbage, but quoted. And then you will need SME to verify the correctness, and the reputation gained from accidentally correct answers will be enough to counteract other bad posts. An then what. Do we delete all answers from user who post crap, or just incorrect ones. How can we judge them all?
    – Dalija Prasnikar Mod
    Commented Oct 24, 2023 at 19:22
  • 6
    Once you open the doors, you will not be able to shut them down. If we go that route it is game over.
    – Dalija Prasnikar Mod
    Commented Oct 24, 2023 at 19:55
  • 2
    Because it creates a problem for the future, where AI posts that will be left on the sites will create examples people will refer to. For instance, we regularly have questions why is some question closed when some other highly upvoted old one is not, where we need to explain that policies were different in the beginning. When we encounter such old post that does not follow current rules, we can vote to close it. But what we would do with AI posts that were allowed during the "experiment" and then policy changed that we wouldn't allow them?
    – Dalija Prasnikar Mod
    Commented Oct 24, 2023 at 20:13
  • 2
    The root of the problem is that the various GenAI are chat bots. They are not designed for the purpose of writing code. "Responsible use of a chat bot to write code" is silly, it's like asking a question on the cooking site about cutting up a cucumber using a chain saw. And that such practice would be fine as long as you ensure food hygiene. It doesn't matter if you have somehow managed to sanitize the chain of your chain saw so that it doesn't contaminate the food - it is still madness to use it for food processing since it was never a tool designed for that purpose.
    – Lundin
    Commented Oct 25, 2023 at 9:31
  • 5
    @Lundin That's far from the case - Most LLMs are specifically trained on and modeled to write code, and the streaming/langchain format of a putting a Chat interface "in front of" the LLM model doesn't negate that. ChatGPT is, at its core, simply an extra layer of code on top of the GPT 3.5-turbo model that can be used with or without chat (and is the same core model that underlies Copilot). We can (and should) certainly question the quality of code from an LLM (via chat interface), but there's no question that they are designed to serve this function. Commented Oct 25, 2023 at 11:49
  • 5
    @Lundin Except this chainsaw is half a millimeter wide and actually does a decent job cutting a cucumber.
    – CherryDT
    Commented Oct 25, 2023 at 15:53
  • 2
    @Lundin To stick with the analogy, not everyone here cooks with cucumbers. The thing is surprisingly - meaning compared to the average poster - reliable for cutting tougher ingredients, especially in bulk. Commented Oct 26, 2023 at 7:28
  • 4
    @Lundin I think your experience with seeing some examples AI generated answers posted on SO can't be generalized to the quality of AI code in other situations. I and many developers use GPT and other LLM's on a routine basis for both professional and personal projects. While I won't blindly trust/copy/paste the code generated by ChatGPT, I've found that when I give enough detail in my prompt, the generated code is often completely correct, and in cases where it is not, there are only some small changes that need to be made.
    – Brian H.
    Commented Oct 26, 2023 at 15:10
  • 3
    "From what we've seen, the community is fairly good (i.e., brutal) about tearing apart the problems in bad (or even borderline) AI-generated/assisted answers" -- The problem is in smaller tags, where few people actually see an answer when it's posted. These are also the areas where the LLM has fewest material to train on, and is most likely to be making shit up. Commented Oct 27, 2023 at 5:38
  • 5
    @Lundin - you made a generalized comment about generative AI in general that stated The root of the problem is that the various GenAI are chat bots. They are not designed for the purpose of writing code. "Responsible use of a chat bot to write code" is silly. I was responding to that generalization with a counter opinion and suggested that your belief might stem from seeing a very small sample of LLM generated code. The topic being discussed here is not weather LLM generated answers can be used verbatim, but rather if they can be used in some manner at all.
    – Brian H.
    Commented Oct 27, 2023 at 17:43

10 Answers 10

62

The debate around using genAI in information gathering reminds me a lot of the debate that happened around using Wikipedia when it first came out. I think the answer is very similar too:

  • Never use it as a primary source of information.
  • If you find it helpful as a secondary source, and a stepping stone towards finding good primary sources, that's fine, but your final primary source must be solid.

So, you should never be copy-pasting from a genAI into an input box, just like you should never copy-paste direct from Wikipedia.

But if it helps you get an initial understanding of a topic, or find good sources, that's fine: use whatever you want for that initial step: Google, Wikipedia, ChatGPT, Reddit, prayer, divination, guesswork, throwing darts at a keyboard, butterflies... so long as it gets you to actual reliable sources and your actual code is tested for real or based on real experience.

15
  • 1
    Would you consider this to be a good example of that? That's the type of answer I'm trying to support in this. Commented Oct 24, 2023 at 15:56
  • 16
    No. If someone started an answer "I spend several hours browsing Wikipedia then figured out...", that's a bad sign and irrelevant. It shouldn't matter how the person got there, the content should stand alone. Commented Oct 24, 2023 at 15:59
  • 33
    The answer doesn't need all of the "I asked chatgpt x, y, and z", it's irrelevant to the question. We similarly don't need people saying "I googled x and it sent me to y site where i read section z and found the answer." It's just fluff that needs to be removed from an otherwise acceptable answer
    – Kevin B
    Commented Oct 24, 2023 at 16:00
  • 9
    difference being that the Wikipedia content did come from people right from the get-go - actual intelligence and preserved knowledge. And combined with the intelligence and preserved knowledge of other human beings the wiki is not trained, but tuned to precision. It is a very controlled process and because that process has been allowed to run it's course for decades, I have a good amount of faith in its quality. The AI stuff exists basically since yesterday. It is kind of disheartening that people are so willing to have this much faith in it so fast. That's convenience speaking, not wisdom.
    – Gimby
    Commented Oct 24, 2023 at 16:26
  • 11
    I think a lot of people have forgotten how bad the hype-train was when Wikipedia first got big. Airport bookshops and the business consultant circuit were full of management hype telling CEOs they could save money and sack their experts because "crowdsourcing" and "wisdom of the crowds" were the future. Just like with the genAi hype train, they willfully forgot that averaged summary information relies on having quality human experts in the first place (garbage in, garbage out). Commented Oct 24, 2023 at 16:39
  • 9
    OP of post @NotTheDr01ds linked here - I was trying to be a good SO citizen and unclear how/if to mention ChatGPT's support, so I erred with honesty. The q I had was on obscure syntax w/o good docs (R port from a JS lib); no one seemed to have an answer. ChatGPT got me started, and I spent time understanding the syntax until I could post my own answer. I see the point re sources being irrelevant, but if content stands alone then I wonder what a ban on AI-sourced answers really means? In any case, I hope the answer stays, as it's hard to Google this specific syntax. Thanks for the discussion. Commented Oct 25, 2023 at 1:42
  • 3
    @andrew_reece And apologies for "outing" your answer without checking with you - I feel it's a good one, personally. The way I'm interpreting the community response, it's good, but people just pretty much feel that the ChatGPT usage in your case wasn't relevant enough to warrant the honesty. I certainly appreciated it, though :-) Commented Oct 25, 2023 at 2:29
  • 4
    "We similarly don't need people saying "I googled x and it sent me to y site where i read section z and found the answer."" - sometimes this "fluff" is useful though - not just as a simple search description, of course, but as something like "here's how to search for the answers when the similar questions arise, and here's an example for this specific one". At least one answer of mine is like this, for example - stackoverflow.com/a/62927160/3003401, to be precise (not about Googling though, more about reading code, so that might make a difference).
    – Cerberus
    Commented Oct 25, 2023 at 4:11
  • 1
    Think also about questions. If one used ChatGPT to improve his poor English writing in a paragraph, does (s)he have to rewrite it before pasting it to the rest of his/her question? Commented Oct 26, 2023 at 14:45
  • For any automatic translation service, at minimum you should review what it outputs and be prepared to make some edits. What we shouldn't do is blindly copy-paste, just trusting it hasn't misrepresented what you meant. I think ChatGPT etc should be treated similar to how we use Google Translate and similar services (but also taking into account that the fact it can do more also means it can take your text further in the wrong direction). Commented Oct 26, 2023 at 16:12
  • I think for something like Stack Overflow where a goal is to convey information concisely, if the Wikipedians have already consolidated the answer into a small quotable chunk that directly address the question, I think dirctly quoting it (with proper attribution) would be perfectly fine.
    – M. Justin
    Commented Oct 26, 2023 at 18:39
  • 1
    "So, you should never be copy-pasting from a genAI into an input box". One of the things I like to do with genAI is to paste in my own formulated answer (or e-mail, ...) and ask it to make it more readable (like fix grammatical issues, replace difficult words with simpler alternatives). In that case I would proofread and copy-paste from the genAI. What do you think about that? Commented Oct 27, 2023 at 6:06
  • @g00glen00b I refer to Can I take sentences like "with the help of ChatGPT..." as self-admission and flag only based on that?. It doesn't matter what anyone thinks. The rule is 100% rigid; all use is disallowed. No ifs, ands or buts.
    – Gimby
    Commented Oct 27, 2023 at 8:50
  • 3
    @Gimby I'm not talking about the current situation, I'm talking about what the answerer suggested, which is that you shouldn't use it as a primary source of information, and that because of that, you should never copy-paste from an AI. I'm asking where copy-pasting a grammatically improved version of your own answer falls within their suggestion. Because how I see it, if I write my own answer and use an AI to solely improve the grammar, then that AI isn't the primary source of information, I was. Commented Oct 27, 2023 at 9:38
  • 2
    @g00glen00b: That usage has also been discussed to death (on prior meta questions) and the conclusion is that mindless editing for grammar (AI is mindless) makes the content worse. Human readers can tolerate significant grammar problems and still understand you, and human editors can clean it up, and those edits are visible in the post history. On the other hand, mistakes introduced by a tool are hidden. Better to be technically correct with imperfect grammar than have perfect grammar with technical mistakes.
    – Ben Voigt
    Commented Oct 27, 2023 at 15:52
35

I don't think there is any.

Going to simply cut to the bone on this one: AI is a pretty heavily glorified code completion tool at this point, and it's tough to say that an answer that is based in code completion hints alone is sufficient to be an answer at all.

If you were to then add more material to support it, edit it so that it didn't look like the wildest hallucination, and put it together to make it a serviceable answer, well then congratulations, you've just made an answer and you didn't really need the AI to do it.

While the issue of attribution is one that is rooted in our own policies and copyright, the bigger matter in my head is one of accuracy and reliability. Not many people care about IP infringement in an answer, but there's a whole problem that goes beyond the mere moment if the answer is terrible.

If someone is willing to take the time to make their AI-derived answer not terrible, then...the AI part of this isn't the problem IMO. But most people aren't going to do that, so it's simplest to just say "please come up with a good answer", and if folks aren't up to that task, perhaps they shouldn't be answering.

15
  • 2
    Focusing on your last paragraph - It sounds like a perfect summary of my proposal ... Commented Oct 24, 2023 at 15:50
  • 12
    @NotTheDr01ds: If someone is going to go through the effort of putting together an answer that's well-cited and reasonable (with links to documentation when necessary, code that compiles/works, etc), then AI is a red herring. If someone's interested in actually committing to working on an answer, then I personally strongly doubt that they'd reach for AI at all. So my stance is to say "no, don't use AI in your answer" and see if an answerer can come up with their own thoughts. If they can, then we all win. If they can't, then at least we don't have to deal with a bad answer here.
    – Makoto
    Commented Oct 24, 2023 at 15:52
  • 1
    Err, we already have to deal with a lot of bad AI answers here. I'm proposing that we might leave the good ones without throwing the baby out with the bathwater. Commented Oct 24, 2023 at 15:53
  • 4
    @NotTheDr01ds: The reason we had to deal with that was because of a weak and impotent policy around dealing with AI answers. As in, we should have shut the gates on those and never given it another consideration ever again. I challenge you to find a good AI answer here. I haven't seen them.
    – Makoto
    Commented Oct 24, 2023 at 15:57
  • 3
    I've come across a few gems among the crap. Most, of course, are uncited, but the policy is "don't ask; don't tell", so it's unclear if they would have properly acknowledged it if allowed. Here's my most recent example of one where ChatGPT was used (which is against policy) but didn't generate the answer. Here's another (10k deleted) that probably should be allowed. Those are just two recent ones that are cited that I can find. Commented Oct 24, 2023 at 16:05
  • Honestly, quite a few of the "with the help of ChatGPT" answers that aren't copy/paste appear to be validated. Commented Oct 24, 2023 at 16:05
  • 2
    @NotTheDr01ds the deleted one is clearly copy pasted directly from GPT. Why would that one be allowed? it is about as against current policy as it can be.
    – Kevin B
    Commented Oct 24, 2023 at 16:08
  • 4
    I can't follow what this answer is trying to say. It leads with completely rejecting the idea of responsible AI usage, then immediately compares it to established and commonly used tooling, then outlines how one can responsibly use it, and the final paragraph seems to even explicitly accept the usage given bog standard criteria for responsibility - namely not producing terrible answers. So do you think there isn't any, or do you think what you've outlined is? Commented Oct 24, 2023 at 16:10
  • @KevinB There's certainly too much copy/paste in the deleted one for my comfort, but (1) the answer was clearly validated and (2) includes a statement on its caveats. Commented Oct 24, 2023 at 16:15
  • 3
    @NotTheDr01ds: The first answer you've cited is terrible since it's just a code dump. In my assessment, if I wanted to learn from this, I've got nothing but a heap of code that I have to copy and paste and hope that it works. The deleted one suffers from the same symptom and uses the new style switch that seems unnatural/unneeded in this context (and it took entirely too long to look it up, the usage of it isn't widespread yet). In both cases, the answers are saying "do this" which isn't a very good answer IMO.
    – Makoto
    Commented Oct 24, 2023 at 16:22
  • 3
    @MisterMiyagi: Basically, what I'm getting at is that if someone is going to go through all of the effort to make a good answer on the site, I don't think they'd bother with using AI as a crutch. As they dig into it they'd realize that it's not really helping their case all that much. But policing that is a challenge, so instead it's easier to require folks to own their own work so there's less ambiguity over it.
    – Makoto
    Commented Oct 24, 2023 at 16:24
  • 7
    I agree with this answer. I think the effective response to OP's question is something like "Not Applicable", because any use of genAI here where it is truly responsibly used is going to be a case where that AI use is completely irrelevant to the question, its phrasing, and any answers, and we should never know that AI was involved in any way, obviating the need for such a policy allowance in the first place. If we know that AI was used, it is because there is something problematic in the content that fundamentally creates issues with the question or answer. That's, IMO, irresponsible use.
    – TylerH
    Commented Oct 24, 2023 at 16:36
  • @NotTheDr01ds "Focusing on your last paragraph - It sounds like a perfect summary of my proposal" It is not at all your proposal.
    – philipxy
    Commented Oct 24, 2023 at 22:21
  • This only speaks about answers. However, a very large portion of SO posts are questions. A clear policy must also deal with questions. Possible use of AI in SO is not at all just about generating code. Commented Oct 30, 2023 at 15:44
  • I think part "you didn't really need the AI" is a less applicable to people for whom English is second language. For me, for example, improving a style and a grammar of posts I write is difficult work and quality of English I write would not be as good without the AI tools which are banned here. So I think policy which would allow one to responsibly use LLMs to improve style and grammar would be very helpful.
    – fiktor
    Commented Jan 20 at 8:13
25

Note: this is a statement of what I would like an eventual policy to be, not necessarily a statement of current policy

The AI should not be the primary writer of the post or the only verification of any idea within it

The main issue with LLM usage, in the context of Stack Overflow, is that it produces confident, well-written answers that are nonetheless completely wrong at a fairly high rate. This wastes the time of anyone who reads the answers and is fooled into thinking they were written by a human, and the amount of time wasted on the receiving end is wildly disproportionate to the amount of time spent by the poster to generate it. Secondarily, there are plagiarism issues with representing text that you did not primarily author as your own. Responsible AI usage is that which does not have these issues, and also is distinguishable from irresponsible AI usage via some criterion other than "was the answer correct"1.

Consider a self-answer that says something like:

After some back-and-forth with ChatGPT, I came up with the following solution:

a brief section of code goes here (but actual code, not just this placeholder for this meta post)

It outputs foo: 42, bar: 128, which was the result I expected.

That seems ... fine2. I'd like to see some more explanation, but we have lots of not-really-explained answers on Stack Overflow, and while I wish people would explain them better, they're allowed.

Similarly, someone once wrote an answer that demonstrated a solution to some graphics problem, with screenshots of the results, noting that they got some help from ChatGPT. That seems fine, too: they had clearly verified the results, as evidenced by the screenshots, and the post seemed human-written.

In both cases, the only evidence of ChatGPT usage is that they said they used it somewhere in the process. I don't think it makes sense to delete answers on that basis alone.


1 That is, it is not viable to allow copy-pasting attributed ChatGPT answers that you claim to have verified, because 1) people seem to be pretty bad at actually doing that, and 2) the effort required to verify every LLM answer is infeasible at Stack Overflow's scale, especially given the ease of generating them in vast quantities.

2 Arguably, that first sentence could be edited out as unnecessary to the answer.

8
  • 1
    Upvoted, but I'm not a huge fan of the first example. In reading comments, at least, it seems that ChatGPT will too-often suggest an answer that gives the "expected" results in a "too narrow" use-case (and "Output" is quite often part of the GPT response). As you said, I'd at least like to see some (non-GPT generated) explanation -- I know that we get a lot of "code-only" answers, and we (much to my dismay) allow these. But I think with AI answers we should expect some level of explanation (even if it's just well-commented code) rather than putting the onus on the reader to interpret it. Commented Oct 24, 2023 at 19:23
  • "Responsible AI usage is that which does not have these issues..." This is a bit of a negative definition. What are responsible usages? Are there responsible usages? And can you actually reliably distinguish between responsible and irresponsible usages? On the other hand I think this is the criterion that really should be used. Commented Oct 24, 2023 at 21:56
  • 1
    See also How accurately could one differentiate between simple copy & paste of AI generated content and improved and fact checked AI generated content?. Unfortunately the heat of the argument was too high at that time. But we are getting there. Commented Oct 24, 2023 at 21:58
  • 8
    "After some back-and-forth with my cat, I came up with the following solution". Such meta commentary does not belong in a post. Research including vet then compose in your own words out of your head. "AI should not be the primary writer of the post or the only verification" It is NOT verification, it cannot be the "only" verification.
    – philipxy
    Commented Oct 24, 2023 at 22:16
  • 4
    @philipxy Sure, I agree with all that. But it's commentary that often appears in posts, and shouldn't, alone, be a reason to delete them. "After reading the docs, I..." is another common one. Edit it out, sure, but the question here is whether that sort of consultation should be allowed at all, not whether mentioning it is useful (edited to clarify). As for whether it's "verification"...that's really just semantics. It's evidence (not good evidence). It's a bit like asking someone and them saying "that sounds generally right, I guess." The point is, you need reliable verification.
    – Ryan M Mod
    Commented Oct 26, 2023 at 20:27
  • Re "such commentary" I was responding to "That seems ... fine." before you footnoted it & all I said was what I said.
    – philipxy
    Commented Oct 27, 2023 at 5:23
  • "As for whether it's "verification"...that's really just semantics". No, it is not, AI is NOT verification & calling it verification is WRONG. (Maybe you have in mind something about "verification" as a person's process intended to verify, but that's not what you wrote & if so the problem is not "semantics" it's "reading what was written when the author didn't write what they meant".) And it is only "evidence" in the sense that it can suggest research & not in the sense that it implies anything. PS "AI should not be the primary writer" It should NOT BE a writer.
    – philipxy
    Commented Oct 27, 2023 at 5:28
  • 3
    "they had clearly verified the results, as evidenced by the screenshots" Checking one test case is not verification. And the level of confidence from running even a large number of test cases through a black box is far worse than running tests on code you authored.
    – Ben Voigt
    Commented Oct 27, 2023 at 15:57
17

What might "responsible" (a.k.a. acceptable) use of AI look like on Stack Overflow?

I want to challenge the frame and say that AI is much less important here than many people may think and the essential keyword here is responsible use instead. With responsible use almost everything becomes acceptable, without almost nothing.

I define responsible use as:

  • Fully knowing and understanding your own contributed content
  • Having made a best effort to ensure that all of the contributed content is fitting to the topic, working, describing or solving the problem to the highest possible extent.
  • Describing all used sources (things you used to come up to the solution) to a reasonable extent
  • Obeying legal requirements (copyright, giving attributions) and ensure that the content actually adds to the amount of knowledge (unsuccessful check for duplicates)

This is responsible use of Stack Overflow and ensures that the library of knowledge remains high quality, which is the primary concern here. It requires humans to be at the front and center of the whole process. If people use Stack Overflow responsibly, I'm personally fine with them using whatever including AI, but if they do not responsibly use this platform then many things, and especially AI, become really dangerous. That's why we should predominantly think about how to ensure responsible use. My gut feeling is that it may not only be necessary but even also sufficient to ensure the high quality.

I hope, I could convince somebody that it's not really AI that is the problem, but non responsible use of Stack Overflow. Mindless copying of AI generated content is the problem, but this is also true if you leave AI out. It's just a subset of mindless copying in general.

13
  • 2
    As a comment: AI is going to stay and more and more people will use it, maybe even without realizing it. The real issue is what to do with it and what else to do additionally. Commented Oct 25, 2023 at 11:29
  • 6
    I agree with this in principle, yet a speciality of GenAI is that irresponsible and responsible use can be indistinguishable for anything but an in-depth expert analysis. In that sense, GenAI may need stricter rules not out of principle but sheer practicality. Commented Oct 25, 2023 at 12:34
  • 5
    We cannot successfully impose responsible use anywhere on the site without AI, so expecting we will be able to do so for AI use is a pipe dream, I think.
    – TylerH
    Commented Oct 25, 2023 at 14:11
  • 1
    Strongly agree with the overall answer here plus (again, coming from the perspective of having examined ~10k likely-AI-answers here) @MisterMiyagi's view that "GenAI may need stricter rules not out of principle but sheer practicality." Response-use may be the same in both cases, but because of the problems we face with AI here, we likely need to require more of an AI-assisted answer (again, if it's even possible). Commented Oct 26, 2023 at 15:57
  • 1
    @TylerH Fully agree that responsible use is a strong minority and it looks like a pipe dream overall. I see it for myself in that I contribute much, much less to Stack Overflow than I used to. I have basically given up on it. But there are such people that can use StackOverflow responsibly and they shouldn't have any problem with using AI tools either, which is the question here. It's just that the advent of LLMs, which cannot be undone anyway, helps the spammers (people who think that copy and paste is cool) more than others. Even forbidden all use of AI won't help there. Commented Oct 27, 2023 at 10:26
  • @NoDataDumpNoContribution I think forbidding the use of AI is far more effective than trying to set up a framework for reviewing its use to allow 'responsible' uses only. We don't have the manpower for either, but the former takes far less manpower than the latter.
    – TylerH
    Commented Oct 27, 2023 at 14:05
  • 1
    @TylerH The use of AI is already forbidden and whether that actually has a significant effect nobody knows for sure. Remember that debate during the moderator strike about false positives? It's not only the precision it's also the recall that is unknown here. And most people probably won't say if they used AI. Responsible use of AI means not only checking but also reformulating where necessary. I don't believe it will be reliable detectable then because it's a difficult problem and manpower isn't the limit. Being strict might make people feel better but it won't make a difference in practice. Commented Oct 27, 2023 at 17:24
  • @NoDataDumpNoContribution It's certainly had a significant effect. If we did not ban it and delete every instance we found, do you not think we would be flooded with AI content right now?
    – TylerH
    Commented Oct 27, 2023 at 19:06
  • 3
    @TylerH Yes, I think it had an effect, but I'm not sure how big it is. Did we catch 10%, 50%, 80%, 120% or 150% of all AI generated simply copy&pasted posts? I don't know and I would like to know that number better. And this isn't really the topic here. I understand why you are against it but I cannot share in it. I'm fine with saying: people probably won't use AI responsibly but if they did so it would be okay instead of saying that AI is evil in the hope that this helps fighting the AI spam wave. It wouldn't be completely true and I'm not fully convinced it would really help. Commented Oct 27, 2023 at 20:34
  • The problem with this answer is that it rises the bar to contributing answers... which is something some aren't comfortable with. Specifically the "contributed content is fitting to the topic". I've seen more times that I would be comfortable with, people arguing that any answer, even if it doesn't solve the problem described in the question, is a valid answer and can not be deleted by moderators.
    – Braiam
    Commented Oct 28, 2023 at 16:48
  • 1
    @Braiam You can surely argue that responsible use should be defined differently or in general there is a grey zone. I thought that the "any answer that is an attempt to answer anything should be left" is simply because moderators are not supposed to be subject matter experts. Users who are subject matter experts should still be able to delete these answers (maybe as very low quality) or at least to heavily downvote them. Commented Oct 28, 2023 at 21:23
  • 1
    While what you say makes sense, we are in the real world. Moderators have very strongly, under threat of suspensions, very against users using any mechanism that isn't the 20k delete vote to remove answers. That puts us in the conundrum that only the people that contributed many posts (some of which with questionable quality) have the only say on the matter. If I flag something, I expect my peers to review it, not just moderators. But moderators don't allow that, and expect users to behave like moderators too. We already discussed this before.
    – Braiam
    Commented Oct 29, 2023 at 16:17
  • The experts lost.
    – Braiam
    Commented Oct 29, 2023 at 16:17
10

I would say that use of an AI tool as a stylistic\grammar helper is acceptable. In my experience it helps non-native English speakers (like myself) to polish answer (or question) to make it clearer and less ambiguous.

Having said that. On the other side, I don't know if a question\answer that was formulated by a human being and was grammarly/stylistically polished by an AI tool will trigger moderators attention as a false-positive case.

9
  • 2
    You're using those services in a supportive manner - that is what makes it not only acceptable but actually great. You're not cutting corners, you're not trying to save time, you're not trying to be lazy. Doing what you do actually means you need to spend more time formulating your content as you do an extra polish cycle. Absolutely nothing wrong with that, quite the opposite.
    – Gimby
    Commented Oct 25, 2023 at 11:23
  • 5
    Upvoted, as I've long been supportive of at least the concept of using AI as a grammar assist. Big caveat, however -- In my experience, you typically can't simply copy/paste your answer into ChatGPT and then copy/paste the result back out without ChatGPT often adding ideas that weren't yours in the first place. And yes, that will often trigger us to believe the answer was AI-generated. You definitely shouldn't copy parts of the answer that you don't want the AI to modify, such as code-blocks, inline code, and especially block-quotes or quotations (of course). Commented Oct 25, 2023 at 11:38
  • 4
    If you blinked and missed it (and I did, so I don't have any direct experience with the experiment), SE tried this. Commented Oct 25, 2023 at 11:41
  • Yep, I actually missed it, will take a look :) Commented Oct 25, 2023 at 12:12
  • Most word processors (including free ones) also include pretty good grammar and style revision tools. Microsoft Word's has been professional-grade for decades, and I'm pretty sure Google and LibreOffice have good ones, too. No "AI" is needed for this.
    – TylerH
    Commented Oct 25, 2023 at 14:10
  • 4
    Yes, but by default ChatGPT wants to completely rewrite the text. The output can be shaped, e.g., by prefixing it with "Correct the following, but only with absolute minimum changes. No complete rewrites of paragraphs, sentences, and parts of sentences. Don't replace any words with synonyms." Even with such a stern prompt, it may try to do too much. How many users are going to use such prompts? (yes, that is a rhetorical question.) Commented Oct 26, 2023 at 1:39
  • Yes, and I would like to stress that we should always think about questions as well! Many people and voters here only think about answers because they are experienced coders who do not ask any. Questions are also very important. Commented Oct 26, 2023 at 14:42
  • @VladimirFГероямслава Are you referring to AI-generated/assisted questions? (edit: from your deleted answer, "yes") Those definitely exist, and I've seen several dozen at least. We (the community, the AI flaggers, and the Mods) have been pretty tolerate of these for the most part, and we haven't seen many problems with them. However, yes (and I agree with what you wrote in the deleted answer) - LLM's have the same danger here of adding or changing meaning. Commented Oct 26, 2023 at 16:01
  • @Peter Mortensen It tries way too much even when instructed not to because the temperature parameter for ChatGPT (UI) is too high for proof-reading. When used through API, temperature can be set to 0 and GPT4 can do a decent job at proof-reading without rewriting.
    – fiktor
    Commented Jan 20 at 8:01
5

Use it primarily as a search engine

Generative AI is very bad at giving accurate answers, but very good at finding the context of the topic. Therefore it can be useful for understanding the context and becoming aware of the relevant topics connected to it. From there on, you should research those topics by yourself.

Just as we should not copy-paste the preview of the result page after a google search, we should not copy-paste the results of the generative AI. (that includes not writing an answer by a mere reformulation of the generated answer)

Generative AI might become useful in the future in cases of problems where the solution is hard to find but easy to verify, but even that would requite strong discipline in actually verifying the answer, which would be nigh-impossible to enforce.

But as a quasi search engine to find related topics, it can be useful and relatively risk-free.

3
  • Please can you clarify how does this relate to posting on Stack Overflow?
    – blackgreen Mod
    Commented Oct 28, 2023 at 7:02
  • That just like when we use google searching as a help in posting an answer on SO, we don't copy-paste the search result or its preview, but we use that search to better understand the topic, and use that information to write our own answer. And similarly, if we use a chatbot, we should not paste its answer into SO (not even reformulated), just use the chatbot's answer as means to learn about the context, which we should then research on our own and then use that to post an answer.
    – vsz
    Commented Oct 28, 2023 at 18:05
  • Worth noting that this is basically exactly what Stack Overflow is experimenting with right now with their OverflowAI Search alpha.
    – zcoop98
    Commented Oct 30, 2023 at 19:57
2

Using AI to improve the English grammar and English stylistics especially in questions should definitely be acceptable but the asker should be responsible for verifying that the rewritten sentences actually make sense. Many questions posted by non-native speakers have obvious problems that automatic helpers should be able to help with.

2
  • By default, ChatGPT will completely rewrite the text. It requires a very stern prompt to limit the damage. As the company found out. Commented Oct 28, 2023 at 17:31
  • 1
    @PeterMortensen There is a big, if not crucial, difference between letting the AI to rewrite the whole post and supervising it to rephrase written text. Here we are discussing what should be alowed at all, not what is currently possible or easy to do. Commented Oct 28, 2023 at 17:38
0

Deleted based on community reaction, but undeleting based on several requests. Also making some slight adjustments based on RyanM suggestions.

As RyanM said, this is what I would like an eventual policy to be, not necessarily a statement of current policy. And it's entirely possible that we can't get here at the moment, but perhaps eventually:

  • AI-assistance with your answers should be the exception, rather than the rule. If you find the need to use AI-assistance for the majority of the answers you are posting, then you likely do not have the sufficient level of expertise in the questions yourself and should not be answering those questions in the first place.

  • However, when you do believe you have sufficient expertise in the question being asked, you may occasionally use AI to assist in finding and/or wording an answer. If you choose to do so, you should, to the best of your ability:

    • Reference/Cite any reliance on third-party sources such as ChatGPT for the answer. Note that this has always been required when posting material to Stack Exchange that you did not create yourself. Make it clear in your answer what the AI provided, and what is your original work.

    • Improve the AI-answer with your own words and experience. If the person asking the question wanted an AI-generated answer, they could have just gone there in the first place. Users here expect (and have a right to expect) more. Try to have something to add that makes your answer worth posting.

    • Whenever possible, Validate that the answer is correct and works. If you cannot validate the answer, you should have reasonable confidence based on your experience (not the AI content) that the answer will work.

    • Understand the answer that you are posting or at least point out areas where you are not sure.

    • Correct any issues with the AI-generated content that can reasonably be identified before posting.

    • Reply to (or edit the answer based on) follow-up questions/comments about your answer using the same rules as above. Do not copy/paste AI responses into comments. However, if you often have to correct AI-assisted answers after posting, that likely indicates that you are not responsibly and properly validating the answers in the first place.

    • Warn of corner cases or potential issues if needed.

    • Follow all other other guidelines for writing good answers. This includes not answering questions that are too vague or are obvious duplicates of already existing questions. If you find that you are commonly answering questions that are subsequently closed by the community, then these answers (AI-assisted or not) are not "good answers" by the definition of the Help Center.


Benefits:

  • I would hope that there would be fewer uncited AI answers to have to hunt down and flag, reducing the moderation/curation effort involved.

  • Since (valid) AI answers would be cited, it would allow the community to judge them based on that knowledge and to make sure they follow the other guidelines (also a possible pitfall - see below).

  • Reward (hopefully) good behavior (cite/validate) while still providing for a policy to remove answers which the Mod team and community feel aren't "responsible" under the existing heuristics.

Downsides:

  • Cited, AI-assisted answers may be judged more harshly by the community, and that may just end up making users reluctant to cite them in the future. Or it may push users toward AI-generated code-only answers.

  • We'll certainly get some folks who will just continue their "irresponsible use" but slap "Generated by ChatGPT" in the answer. Of course, if these show substantial copy/paste or appear to be AI under the existing approved heuristics, then they can be removed under the policy and/or downvoted by the community.

16
  • 1
    The only acceptable use by current policy is as a "research assistant". Taking a gpt response and "improving" it is still relying on gpt to produce answers, regardless of whether or not it is properly cited/vetted.
    – Kevin B
    Commented Oct 24, 2023 at 15:44
  • @KevinB Can you elaborate on your objection to "relying on gpt to produce answers" if they are good answers? Commented Oct 24, 2023 at 15:52
  • 4
    Absolutely. "Artificial Intelligence (a.k.a. GPT, LLM, generative AI, genAI) tools may not be used to generate answers for Stack Overflow. The answers you provide must either be your own original work, or the properly cited work of others." An answer that was produced by GPT and then modified/vetted into being a good answer is still an answer that was generated by AI tools and is therefor not allowed.
    – Kevin B
    Commented Oct 24, 2023 at 15:55
  • The core of this, that occasional per user usage is okay (and the inverse is not okay), seems to go against the established policies of rating/moderating content and not users. Either an answer is good or it isn't, and throwing it in the bin purely based on unrelated posts seems counterproductive. Commented Oct 24, 2023 at 16:02
  • @MisterMiyagi I asked about "moderating users" as well when I first started working on flagging AI here. I've been told by both SE staff and the Mod team that, when it comes to moderating AI-content, we can (and must) take user behavior into account. Commented Oct 24, 2023 at 16:09
  • 1
    @KevinB Okay, I agree that it is not allowed under current policy -- That's why this question and answer is about changing the Policy. What I'm asking is what your objection is to changing the policy to allow good answers? Commented Oct 24, 2023 at 16:11
  • @NotTheDr01ds I see how "user behaviour" is relevant when we are considering a blanket ban of AI usage, where we are actually interested in whether a user uses AI at all. This doesn't apply when responsible usage is accepted, i.e. when we are not interested in whether AI is used at all but how. Commented Oct 24, 2023 at 16:16
  • 1
    My objection to it is effectively "this is why we can't have nice things." Yes, there's value in allowing "good" answers to exist regardless of how they were created. however, given the nature of how these answers are being created, it becomes rather difficult to tell the difference between someone abusing cgpt to inflate their rep by answering in areas they don't have expertise vs someone just using it as an efficiency tool to pre-write answers that they then vet/correct. The latter is what would be useful to allow, however how can we allow that while still banning the former? [cont...]
    – Kevin B
    Commented Oct 24, 2023 at 16:18
  • 1
    Do we only allow high rep users to paste gpt generated answers? Do we limit it by tag badges? how do we determine that the user actually has that expertise vs faking it with gpt? Magic? Who gets to decide who is and isn't an SME?
    – Kevin B
    Commented Oct 24, 2023 at 16:19
  • @MisterMiyagi Err, perhaps? But I think there's room for allowance on that in policy. If "too many" (TBD) of a user's answers cite AI, then that starts to become "irresponsible" (shows the user doesn't have the SME to answer), a recent answer should be flagged, and a Mod would have a discussion with the user if warranted. Commented Oct 24, 2023 at 16:19
  • @NotTheDr01ds I don't see how "too many answers cite AI" and "doesn't have the SME to answer" is equivalent. My SME in, say, Python wouldn't suddenly disappear if I decided to use GenAI to avoid the hassle of writing Python answers in a foreign language. Commented Oct 24, 2023 at 16:21
  • 1
    "it becomes rather difficult to tell the difference between someone abusing cgpt" - That's actually one of the main benefits of my proposal that I see. Once cited, it becomes a lot easier to tell. Trust me, right now it's pretty difficult. We have several users with more than 60 uncited AI answers posted here gaining hundreds of rep each. Eventually we'll get rid of them, but it's challenging under the current rules. Commented Oct 24, 2023 at 16:24
  • We're unlikely to change the behavior of those blatant "bad actors", but letting the community judge cited AI content on its merits is a lot easier than trying to first determine if something is AI or not. Commented Oct 24, 2023 at 16:24
  • Bummer... I had written a comment in response to this but the answer was self-deleted while I was typing it up. Unfortunately I didn't save the comment, so I can't recall what exactly it said.
    – TylerH
    Commented Oct 27, 2023 at 14:07
  • 1
    @NotTheDr01ds Most likely? It'd be great to have a 1st class solution built into the site with a dedicated cadre of reviewers, because I think that is what it would really take, but I think that's way too much for SO these days. They can barely manage changing the colors anymore.
    – TylerH
    Commented Oct 27, 2023 at 14:54
-5

Acceptable uses:

  1. Use it for minor edits to your post. Grammar (already suggested here), style, even code correctness. You could write the post in the generative AI, polish it, and then copy it to Stack Exchange. If the original post is written by you, this seems perfectly fine.

  2. Use it to write the code but then follow up by testing the code. One of the biggest problems now is that people basically ask the question of the AI and then blindly copy the answer here. But if someone tests that the code actually works, then that's completely different. ChatGPT wrote this code and here is my test suite is far more robust than "Hey, I don't know how this code works, but this is what ChatGPT suggests." Particularly since they often don't know if the code works.

  3. Add a user for each generative AI and let the AI answer questions directly. Yes, some of the answers may still be horrible. But at least this way, people would know that it was the AI writing the code and could be appropriately wary. And of course, it's not like answers from humans are guaranteed to be accurate either. Combined with the last point, when people ask the generative AI to answer a Stack Exchange question, it could post directly to Stack Exchange. That would make it much easier to detect copy and paste, as Stack Exchange would have the original content. So the normal protections against duplicates and plagiarism could be used.

The last differs from this answer in that I'm talking about the generative AI itself watching for questions from Stack Exchange. Also, it's not just one potential answer. If there are multiple generative AIs, we can compare results from them. And of course, Stack Exchange could offer user options. Think that AI answers are pure crap? Turn them off. Because with responsible AI use, we know which answers are from AIs. Maybe turn them off by default with a link saying how many AI answers there are if you want to view them ("Click here to view the 17 AI answers.").

My experience with generative AI is that it requires an expert user to curate the results. Sometimes answers are great, just what you need. Other times they're just plain wacky. Telling the difference requires more expertise than writing the code.

It's better than simply asking the generative AI directly in two ways. First, it is subject to normal voting (so if you lack the expertise to distinguish good from bad answers, others can help you). If the AI posts a crap answer, then others could award that with a downvote. Second, answers aren't limited to just the one generative AI. Humans and other generative AIs could answer as well. Stack Exchange answers are crowd sourced. Generative AI answers are single sourced. You are more likely to get at least one good answer from a group of sources than from a single source. Even if the single source is better than average (which, unfortunately, generative AI isn't yet).

2
  • 5
    Re "Use it for editing your post": That ought to be qualified. By default, ChatGPT will completely rewrite the text. It requires a very stern prompt to limit the damage. Commented Oct 28, 2023 at 17:28
  • 1
    I have yet to see a very strong argument for including an automatic AI answer, given that anyone can copy the question into any chatbot themselves already. Presumably, one of the reasons users come to Stack Overflow is because they feel it's a community of experts and knowledgeable folks who know what they're talking about; AI isn't that, and needs to be treated differently as a result. We also need to be mindful about putting AI answers next to human ones, where they could be equated– we don't want to end up with more melting eggs.
    – zcoop98
    Commented Oct 30, 2023 at 20:09
-6

I feel like there is a some discussion concerning something like covert use of AI. That is, "I used AI but it didn't give me a good enough answer so I modified it and now it's better." This may (or may not) carry the implied, "whether I used AI or not as a basis is my business and I'm keeping it a secret," which seems to be causing some friction.

Why not just end run this entire line of reasoning and go straight for the AI answer in every answer?

Add a RHS box with some, "standard," AI answer to every question, essentially from the time it's asked. Label it clearly, color it differently, make sure no one thinks it's a human (umm.. non-AI) answer.

Heck, let people quote it if they want. Color that the same color. Somewhere, something, can benefit from AI. It's another tool. That would settle an acceptable use question.

I realize that there probably isn't really one single standard AI answer. Someone smarter than I am and with more experience in the field than I have, probably has a good answer to this. There are probably some other issues too, ones I don't know for the same reason.

11
  • 3
    Note that this has been proposed dozens of times on the "AI-ban" and other meta Q&A. So far no one had a good answer to there being not one single standard AI answer, nor many of the other issues too. Chiefly, no one managed to convincingly show how even more AI answers would stop regular users from also posting AI answers. Commented Oct 27, 2023 at 15:39
  • 4
    Listen, if you're answering with AI, it's not really you that's answering, now is it? If I wanted an AI answer I could go there myself, but I'm here on Stack Overflow instead.
    – Makoto
    Commented Oct 27, 2023 at 16:38
  • @MisterMiyagi "...no one managed to convincingly show how even more AI answers would stop regular users from also posting AI answers. " To be fair, it was never tried. In order to show that it does reduce the amount of posted AI answers by regular users one would need to try that out actually. One can kind of motivate it though, I think, with the duplicate effect. Posted AI answers by regular users would probably emerge later and could be closed as duplicates, if only we had better duplicate detection (maybe AI powered). Commented Oct 27, 2023 at 20:40
  • 2
    @NoDataDumpNoContribution I mean "shown" also as just having a theory of how it could work - just like you tried here. Yet on the part of actually being convincing I have no idea what "duplicate effect" you refer to, nor what you mean by closing answers as duplicates; and the crux remains that it hinges on AI answers allowing us to spot other AI answers, which would in fact be very simple to try/demonstrate by going AI hunting by asking questions yourself to AI and comparing similar answers. We don’t need "official" AI answers for that test. Commented Oct 28, 2023 at 6:36
  • 1
    @MisterMiyagi With duplicate effect I meant that AI generated answers to the same question may be very similar to reach other and with closing answers as duplicates I mean that already in the past we removed simply copy and pasted answers as not necessary or plagiarism (a concept that is kind of difficult in the AI age). With "don't need official answers" you raise the bar quite high. I don't need a lot of things, doesn't mean I'm happy without them. I would simply try it out and see. Commented Oct 28, 2023 at 7:00
  • @NoDataDumpNoContribution My point is that for the claim of reference AI answers being sufficient to spot "duplicate answers", the references don’t have to be officially provided by SO itself. If anyone wants to show that this scheme actually works, they can just run the experiment themselves. That would be a good grounds to request such a feature, provided the experiment comes out positive. Commented Oct 28, 2023 at 7:06
  • @MisterMiyagi Oh yes, you're right. One could do it like that. But doing the experiment online within the live SO environment would give much more information, for example additional voting information. I would prefer to just try it out. It has been proposed many times, there is a rationale for it (at least I tried to present one), and it seems like the obvious best combination of both worlds. I really want SO to be like this in the future. Commented Oct 28, 2023 at 10:21
  • 1
    Re "Label it clearly": That should include "70% probability of being completely wrong". Or if it must be positive, "30% probability of being correct" Commented Oct 28, 2023 at 17:37
  • 1
    @PeterMortensen We could do the same with all users contribution. I'm not sure what the prediction for newly registered users first answer would be. Commented Oct 28, 2023 at 21:26
  • @PeterMortensen I was unclear: label it as being a direct AI contribution. Sorry if this violates the spirit of the community here, but no one knows the actual, "correctness," of any answer on SO. Let the reader, the OP or anyone who visits later, decide if it's 30% or 70% or 5%... just like we let the reader decide which answer is correct. Let 'em vote on the AI contribution as well if they like. Commented Oct 31, 2023 at 19:54
  • @MisterMiyagi I doubt you'll ever be able to stop people from using AI, or plagiarism, or lying.. or fraud, or theft, etc., all for the same reasons. I seem to recall some cases around the world where decriminalizing some crimes regarded as less heinous worked out. If you're answering the same thing the AI in the sidebar says, people will know. And as for which AI? I propose the one that costs the least. Maybe SO+(upstart AI) leads to symbiotic benefit. Commented Oct 31, 2023 at 20:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .