83

There was a user who wrote an answer using AI, and it was very obvious (a long answer in a short period of time!). And, I, as a humble user who doesn't really know anything, thought that this answer is written using ChatGPT, so I posted the following comment on their answer (my comment got 2 upvotes):

This is written using ChatGPT? Right...?

And their response was like this below:

Hey @SallylovesLightning , don't worry about whether the answer comes from a human or a bot algorithm. As long as it can handle the task and provides helpful information, it's all good! I'm actually working on a similar project that includes a picture slider and a complex video player. Some parts of my code were generated by a bot, but I'm also reviewing and understanding each line of it as a human. So, thanks for your comment, and let's keep supporting each other! 😊

I didn't really respond or do anything because I don't know all the rules about the use of AI, so I just moved on.

Today, I got 3 downvotes on 3 answers, and the user commented on 2 of my answers with the following comments:

Although it's okay to use AI, this answer also look likes answerd by chatGPT with few edits.

It looks like a chatGPT answer, right?

And my responses were like below (respectively):

Hmm, What makes it look like it's written by AI?

Again, it's not, you are sure?

So what should I do in the current case, and future cases? Currently, should I move on or flag that user probably?

I don't really care about the -6 I got, but it's not really nice when you have actually valid answers that get downvoted (even though they have had upvotes before).

And in future cases, what should I do? Never tell people that their answers are written using AI? Just let them do that? I don't really like to see a troublesome post and move on.

Downvote them in peace? I can't really afford the -1 and I'm really worried that I might downvote a valid answer one day.

And, even the greater question is, how do I prove that my content isn't written using AI? Y'all can see my profile page. I've about 43 answers, and really none of them was written using AI. So it's really bad when people say otherwise.

38
  • 36
    If you suspect it's ChatGPT content, then use a custom mod flag to bring it to mod attention; it might be look at at some point this year (when/if the strike finishes). You may want to check the other user's answers to see if they appear to be CrapGPT generated too and link those in your flag. Rarely, however, it's worthwhile engaging with the user as they care not for the rules of the site and only their (insert Golem voice here) precious internet points. The user in question has several hundred of those points, so should be aware of some rules.
    – Thom A
    Jul 24 at 12:33
  • 67
    So, TL;DR, you called out someone for using AI, and now they're taking revenge by claiming you're using AI?
    – Cerbrus
    Jul 24 at 12:36
  • 8
    @Cerbrus Yeah, basically, but I don't know whether they downvoted or not, but I'm sure about one because they just lost a -1 at the same time they downvoted me so yeah I guess so.
    – Ghost
    Jul 24 at 12:37
  • 39
    I've flagged his comments as harassment, it's clear he's just "taking revenge", whilst hypocritically using AI...
    – Cerbrus
    Jul 24 at 12:39
  • 3
    Well OP is not the only person who can downvote, anyways all 3 might be from the same user, we don't really know. Flag them for harassment as Cerbrus states, if the downvotes are still there after 24 hours you can try raising a flag for that as well Jul 24 at 12:46
  • 16
  • 35
    "I can't really afford the -1" Votes are yours to use as you want, but you have 737 reputation (ATOW according to Meta); going to 736 isn't going to dent that. They're just internet points, they aren't a currency. If you aren't downvoting a bad answer because you don't want to lose reputation yourself, then you aren't helping push bad content down. My personal opinion is that not enough people use their downvotes, and is why so much bad content is still around on the site.
    – Thom A
    Jul 24 at 13:17
  • 5
    How else do I denote that answer is wrong/unhelpful/not useful to other users, @SolarMike .
    – Thom A
    Jul 24 at 14:09
  • 26
    @ThomA Reputation is certainly a form of currency, given that you spend it to perform certain actions, and gaining it allows access to almost all the different site features here.
    – TylerH
    Jul 24 at 14:10
  • 3
    Not on a whim, @SolarMike . There are processes in place so that well-received answers can't be deleted; even if by modern standards they are terrible.
    – Thom A
    Jul 24 at 14:11
  • 6
    For an answer to be removed, by the community, it needs to have a score of <= -1 and have three users, or the OP to vote to delete it, @SolarMike . If it has a positive score then deleting that answer is much harder, and even the OP is unlikely to be able to delete it. Comments, on the other hand, can be deleted even if they have 1,000's of upvotes, and could be deleted due to a single quickly handled NLN flag.
    – Thom A
    Jul 24 at 14:15
  • 13
    @ThomA While I agree with the sentiment of your "[737] to 736 isn't going to dent that" comment, it always seems to be the users with tens-of-thousands of rep who make that argument. At lower rep, each point feels worth more. E.g. proportional to their total rep, that's like you losing ~120 rep per downvote.
    – DBS
    Jul 24 at 15:41
  • 3
    Honestly, I'd happily downvote bad answers and lose 120 rep, @DBS .
    – Thom A
    Jul 24 at 15:48
  • 27
    @ThomA I'm not raising a debate, but also you're 88k, maybe you're up to lose 120 points per each downvote, but once you're 25k, you'll probably stop, because you're just going to lose your privileges. As a small user, I didn't get all the privileges yet, so I can start downvoting, every point matters until you're 30k or something. And if I lost all my privileges, I won't be able to contribute again...
    – Ghost
    Jul 24 at 16:34
  • 7
    Yet you're refunded the rep when such an answer is deleted, which OpenAI content, eventually, will be.
    – Thom A
    Jul 24 at 17:47

6 Answers 6

75
  1. If you suspect that someone's posted an AI answer, say nothing to them and flag it. Don't go on a witch hunt.

  2. If someone suspects you of posting AI content, and you know you didn't, just don't respond to it, and flag appropriately.

You're not going to be able to prove anything to anyone that would satisfy them about whether or not you're using AI, so I wouldn't waste energy on it. If you know you're not, that's enough.

4
  • 13
    Could you clarify what "flag appropriately" means for point 2? What if the comment seems to be merely asking if it is AI in good faith, like the OP did? (I'm not contesting that flagging might still be the right response, but flag it how? Should the OP's original question be flagged as harassment?) Jul 25 at 9:07
  • 3
    @ScottMcPeak Flag the comment as No Longer Needed if it is just asking, or one of the other higher levels if it becomes demeaning or abusive.
    – E_net4
    Jul 25 at 11:23
  • The problem with flagging is that the flags are being ignored. Since the strike started I've flagged around 20 answers. 17 are still pending.
    – dbc
    Jul 26 at 6:36
  • 4
    @dbc [insert first time/you must be new here meme]. I'm at over 50 GPT flags and I know of users having ten times as much. If that's the site the company wants, that's the site they'll get (but not one I'll keep participating on).
    – CodeCaster
    Jul 26 at 7:21
41

Be right, consistently

This is the one thing that current generative AI tech cannot do when writing answers. It "hallucinates" information that is false, sees patterns that don't exist, and otherwise fails to respond correctly.

With users who are posting AI-generated content, it is virtually always possible for a subject-matter expert going through their answers to find things that just don't make sense. If you're consistently right and not just throwing guesses at the wall to see what sticks, you're almost certainly not an AI.

A few tips:

  • Make sure that you fully understand a question before answering it
  • Test your answer to make sure that it does the things you think it does
  • Ensure that your answer solves the problem and isn't just general debugging advice or wild guesses

(note: these are general tips; none of this is meant to imply that you did not do these things)

12
  • 4
    On the other hand, if there's a temptation to throw "general debugging advice" at a question, this is a sign that the question needs improvement. Jul 24 at 22:01
  • 8
    Some things I haven't seen the bot do, but that I usually try do when writing answers: Provide references (especially to relevant docs). Provide illustrations where relevant to the explanation. Provide a MCVE with sample output. Answer novel questions, or ones that it has no hope in hell to get right (yeah, that's a hard one).
    – Dan Mašek
    Jul 24 at 23:21
  • 2
    Debugging advice is valuable, but should be in a comment not an answer, of course.
    – tgdavies
    Jul 25 at 0:34
  • 3
    @DanMašek GPT can do all of these things except illustrations, if prompted appropriately; it's just that it generally doesn't do them correctly. The test needs to be not "does it have references/MCVE/etc.?" but "do those references actually exist, and say what the answer claims, and does the MCVE actually produce the sample output when I run it?" etc. Jul 25 at 2:48
  • 2
    It's very difficult to always be right. To err is human, they say. Obviously it's AI too. Jul 25 at 11:00
  • Exactly. We're here to help fellow practiciioners of our great trade. So let's help each other. Let's give good answers to questions by explaining the hows and whys of each situation. Ignore the rubbish. There's too much of it to get rid of it all.
    – O. Jones
    Jul 26 at 11:21
  • @O.Jones There isn't too much rubbish to get rid of. There are dozens of us; that's less than 10000 pieces of rubbish each. We can totally get rid of that, if we work together.
    – wizzwizz4
    Jul 26 at 11:47
  • 1
    @DanMašek that would make a good answer to this question.
    – qwr
    Jul 26 at 17:19
  • What ever happened to "to make mistakes is human"? I see your logic in how this would, if possible, be a weak proof of humanness in 2023, but this is an impossible standard
    – Luc
    Jul 31 at 21:56
  • @Luc the kinds of mistakes humans make and the kinds of hallucinations that LLM's produce differ quite wildly.
    – Kevin B
    Jul 31 at 21:58
  • @KevinB The answer says to never be wrong, not to make/avoid a specific kind of mistake. You may be right though, though I wouldn't know what kind of mistakes I should or shouldn't flag then. Such information would be a useful answer to add!
    – Luc
    Jul 31 at 22:00
  • I should note that "Be right, consistently" is not exactly "Never be wrong". One can be pretty consistently right while making an occasional reasonable mistake. I could probably clarify that somewhat.
    – Ryan M Mod
    Aug 1 at 18:03
6

Why is AI-generated content problematic?

  1. It may be irrelevant or incorrect
  2. Easy to generate, but hard to verify

I am going to concentrate on problem 2. The problem is, unscrupulous people generate much content with little effort, getting some good things unfairly. If someone suspects you are one of these people, you can prove the contrary by showing that you spent (or wasted) time on your content. Edit your answer!

  • Clarify; fix typos, wording and formatting
  • Add examples
  • Expand some of the points you mention in your answer

One of these unscrupulous people would never do any of this. If you show that you wasted some time writing your answer, you make it very clear that your content didn't come directly from AI. And this is what really matters.

I don't want to say that you should always do this: normally I would write a good answer as quickly as possible and be done with that. But I think this is a valuable idea to use sometimes.

4
  • 3
    Easy to generate, but hard to verify. To clarify a bit, answers generated by AI always SEEM to be correct, but often the answers are low-quality, or flat-out incorrect hallucinations. The problem is that often only subject matter experts can tell the difference. Because AI-Generated content is so easy to produce, a problematic user can generate incorrect answers much faster than those few people who are experts can validate them. Therefore Stackoverflow can quickly get overwhelmed with low-quality content if AI-generated answers are allowed. Jul 26 at 8:34
  • 1
    "Getting some good things unfairly" isn't really the problem, it's the flood of content that sounds good but is actually wrong, but only an expert can tell the difference that's overwhelming to attempt to moderate. (Though if the website is like, writing prompts or something, then this is less likely to be an issue.) Jul 26 at 8:38
  • 2
    I agree. I only referred to these concepts, didn't want to explain them fully — full discussion is under the "ban AI" policy post.
    – anatolyg
    Jul 26 at 8:49
  • 1
    One problem with this suggestion is that ChatGPT can actually do this quite convincingly with minimal prompting. See the edit history of this answer for a practical demonstration. Jul 27 at 12:05
-16

You don't

You don't owe it to anyone to prove you did or didn't use AI. If someone wants to go and dig deeper into your post, that's their problem. As long as you provided something of value to the asker (and all the other users at large), even if you did use an AI for help, that's your business and nobody else's. And in case you didn't, anyone claiming otherwise isn't going to change it anyway

As for "it's not really nice ...", this is just something we all need to get used to when using the Internet. Just do the best you can, make sure your post actually helps others to the best of your ability and don't worry what others say.

16
  • 3
    One of the issues is that, many times contributions that are generated by LLM are not actually useful, since the authors, don’t have the expertise to verify the generated content is accurate and appropriate to the question Jul 25 at 11:18
  • it seems like putting any weight in the AI pan is met with a hail of downvotes. Sad reality, considering how revolutionary AI is, but +1 from me
    – machine_1
    Jul 25 at 11:26
  • @SecurityHound I never said anything generated by LLM should be copy/pasted without understanding what it is
    – user13267
    Jul 25 at 11:30
  • @machine_1 I have been seeing that trend for months now. Thank you for the +1 and for not downvoting something just because it does not say "all AI should be banned"
    – user13267
    Jul 25 at 11:32
  • 8
    @machine_1 AI isn't revolutionary yet. Despite the fact that the technology is only a seed in the ground currently, even if it would be advanced - the people aren't at all caught up yet. AI will require a massive jump in mental maturity before it can be applied effectively by the average human being. But hey - we can always use more funny videos of Arnold Schwarzenegger put where he doesn't belong until that state is reached.
    – Gimby
    Jul 25 at 11:41
  • 6
    @user13267 - You also didn’t make it clear that AI generated content is currently banned. It’s fine you disagree with that fact, your welcome to that opinion, doesn’t mean I have to agree with it. Jul 25 at 11:58
  • @SecurityHound so don't agree with it, I never asked you to. And its not my duty to make anything clear about the AI generated content "policy". I neither made that "policy" nor think it is useful in its current form
    – user13267
    Jul 25 at 12:00
  • 6
    @machine_1 The problem are the strings attached to that weight. Statements such as "As long as you provided something of value to the asker (and all the other users at large), even if you did use an AI for help, that's your business and nobody else's." are simply false. Perceived value has no bearing on the ban of AI usage. Whether one disagrees with established policy does not change said policy, and advice that conflicts with policy is generally not useful. Jul 25 at 13:18
  • 2
    If someone used an AI for an answer, it is my business as a consumer of the content. I want to know where the content came from. If it's from an AI, that's not valuable to me. I would ask an AI directly if I wanted their opinion. I'm on Stack Overflow because I want human answers. Same thing in general--if an AI wrote a news article, blog post or movie, it's irresponsible and dangerous not to disclose that. It's a form of misinformation. Correctness is beside the point here (but AI loses that argument miserably, on average, if you want to go there).
    – ggorlen
    Jul 25 at 17:07
  • 1
    @ggorlen perhaps you would like to know everytime someone used a grammar or sentence completion tool as well. And anyone is free to disclose their use of ai, if they want to. Or may be they'll be willing to if you ask nicely.but they shouldn't have to, as long as what they posted isnt vandalism
    – user13267
    Jul 25 at 23:11
  • 4
    @user13267 There's a major difference between grammar and sentence completion and GPT AI. Grammar correction and sentence completion tools don't write code or generate novel content or ideas, which are the main contents of the post. The grammar and linguistic syntax is just dressing on the meaning/ideas (but it's nice when writing is reasonably coherent syntactically).
    – ggorlen
    Jul 25 at 23:15
  • 1
    @ggorlen and I've never said people should just blindly copy/paste whatever was generated either. There should always be human verification involved and the answerer should be held liable for whatever they post
    – user13267
    Jul 25 at 23:18
  • 5
    @user13267 I never accused you of saying that, but the truth is, most users of GPTs I've seen are doing exactly that--blindly copy/pasting and spamming. To be clear, I'm against any and all usages of GPT on SO, regardless of whether it's correct or attributed. The overall impact is a net negative, in spite of the occasionally correct and coherent lucky answers GPTs happen to spit out from time to time. This doesn't justify the fundamental problem of trust I discussed in my first comment, again never minding the accuracy issues.
    – ggorlen
    Jul 25 at 23:19
  • 2
    @user13267 Even if everyone attributed their answers and GPTs were correct 100% of the time, the exercise would still be pointless--we'd all drop SO instantly and use GPTs directly. There'd be no sense in dumping that stuff here as well. Even if we grant that as well, it's been shown in multiple studies that training GPTs with GPT output leads to a degradation of quality. So basically there's no value in GPTs on the network in any scenario, even if we ignore all of the obvious downsides and trust/quality issues that aren't going anywhere anytime soon and generously assume they're perfect.
    – ggorlen
    Jul 25 at 23:23
  • 1
    @user13267 Like the other thread, you seem to have misunderstood the purpose of my original comment. I understand what you wrote in your answer, and I'm posting a comment to explain why I disagree, a normal thing to do on meta. Your responses ignore the points I've made and repeat your original stance from the answer as if I didn't understand it the first time, or it's incontrovertible. "I've never said" is usually used to indicate that someone is putting words in your mouth. GPT being unreliable is beside my point too, but you mention "value" in your post, which is related to AI reliability.
    – ggorlen
    Jul 26 at 1:16
-17

How to prove that my content isn't written using AI in any way?

You cannot prove that.

A short human text may happen to share the same features as some AI-generated text. Also, "my content is written using AI" can be very far from copy-pasting an AI-generated text.

12
  • 5
    But in a world where the Community bot has the "Not a Robot" badge can we trust anything? Jul 24 at 19:52
  • 1
    Who says you don't simply type the text you read from ChatGPT on another screen? Video recording doesn't help. Jul 24 at 21:30
  • @ThomasWeller screenrecord too. But yeah always way to cheat. Can't prove someone didn't learn the output by heart and retype. Jul 24 at 21:30
  • 4
    Would you care to elaborate on why one cannot prove that? Right now, this answer contributes nothing that the other answer does not, and is entirely conclusory.
    – Ryan M Mod
    Jul 24 at 21:35
  • @RyanM-Regenerateresponse the previous answer was different. It stated "The simple answer is: You can't; AI can now write text indistinguishable from human writing." Jul 24 at 21:36
  • 3
    Also, this answer came first, ;)
    – Kevin B
    Jul 24 at 21:37
  • 3
    Oops, I had the edit time in my head rather than the post time. Nonetheless, my point about it being conclusory stands. "You cannot prove that." provides no reasoning or logic behind why that is the case.
    – Ryan M Mod
    Jul 24 at 21:45
  • @RyanM-Regenerateresponse A short human text may happen to share the same features as some AI-generated text. Also, "my content is written using AI" can be very far from copy-pasting an AI-generated text. Jul 24 at 22:01
  • 2
    Now this is just a comment at best.
    – Cerbrus
    Jul 25 at 6:20
  • 1
    @Cerbrus no, it's a concise answer. I need to write a thesis. Jul 25 at 14:40
  • 4
    "AI can now write text indistinguishable from human writing" is completely wrong. Just look at any human-generated text next to any AI text. Grammar/syntax alone, the human text will almost always be imperfect. AI output is rigid and tone-deaf, regurgitating the same lines over and over and usually missing the key point of the question. Check out the deleted answers in this thread to get an idea of what ChatGPT looks like relative to a human. "To achieve the requirements you listed..." shows up in two different answers! Who writes like that?
    – ggorlen
    Jul 25 at 17:01
  • @ggorlen--onLLMstrike yes, that's why my answer is different Jul 25 at 17:03
-36

how to prove that my content isn't written using AI?

The simple answer is: You can't; AI can now write text indistinguishable from human writing.

Although AI might exhibit a certain pattern in writing, it can be given instructions to adapt its style, tone, and content to match specific human-like characteristics or tailor it to different purposes.

My recommendation is to completely ignore accusations of using AI models to post your answers. As for the possibility of being downvoted just because someone thinks you have utilized AI in writing your answer, there's little to nothing that you can do about it.

Hopefully, someone will neutralize the downvote if they find your answer useful.

10
  • 22
    "AI can now write text indistinguishable from human writing." - If that is the case, why can I easily detect, OpenAI (ChatGPT) generated content? Jul 24 at 18:58
  • 4
    @SecurityHound: Because it wasn't given those instructions I guess
    – machine_1
    Jul 24 at 18:58
  • 11
    What exactly is the purpose of this answer? It's factual wrong since many users can detect the differences between human and LLM generated content. One of the reasons hardly any moderation is happening and here is spam on the front page of SO. Jul 24 at 19:05
  • 6
    You can give it instructions not to make things up that aren't true, but that won't stop it from doing that.
    – Ryan M Mod
    Jul 24 at 21:34
  • 4
    "AI can now write text indistinguishable from human writing." Simply not true.
    – Cerbrus
    Jul 25 at 6:19
  • 1
    Re "it can be given instructions to adapt its style, tone, and content to match specific human-like characteristics or tailor it to different purposes.": That is indeed possible, but statistically it never happens. Even the company couldn't be bothered to do it. Jul 25 at 6:48
  • You're right in a way, but text written by human A is usually very much distinguishable from text written by human B. So while I agree that ChatGPT as an example case produces text that looks very real, that does not imply it is invisible among the crowd. No it has a very distinguishable style, like any human being or pretend human being.
    – Gimby
    Jul 25 at 15:26
  • @Gimby: you can try experimenting with it. try to give "her" fine-tuned instructions and see the result. You can give her further instructions if needed to see how far you can go and if the generated responses are better and closer to human-level writing style.
    – machine_1
    Jul 25 at 15:37
  • 1
    "AI can now write text indistinguishable from human writing" is wrong just based on punctuation analysis alone, never mind the rest of the "It looks like you're encountering a problem with..." and blatantly tone-deaf AI tropes that totally miss the point of the question half the time, running you in endless circles with "You're right, my last response was incorrect. To fix your code...".
    – ggorlen
    Jul 25 at 16:56
  • @machine_1 I don't need to try anything, I was already agreeing it looks pretty lifelike. My point was that this does not matter. Like any other "human being" it has tells and it is a pretty generated and patterned style at that. For now. Write in your own way and there should be no reason why your style just magically overlaps with how the bot is designed to write. Until they change it, and then everyone who considers themselves an expert on detecting it will have to start over. Nobody ever said that curation on Stack Overflow is fun.
    – Gimby
    Jul 26 at 7:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .