30

English is not my mother tongue so I am not very good at it. I use a translator tool to make my answer more efficient. Then I correct the grammar of the translated sentence using 'Grammarly extension'.

But recently some of my answers were removed and I was banned for a week. They assumed that I gave the answers using 'AI generated' tools, (which is totally wrong). Their message:

...Use of ChatGPT for content while its use is banned: The use of ChatGPT as a source for content on Stack Overflow is currently banned. Please see the Meta Stack Overflow question "Temporary policy: ChatGPT is banned". It is not permitted for you to use ChatGPT to create content on Stack Overflow during this ban....

Recently I used an 'AI Content Detector' tool for testing and found that sentences that were corrected for grammatical errors using 'Grammarly Extensions' gave an 'AI DETECTION SCORE' below 50%.

Now my question is:

  1. How can I stop them from getting confused?
  2. Should I stop using 'Grammarly Extension' to correct grammar?
19
  • 9
    I'm using Grammarly all the time, but Grammarly can't translate, and it's far from ChatGPT. What is the tool you used for translation?
    – Andrew T.
    Apr 5 at 16:43
  • 30
    English not being your mother tongue is not an excuse to uncritically use two different computer tools (presuming that is the case). Machine translation usually creates an incomprehensible mess. It is your responsibility to check that the result actually makes sense. Don't put too much trust in the tools. Apr 5 at 16:52
  • 2
    @MetaAndrewT. I usually use Google Translate to translate sentences. Apr 5 at 16:53
  • 14
    Google Translate and other tools are very good, but they also often make mistakes in meaning, and definitely make strange word choices. In many cases I'd rather see someone's manual attempt at English than an automated version: the latter comes with an inappropriate level of confidence and makes it very difficult to work out issues when things go wrong. Apr 5 at 17:04
  • This question was originally on Meta Stack Exchange. There are some related posts on on MSE regarding the lack of a network-wide policy: "Ban ChatGPT network-wide" and Is there a list of ChatGPT or other AI-related discussions and policies for our sites?
    – Makyen Mod
    Apr 5 at 17:21
  • 5
    Stack Overflow, the company, has categorically said "With due consideration, we've decided no network-wide, general policy regarding banning ChatGPT, or other AI generated content, is necessary or helpful at this time." That, effectively, means that even a definition of what AI generated content is (which is what you're asking about) devolves upon individual sites. Thus, this question can only be answered on a site-by-site basis and must be asked on the child meta site for each individual site.
    – Makyen Mod
    Apr 5 at 17:26
  • @Makyen These were not my accepted answers. I reply using a translator but still, my replies are mistaken as AI content and I get banned with the reply deleted. So I wanted to know, is there any other method? By following the procedure I can answer nicely (without getting banned). Because I use a translator to translate my own answers and then correct their grammar with 'Grammarly Extension' and give them as answers. Apr 5 at 17:37
  • 24
    The underlying question in this meta post aside (which is a valid question), when you've posted an answer in which you explicitly say "Answer provided by an AI language model, ChatGPT I asked ChatGPT your question just to see, and he surprised me with a nice answer." months into the ban on such content, what do you expect moderators to be assuming about the source of the content you post? It seems fairly reasonable to be considering that it is, in fact, AI generated when it generally appears to be so.
    – Makyen Mod
    Apr 5 at 17:44
  • 1
    @Makyen Hmm, this was my first answer by ChatGPT and I mentioned well in that answer that it was made by ChatGPT. At that time I didn't know that ChatGPT-provided answers are explicitly not allowed. Apr 5 at 17:54
  • 17
    I use Grammarly to check virtually all my contributions. The detection rate is less than 0.01% on that content. Without checking the content you submitted, that had a high probability of being written by a AI, I can't comment more than that. What I can tell you is, content written by you then correct by Grammarly, the probability of that content being written by an AI is typically less than 0.1%. Looking at your answer history is troubling. You have answers, which are literally link-only answers, to another SO answer. Apr 5 at 20:42
  • 6
    Please read about how answer properly & how to give credit properly. Help center Looking at your last 6 posts: Many times you copy without correct format & credit and/or you give code-only answers and/or you answer a question that should be flagged to be closed as a duplicate because there is already an answer post with the answer. Please don't do that.
    – philipxy
    Apr 6 at 0:47
  • 4
    Thank you all. I have made some mistakes in almost all the answers I have given so far. Recently mistakes in my answers have been pointed out through comments. I either deleted them or updated them. Again I apologize for my mistake. Apr 6 at 7:27
  • 2
    @PeterMortensen You might prefer that, but other people will close the question with bad English and suggest using a translator.
    – user253751
    Apr 6 at 23:40
  • 1
    @ARSharifUddinJUMMAN Please note that, in some circumstances, Grammarly (and similar services) are considered to be a security (and/or privacy) leak because they transmit the data back to their home. Apr 7 at 17:43
  • 1
    @ARSharifUddinJUMMAN Note that merely disclosing the fact that it was written by ChatGPT doesn't make it permissible to post such content. My understanding is that type of content is unconditionally banned on Stack Overflow. Apr 7 at 17:50

2 Answers 2

42

As discussed in the comments, these tools were not the issue in this particular suspension. However, to answer the question of whether using tools such as translators or grammar checkers counts as AI-generated for the purposes of the AI-generated content ban:

Tools like translators or grammar checkers are allowed, provided that you wrote the input to them, or, if the source is text written by another human, you cite the original source in compliance with our referencing guidelines (uncited translated material would be considered plagiarism). Use of these tools on AI-generated input is not allowed, for the same reason that AI-generated answers are not allowed in the first place.

As always, if you believe we've made a mistake in issuing a suspension, you can reply to the moderator message with an explanation, and we will consider it.

3
  • 2
    Thank you all. I have made some mistakes in almost all the answers I have given so far. Recently mistakes in my answers have been pointed out through comments. I either deleted them or updated them. Again I apologize for my mistake. Apr 6 at 7:27
  • 3
    Please do not make reference to comments in a post, make your post self-contained. What is the relevant content? (Even if a post is about a comment exchange..) Don't expect the comments to be there or for people to read all the comments or know what is relevant or why.)
    – philipxy
    Apr 6 at 23:17
  • Ryan, on meta.stackoverflow.com/questions/425223/… you wrote "Or in other words: using generative AI for spelling and grammar correction is problematic due to the fact that it doesn't really understand what it's supposed to be doing, and thus is prone to doing things like rewriting the content entirely. " Does that mean that it "tools like translators or grammar checkers" are using generative AI, then they are not allowed? Jun 18 at 10:48
22

Grammarly doesn't detect as AI generated

To quote a comment by Security Hound:

I use Grammarly to check virtually all my contributions. The detection rate is less than 0.01% on that content. Without checking the content you submitted, that had a high probability of being written by a AI, I can't comment more than that. What I can tell you is, content written by you then correct by Grammarly, the probability of that content being written by an AI is typically less than 0.1%.

Your specific issue wasn't Grammarly. It was ChatGPT.

As has been mentioned elsewhere, your use of Grammarly wasn't the issue. You did, in fact, post content generated by ChatGPT. You even explicitly stated that you did:

Answer provided by an AI language model, ChatGPT

I asked ChatGPT your question just to see, and he surprised me with a nice answer.

Toolchains which produce content that appears to be AI generated are banned.

The primary reason for the blanket ban on AI generated content is to allow people performing curation, both regular users and moderators, to detect and deal with AI-generated content on the basis that it was AI generated, rather than differentiating the content on "is this good/correct" vs "bad" or "this was an acceptable use" vs "this was an unacceptable use". The reason for this is that we, people doing curation on Stack Overflow, just don't have the available resources (i.e. the time of subject-matter experts) to make a finer-grained differentiation beyond "this was AI generated". Because of the nature of AI generated content, it takes orders of magnitude more time and effort (and level of expertise in the specific subject) to differentiate such content based on criteria beyond "this was AI generated". We, overall, just don't have and can not reasonably obtain the amount of time from subject mater experts in order to make those finer grained differentiations.

Unfortunately, 99.9% of the tens of thousands of posts which have been posted using AI generated content have been by people that appear to have just copied the question into the input of the AI generation and then copied the output (potentially with some edits) onto Stack Overflow. They do so without any attempt to verify that the response is accurate, or even self-consistent, and with no thought to the negative impact that doing that has, both on the asker and on the site.

So, unfortunately, yes, we have to ban the use by that 0.1% of users posting such content that is, in fact, generated using a tool-chain that might otherwise be acceptable, because the massive majority of people are not using the available tools in a reasonable manner and we just don't have the available curation resources to be able to differentiate, at the scale Stack Overflow operates, between those "potentially good" uses and the massive majority of use that is "bad".

Ultimately, allowing the flood of bad content that is very hard to determine is actually bad/inaccurate/incorrect, other than by someone who is already a subject matter expert, will kill the usefulness of Stack Overflow, as future readers would have to wade through a huge amount of crap in order to get to the content that's actually good. So, we must remove the bad content, even if that sacrifices a small amount of content that would have otherwise been acceptable.

The issue isn't "oh, people will need to work a little harder to do curation". It's that if AI generation was permitted, then it increases the need for time from experts doing curation by orders of magnitude and there's just no way to meet that need. AI generated content often reads as if it is great, but is actually bad/inaccurate/confusing. There have been various times when we've seen question askers, who usually are not experts, accept, award bounties, and (presumably) upvote AI generated answers, but then leave a comment like "this doesn't work". Thus, the additional resources need to already be experts in the specific technology and be critically evaluating such answers. Stack Overflow just doesn't have the necessary resources available to accomplish that and no way to get them.

We'd like to be able to support people using such toolchains where the base work is actually theirs and they have verified their answer. We just don't have a way that's possible while still being able to remove the 99.9% of content that detects as AI generated which is bad/inaccurate/not the work of the author. We're open to ways it might be possible to allow the use of at least some tools that don't cross over into actual AI generation of the underlying content, but we've seen nothing, including no viable suggestions from the community, that would actually work at the volumes at which Stack Overflow operates. [Note: what would be viable from our point of view would need to be something that requires no changes to the underlying site software, as development resources are very limited and getting Stack Overflow, the company, to make substantial changes is ... difficult and time-consuming.]

There will be toolchains in the future (and may be some now) which might have been reasonable, but which are, effectively, banned.

This is an evolving situation. There will be tools developed that use AI generation. Some of these tools could end up being quite beneficial to people in many situations. However, if they generate content that appears to be AI generated, then they are, effectively, banned, because, as mentioned above, we just don't have to the ability to do that differentiation at the scale Stack Overflow operates.

It would be nice to just be able to ask people what they were doing, get a truthful answer and then be able to differentiate the work based on that. However, A) we just don't have the resources to ask about each piece of detected AI content before removing it, and B) a large percentage of "bad actors" lie and do so repeatedly and vociferously. It would be really nice if those facts weren't the case, but they are.

To a large extent, in addition to being a technological issue, it's a societal one. Adapting will take time.

Commonly available AI generation that routinely produces "eloquent bullshit" that reads well to humans is a new capability. It's yet to be seen how we, as a society, will adapt to that capability being available. It's quite unclear what level of AI assistance/generation society will, in general, consider to be acceptable for someone to claim as their own work, or use in a way where they, in the past, would have been expected to be the sole author. Such tools will continue to improve and what uses/level of use society will consider acceptable will change over time.

However, from Stack Overflow's point of view, until such time as such tools the vast majority of the time generate actually correct answers, or answers which can be quickly and easily be identified as good/bad, we must continue to ban such content and remove it when it's detected.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .