30

This isn't a proposal - I'm just wondering how this contradiction will be resolved and starting a discussion on it.

Currently, Generative AI is banned on Stack Overflow. The SO corporate office unilaterally decreed that moderators are severely restricted in how they enforce this ban (or similar bans on other stacks), and essentially can make no inferences about where the text of a post originated - however, one must assume from the text of SO's mod-restraining policy that a user simply admitting that they used generative AI would still be a sanctionable offense. This is based on the public version of the policy, since we as users don't know what was in the (apparently divergent) policy conveyed to moderators in private.

Given that the experimental AI formatting assistant is a first party feature, one would assume that users utilizing it would discuss it openly (assuming SO actually rolls it out.) It's easy to imagine a user responding to a comment asking for clarification on their wording, commenting something along the lines of "the formatter changed my wording, what I really meant was [X]." The current implementation of the formatting assistant does not hide that it is powered by generative AI, it even presents the text "AI suggested edits for your question."

It would seem that moderators would be well within policy to ban these users, as they have admitted to knowingly violating the generative AI ban, and SO corporate has not restrained mods from taking users at their word that they've used generative AI. SO corporate has "asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. This standard of evidence excludes the use of moderators' best guesses based on users' writing styles and behavioral indicators," but a confession is not a guess whatsoever.

11
  • 76
    As much as I dislike the current policy, this just seems absurdly pedantic. Commented Jun 16, 2023 at 20:34
  • 2
    the current guidelines, and coc, would both require disclosure, and support such disclosure resulting in deletion. so it's not really all that useful to try to figure out how the current policy can work with the change given that if the change were to go live outside of an A/B test, changes to the result and/or our policies would be required.
    – Kevin B
    Commented Jun 16, 2023 at 20:41
  • 10
    @MisterMiyagi I disagree that it's pedantic - pedantic would be odd consequences from two policies created by the same entity. This is an odd consequence created from two policies created by different entities - SO corporate, and elected community moderators. Both entities have the option to stick to their guns and refuse to change, leaving this absurd consequence in place. (Or, the worst option, SO corporate makes a second unilateral decree overriding the elected moderators and the AI ban itself, not just micromanaging its enforcement.)
    – Willa
    Commented Jun 16, 2023 at 20:44
  • @Willa As far as regular users are concerned, Stack Exchange Inc and Stack Overflow moderators are one entity. Sure, SO moderators can stick to their guns... but it would be absurdly pedantic. Commented Jun 16, 2023 at 21:04
  • 1
    @EJoshuaS-StandwithUkraine to be clear, I'm not proposing anything - it's an open-ended question. I have my own opinions but I tried to limit this question to identifying the inconsistency and the consequences if neither policy budges.
    – Willa
    Commented Jun 16, 2023 at 21:27
  • In that case, I agree with your point. I upvoted. Commented Jun 16, 2023 at 21:32
  • @MisterMiyagi After the OP's clarification in the comments, I agree with their point: if SE was consistent, they'd have to both offer the feature and ban people for using it - if they implement this feature, they'd basically be trying to have it both ways, which they really can't do. Commented Jun 16, 2023 at 21:37
  • 23
    @MisterMiyagi asking the question is a useful rhetorical technique. It establishes that the feature is fundamentally incompatible with agreed-upon policy. If it is not rescinded, the company is admitting that they no longer extend to us any autonomy WRT setting policy; as such, it becomes a compelling reason for a lot more people to leave - not just the ones currently on strike. Commented Jun 17, 2023 at 9:32
  • 11
    Note: It was turned off on 2023-06-18T074200Z+0. Commented Jun 18, 2023 at 9:38
  • 1
    Note that it will apparently soon be illegal for platforms to host AI-generated content disguised as written by a human. So when SE finally rolls out the formatting assistant, there's a good chance that using it will permanently attach an "AI-generated content" sticker to the post. Commented Jun 19, 2023 at 14:34
  • @DmitryGrigoryev SO is not an EU company, nor does it have any offices in the EU, as far as I'm aware. So in all practicality SO can just ignore whatever the EU says about whatever. The EU doesn't have any jurisdiction fine SO for being out of compliance with their rules.
    – TylerH
    Commented Jul 27, 2023 at 13:58

2 Answers 2

46

I think that you've identified a basic contradiction in the feature: it seems impossible to justify Stack Exchange providing a generative AI feature given that such technologies are currently banned. Either they can't provide such a feature, or they can't continue to ban the use of generative AI. This feature is, at best, a very mixed message, and at worst an outright contradiction that's comparable to Drug Enforcement Agency agents selling heroin on the streets. (A cynic might speculate that that's why they made the ban effectively unenforceable, though).

See also my answer here.

5
  • 27
    It's not just banned by SO's site policy banning AI generated content, but using AI generated content without proper referencing, which is what the tested tool was doing, is a Code of Conduct: Inauthentic usage policy violation on every Stack Exchange site. While there might be leeway with just minor grammar changes, that wasn't what this tools was doing. For examples of its work, see the answers to "We are seeking functional feedback for the formatting assistant"
    – Makyen Mod
    Commented Jun 16, 2023 at 22:26
  • 13
    To be clear, I'm not saying it's a good situation, particularly for unsuspecting question authors. It is, however, something which the company absolutely should have spent time thinking about and resolving in a manner that's not harmful to their users and in a way consistent with the both the platform's policies and the site's policies. IMO, the best way to do that would have been to not use generative AI for grammar correction, which is a task it's really not suited for, particularly given that AI hallucinations are inherent in the current generation of that type of AI technology.
    – Makyen Mod
    Commented Jun 16, 2023 at 22:55
  • 3
    @Makyen So the OP is absolutely correct - using the feature would technically be grounds for suspension for multiple reasons under current site policy? Commented Jun 16, 2023 at 23:44
  • 8
    "It is, however, something which the company absolutely should have spent time thinking about and resolving in a manner that's not harmful to their users and in a way consistent with the both the platform's policies and the site's policies." - fundamentally impossible with generative AI, unless we no longer care about plagiarism (the site staff should be made aware that this could become a source of legal liability, and could also be a cause for many more users to leave). Commented Jun 17, 2023 at 9:37
  • 12
    @EJoshuaS-StandwithUkraine Yes, that's correct. However, in practice, A) most of the mods are on strike, so it's unlikely to happen, at the moment; B) when choosing what to do in such a situation, a moderator is very likely to strongly take into account that the question author was deceived by the company. We're trying to talk with the company to resolve this in a manner that's acceptable to everyone. Our position is that generative AI is fundamentally the wrong approach for this feature, as Karl Knechtel has pointed out, and that such a feature should be built without using generative AI.
    – Makyen Mod
    Commented Jun 17, 2023 at 12:41
-34

I'm just wondering how this contradiction will be resolved and starting a discussion on it.

I think the issue is that the ban is too wide: it's a generic ban against the use of generative AI and subsequently includes all of its potential use cases beyond generating an answer from scratch such as formatting improvement, grammar and spelling correction, text style transfer, paraphrasing, text simplification, text summarization or translation.

I'm against any AI ban, but if the community wants to ban generative AI, at least just ban the case where the answer is generated from scratch. That'd solve the current contradiction.


Update 2023-06-18: SyedM.Sannan pointed me to https://meta.stackoverflow.com/a/424041/395857, where a mod wrote:

Tools like translators or grammar checkers are allowed, provided that you wrote the input to them.

Therefore, the post on the AI ban is incorrect and should be made more precise. It also seems to mean that the formatting assistant is allowed to, since the user wrote the input to the system.

48
  • 20
    If that's your proposal, please explain in detail how we can practically implement such a policy? In particular, explain how such a policy can be implemented in a manner which A) is within the human resources available to the site, including both moderators and non-moderators, and how that scales to the volume of content which SO sees; and B) will keep 99%+ of AI-generated content off of the site. Also note that a large number of people will not obey any policies which you propose, and that you have no resources from Stack Overflow developers to make any changes to the site.
    – Makyen Mod
    Commented Jun 17, 2023 at 12:48
  • 4
    @Makyen one could make the same arguments against banning all AI based contents as well. How would anyone know what content has been generated/retouched by AI? Just because some people claim they can tell by looking at a post? If a post is genuinely bad, close/downvote/delete/whatever for being that, not for possibly being AI.
    – user13267
    Commented Jun 17, 2023 at 13:12
  • 18
    @FranckDernoncourt Thinking that users with > 1k rep won't abuse it is very naive, because they definitely already do.
    – Makyen Mod
    Commented Jun 17, 2023 at 13:19
  • 3
    @Makyen the type of abuse you're talking about can be done by a 100k+ user. The OP is just using 1k as a reasonably good rep level for his example. "The argument that only a quality metric be used is flawed" I had the feeling reading previous posts that the metric most of these people want to use is their judgement of what is generated by AI, since even they can't practically support the idea that AI content recognizer services are accurate. And there is already a workaround for this, which is judge a post on it's merit. not on the possible tool that was used to create that post
    – user13267
    Commented Jun 17, 2023 at 13:56
  • 9
    @user13267 As far as I know, nobody significantly involved in finding AI generated content on SE has thought that AI detectors were accurate since early/mid December 2022, except the CMs/staff. We, of course, hope that changes, but we haven't seen it. OTOH, AI detectors can be of some use, similar to some of the detections which SmokeDetector uses which, historically, have a very low TP rate, even as high as 84% FP. The human evaluating it needs to know and account for the bad signal. Everything that SE staff has said about AI detectors was only news to them and is entirely a red herring.
    – Makyen Mod
    Commented Jun 17, 2023 at 14:21
  • 14
    @user13267 As has been explained, many, many times, since ChatGPT was first announced, evaluating these only on a quality metric doesn't work, due to AI generator's ability to spout "eloquent bullshit", and can't work at SO's scale with the available human resources, particularly subject matter experts. We can't get even close to the human resources which would be required. Allowing AI generated content masquerading as human generated content is antithetical to the site's goal of "working together to build a library of detailed, high-quality answers to every question about programming."
    – Makyen Mod
    Commented Jun 17, 2023 at 14:39
  • 3
    @Makyen yeah and just banning anything that might have something generated by AI in it works? As I have also pointed out in my previous comment, there's no way for anyone to tell for certain if something has been generated by AI, so your problem is still not solved. There is no reason for a group of users to get to decide all the other users don't get to use whatever tool it is that they prefer
    – user13267
    Commented Jun 17, 2023 at 14:47
  • 11
    @user13267 Yes, banning AI generated content has, generally, been working. Yes, there is definitely reasons for a group of users to get to decide that others can't dump work which they haven't actually done onto the site. A) It is company policy, as codified in the Code of Conduct: Inauthentic usage policy, that all work by such tools must be properly referenced; and
    – Makyen Mod
    Commented Jun 17, 2023 at 17:06
  • 18
    B) the "Temporary policy: Generative AI (e.g., ChatGPT) is banned" has overwhelming support, with 98.11% of users who voted approving the policy (3850 up, 74 down). Even when including responses from users who can't actually vote (1,362 up, 243 down, or 84.86% approval of just those who can't vote), the total approval rate for the AI -generated-content ban is 94.27% (5,212 up, 317 down). These sites, generally, operate on a consensus of the users. The approval of the policy to ban AI -generated content is one of the clearest consensuses we have.
    – Makyen Mod
    Commented Jun 17, 2023 at 17:06
  • 27
    Yes, it is typical for people who aren't getting everything they want to call the situation "toxic", when that's not even close to the real situation. It's unfortunate that people reach for that word merely because they aren't getting everything they want, because it significantly devalues actual issues of real toxicity.
    – Makyen Mod
    Commented Jun 17, 2023 at 17:33
  • 21
    @prusswan The score on the "Temporary policy: Generative AI (e.g., ChatGPT) is banned" post is the highest score ever on Meta Stack Overflow. It's been featured for 6 months, has appeared on every question page for that entire time. There was an all-page, site-wide banner displayed for 2 weeks. If that isn't enough to indicate a consensus, then nothing is, and we should pack-in the entire concept of running the site by consensus.
    – Makyen Mod
    Commented Jun 17, 2023 at 19:04
  • 5
    @prusswan No, SE definitely can't ban 3,850 users, particularly productive and engaged users, without batting an eye, not even close. In fact, SE's claims about the company's new, secret, AI-generated-content policy, about which many SE mods and users are on strike across all sites, are, according to SE, entirely about moderators giving, over a 6 month period, a short suspension (1 to 7 days; which SE authorized and encouraged moderators to do) once to each of about that many users who aren't already productive and engaged with the platform, so a big deal.
    – Makyen Mod
    Commented Jun 17, 2023 at 19:10
  • 12
    @user13267 SE's case would be a lot stronger if they could/would point to even a single moderator action which they think was wrong. They can't/won't. We've asked them repeatedly to show us where they think we're doing things wrong, but they haven't provided any. We want to know. We want to learn. Moderators know we're not perfect. We know we make mistakes. We can point to a few where we've reversed the action, ether on our own after review with another moderator, or where a user has replied to the moderator message and requested things be re-reviewed, but SE refuses.
    – Makyen Mod
    Commented Jun 17, 2023 at 23:03
  • 7
    @prusswan "4000 is far from representative of all SO users, this is plain as day." Amazing. Next you will tell me that political polling can't possibly ever give meaningful results. Alternately: if 3855 votes in favour don't mean anything, why should we then give any consideration to the position against, which only gathered 74 votes? Commented Jun 19, 2023 at 8:13
  • 6
    Because it's against the site rules. You can tell it's against the rules because it says so in the help center. Note that only the company can edit that page. The dispute with the company is over the manner in which we are permitted to moderate it, not whether it is forbidden in the first place, which, again, it is. If there is something unclear about this, you can ask a meta question about it, but please stop discussing it on answers.
    – Ryan M Mod
    Commented Jun 19, 2023 at 20:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .