112

Recently Stack Exchange announced a new network-wide policy regarding AI-generated content: What is the network policy regarding AI Generated content?

In short: moderators are prohibited from banning users who post AI-generated answers.

This announcement caused outcry among moderators and curators: many sites of the network have rules prohibiting AI answers (and for good reason IMO) and those rules suddenly became unenforceable. Some additional reasons for displeasure were the fact that the announcement is not featured (so that it doesn't appear in the right sidebar or sites), and it feels like an attempt to hide the policy.

I feel that the new policy should be brought to wide attention and discussed by the community, and hope this post can be a signpost for the new policy and related discussions.

As of now behavior of users flagging AI generated post should not change.

Important strike has began: Announcement of the strike. (Mirror on global meta)


Related links:

2
  • 3
    "In short:" I'm not sure the summary is accurate. As far as I understood the meta posts, the publicly and privately declared policy is not the same and there is generally some ambiguity. It may be better not to set in stone a specific interpretation. Commented Jun 2, 2023 at 12:56
  • 13
    @MisterMiyagi, I don't have access to private announcement, so I don't know. But prohibition of banning users with chatGPT answers is quite clear to me, and widely interpreted this way network-wide.
    – markalex
    Commented Jun 2, 2023 at 13:00

4 Answers 4

82

Just some thoughts of mine on the policy...

Issues

  • One of the reasons that AI content is thriving is because it functions partially as an extremely easy to use search and collation engine, one that has successfully extracted and presented much of the content from this site.
  • And there has long been a huge unmet need for this type tool since the site's own search capabilities have been poor, something that many of us have complained about, many times, in the past.
  • This drives eyeballs away from the site, since programmers no longer have to search the site directly, leading to loss of profits for the company, something that may be partially behind recent policy changes.
  • The site has thrived because volunteers are willing to share knowledge that they have with others.
  • Much of the motivation for users to do this comes from pure altruism.
  • But a significant portion of this motivation comes from gamification of participation, which has always been an essential component of the site.
  • Allowing for AI generated answers with little control is equivalent to allowing cheats in a massively multiplayer online game, and this will likely demotivate many to offer contributions.
  • This can lead to AI generated content increasingly replacing original human-generated answers.
  • Which will lead the site to degenerate into an echo chamber of old content, which is the main source of the "new" AI-generated content.
  • AI detection can never be done with 100% specificity and sensitivity, and in fact, is not easy to do, is fraught with potential errors, and will likely get harder to do as AI improves, which it has been continuously doing.
  • The site is profit-driven and its motivation will always be the realization of both short-term and long-term profit.
  • While the site is meant to be a question and answer repository, many use it as a personal help site and most don't understand the difference.
  • I'm not sure how that the corporation cares about this distinction, as long as people continue to want to visit the site.
  • Detection of AI may develop into an arms race.

Possible Solutions

  • Have the site embrace AI, but not for generation of new content, which should be aggressively and effectively prevented, but rather to use it specifically in a new and enhanced site search tool.
  • Use AI to help with new-member question creation, to help guide them to decompose their problems, and to suggest similar Q&A's that help solve each sub-step of their problems.
  • There's been much discussion of a moderation "strike" as a way of encouraging the site owners to modify moderation rules to be more in line with those of the official site moderators and the non-moderator site members who are heavily invested in site moderation. I have to wonder though how this sort of action would affect the site's bottom line, quite possibly not much.
15
  • 2
    I couldn't agree more. One more thing I'd suggest, is using some kind of AI answer as a question wizard step. Something like "maybe this will help: ....". It will increase general quality of questions by (partially) removing low quality "how to add button to <framework>" and "give me an SQL for this data"
    – markalex
    Commented Jun 2, 2023 at 13:52
  • 7
    In general I believe SE should embrace AI, but in a very controlled and adequate, not "post anything you like" way.
    – markalex
    Commented Jun 2, 2023 at 13:53
  • May I ask you, what is the real value of the internet points we call "reputation"? I disagree that creating content with the pure intent of collecting internet points would be inherently bad.
    – peterh
    Commented Jun 2, 2023 at 14:12
  • 25
    @peterh, shows level of competency (and sometimes relate to trust of community). Also they strongly related to level of permissions. I don't want user who fgitw'ed 100 questions trough chatGPT to edit post left and right. And even less so, eventually to dupe-hammer and dupe-unhammer question. Do you?
    – markalex
    Commented Jun 2, 2023 at 14:25
  • 8
    Great analysis. Only problem is that the solution is easier said than done. Even detecting AI generated content isn't simple or reliable. We would need something like "verified human generated content" and I don't see how that would be implemented nowadays. What I think we really need is a laser-sharp focus on quality. Better to remove all answers we aren't sure are absolutely right, maybe just convert them to comments. The existential issue is the quality of the posted content (whoever delivers it). Commented Jun 2, 2023 at 14:31
  • 3
    @peterh: there is no real value, only perceived value, but that matters to many. For some it provides an ego boost. Others may feel that a high reputation listed on a CV may enhance their employability (not likely true in my opinion) Commented Jun 2, 2023 at 14:41
  • 5
    When using an AI searching tool, we should also pay attention that the author gets the right credit (and votes) for their work. Otherwise, the whole system of the site will break too.
    – The_spider
    Commented Jun 2, 2023 at 15:12
  • 4
    @peterh - w.r.t. "real value of internet points we call 'reputation'" - admittedly nothing, as yet, but just wait: I'm about to start minting a rep-points NFT so we can tap into the magic web3 money ...
    – davidbak
    Commented Jun 2, 2023 at 16:06
  • 2
    @DontKnowMuchButGettingBetter Yes, also I have made a little bit of... "public opinion research" in the circles of the HR guys, bosses having effect to personal decisions. The result was that they have absolutely no idea what is reputation or how this site is working et al. They know that it exists and Q&A for developers.
    – peterh
    Commented Jun 3, 2023 at 3:58
  • 5
    "Allowing for AI generated answers with little control is equivalent to allowing cheats" this is true for newer users, but mostly inconsequential to high-rep users which is much of the meta peeps. I think people here just hate seeing crap, particularly plagiarized crap.
    – Passer By
    Commented Jun 3, 2023 at 12:44
  • 2
    'Which will lead the site to degenerate into an echo chamber of old content, which is the main source of the "new" AI-generated content.' I think this is a very good point. Commented Jun 3, 2023 at 17:21
  • I knew everybody was gonna give it to chat GPT at some point. It can even debug errors in code pasted to it
    – user16612111
    Commented Jun 4, 2023 at 12:01
  • 7
    @PassionateSE Many of us are avid users of ChatGPT for getting answers to our own questions, in private. The problem is with AI-generated content which the user has posted here as an answer, but not vetted for accuracy, often because they have no idea whether or not it's correct. A proper assessment requires in-depth subject-matter expertise and time; the temptation to cheat to earn points is pushing many users to completely ignore this in pursuit of easy and undeserved reputation points.
    – tripleee
    Commented Jun 5, 2023 at 4:26
  • @triplee, yeah that is also abusing chat GPT by itself.
    – user16612111
    Commented Jun 5, 2023 at 4:30
  • 3
    Moderators at strike meta.stackexchange.com/questions/389811/… - so I also will not do any content/curation-related action here - except upvoting this :)
    – kleopatra
    Commented Jun 5, 2023 at 5:10
63

It's been apparent for some time that Stack Overflow's goals are mostly unrelated to those of community members who want to build and curate a library of useful high-quality answers to various questions. So it's rare that any of the changes they make help with that. (wizzwizz4's answer on Stack Exchange is failing its community cites many examples of SO actively ignoring community feedback and making the site worse.)

But this appears to be crossing the line into actively opposing community moderation, forcing us to keep crap, rather than just be more polite while downvoting it, or making the close reasons less clear, or things like that which most of us can put up with. (Being polite actually is a good policy, even when it's likely the recipient rudely doesn't respect other people's time or the community they're posting in.)

I wasn't happy about many previous SO changes, but they never seemed like they'd make the site unusable for me personally, especially for the corners of it that I cared about and spend time caring for.

But this feels different. Previously they were mostly just providing a platform for the community to moderate as we saw fit, without any really onerous restrictions. (At least none that I knew of, but I'm not a moderator and maybe have missed some discussions of previous policies that restricted how moderators could moderate. Update: thanks @Cody Gray♦ for confirming that this is the first time SO has really tied the hands of moderators' use of their tools.)

This seems to me like the difference between a newspaper owner having shoddy offices or something vs. interfering with the editors and journalists. I know SO's community and elected moderators didn't have the equivalent of editorial independence on the platform provided by SO even before this, so it's not a perfect analogy. I'm just trying convey that this feels like a major heavy-handed new level of interference, of the same level of significance as the owners of a newspaper telling the editors what they can and can't publish.

It's their platform so they can make rules like this, but we don't have to keep using it.

This is the first time I've seen a policy from SO that made me think the site has finally gone fully off the rails and that I should be spending my time contributing to a different Q&A site.

Previously, after events like Monica-gate and the surrounding dishonesty by Stack Overflow, I realized there were big problems. But I expected the SO community would still survive, or at worst be in for a death by a thousand cuts from short-sighted profit-driven policies aimed at boosting clicks and site traffic (like new question posts) in the short run, not helping people find answers without having to bother other humans by posting new questions. I didn't expect such blatant active interference with obviously-useful moderator activity trying to hold the site together and keep it useful despite the corporate policies that encourage crap to get posted with as small a barrier to entry as possible, whether or not it will be well received.

This is a new angle of attack, acting in favour of those making posts most of the community doesn't want, so they'll be able to make more of them. This is unacceptable. What's next, removing the posting rate limits for low-rep users whose posts are downvoted?

Being dishonest about the actual policy is not a new low for Stack Overflow; IIRC they previously lied about discussions with Monica before their stupendously poorly-timed banning of her. But it certainly doesn't help matters that they've reportedly told moderators they can't moderate in much more absolute terms than it sounds like from reading the policy announcement (What is the network policy regarding AI Generated content?) and the claimed justifications.

One might hope that they'd learned their lesson about honesty from the Monica incident which destroyed huge amounts of community goodwill, and made many moderators re-evaluate what they were contributing their time towards. But it seems they haven't.

The only hope I have for the situation is that there was some miscommunication with what they meant, like that moderators can still moderate posts that are obviously churned out by clueless and/or selfish users who post AI-generated garbage without having the expertise to check it carefully, or without taking the time to actually do so. (i.e. the problem posts that were the reason behind the ChatGPT ban which is still in force. The usage pattern that's the reason we can't have nice things, i.e. AI help for idea generation and/or better wording of explanations, for users who do understand the entire post they end up posting.)

Or if not miscommunication, then that SO quickly changes this bad policy to still allow moderators to use their human judgement when enforcing the ChatGPT ban and suspending users who waste everyone's time by posting AI output. (Especially unchecked AI output. I don't have any inherent objection to posts where people used ChatGPT as part of their process of creating it, as long as the final result is 100% good. But I accept that we need the total ban because there's no reasonably time-efficient way to distinguish such posts from the flood of unchecked crap, compared to how much time it takes people to generate such answers. Perhaps it could work to allow people to claim that the idea or code was AI-generated, but that they've understood all of what it did. With some kind of bigger penalty than a downvote if there's solid evidence they didn't actually make a good-faith effort to do what they claimed.)

If not changed, this current policy of not being able to suspend users flood-posting AI output will quickly have strong negative effects for the overall quality of answers on SO. And regardless of exactly how big the impact is, it's simply not ok for Stack Overflow to stop the moderators from moderating.

19
  • 16
    "It's their platform so they can make rules like this, but we don't have to keep using it." This might deserve being an answer in its own right. Commented Jun 3, 2023 at 5:50
  • 2
    I can't see the line to be honest. Wasn't all the closure reason changes and welcoming nonsense the same kind of forced moderation changes?
    – Passer By
    Commented Jun 3, 2023 at 12:57
  • 20
    For what it's worth, this is an accurate description, and matches with my personal assessment of the situation, namely that the company has done a bunch of really silly, inexplicable, frustrating, and downright inappropriate crap to us many times over the years, but this is the first time they've really done anything tangible or concrete to tie moderators' hands in the use of moderation tools. That makes this a whole new level for me, and many of the other mods. I just honestly lack the energy now to write much more, so thanks for this
    – Cody Gray Mod
    Commented Jun 3, 2023 at 13:06
  • 9
    @PasserBy Changing the labels on the tins didn't stop closing questions that needed to be closed. The way the "welcoming wagon" was presented wasn't ideal, but, again, as Peter mentions, being polite (and, more importantly, being more effective in teaching people the norms/expectations of the site and hopefully turning them into productive contributors in the future by doing a better job of communicating how this place is different) is a desirable goal. I can't think of any changes that were made previously that stopped users or mods from giving garbage the boot that it needed. This does.
    – Cody Gray Mod
    Commented Jun 3, 2023 at 13:14
  • 1
    @PasserBy: Well sort of, but those changes didn't stop us from closing questions, and probably only prevented closure of some borderline questions. Really bad questions could still get closed, even if that meant choosing a reason whose text didn't fit as well. Nobody was getting suspended for picking close reasons SO corporate didn't like. The "welcoming" stuff didn't stop anyone from downvoting bad posts either; and there was a kernel of a good idea in it: don't be rude to people, even if they probably suck. In rare cases they had good intentions and the situation can be salvaged. Commented Jun 3, 2023 at 13:16
  • @CodyGray I guess I don't know how mods deal with flags, but is there no way to cope with the new requirements more or less like the way regular users close with inaccurate reasons (in response to old reasons being removed)?
    – Passer By
    Commented Jun 3, 2023 at 13:18
  • 26
    It's worth noting that this is complicated by the fact that the rules staff have given us mods in private does not match the guidance that they [finally] posted on MSE. There are nuances to the private guidance and nitpicky rules that are not mentioned in the MSE post. And it's been made very clear that trying to find loopholes ("malicious compliance", effectively) or totally disregarding the policies (akin to closing with inaccurate reasons…which regular users shouldn't have done or be doing, either—shame) will not be allowed.
    – Cody Gray Mod
    Commented Jun 3, 2023 at 13:23
  • 3
    @CodyGray That is... not a small detail they chose to omit.
    – Passer By
    Commented Jun 3, 2023 at 13:24
  • 17
    There are, of course, probably ways that we could thread the needle, like getting regular users involved to more heavily moderate the suspected AI-generated posts, since the [current] rules only apply to mods, but that's just not ideal for about a dozen reasons, and there's no guarantee that it isn't a loophole that will be eventually closed. And then there are compromise solutions, which I'm sure that we can find. The frustrating thing is we could have found them if staff had just talked to us before announcing a draconian new policy, and this whole situation could have been avoided.
    – Cody Gray Mod
    Commented Jun 3, 2023 at 13:24
  • 24
    @PassionateSE: IDK what rock you've been living under, but the majority of the community agrees with the mods on this, at least that bad posts with wrong information from AIs are a problem. Not everyone agrees with a total ban, but there is clearly a real problem with people posting bad copy/pasted answers as fast as a couple minutes per answer. I've personally seen lots of ChatGPT answers about assembly language, performance, and computer architecture, and it frequently just makes things up and claims they're facts. Commented Jun 4, 2023 at 14:30
  • 1
    You've made numerous brilliant posts, Peter, on both the main site and on Meta. This is up there with the best of them! Commented Jun 5, 2023 at 14:52
  • 5
    @AlexCeleste: I agree, posting AI output without fully understanding it is equally bad whether it happens to be right or not. There needs to be a total ban on that. But the main reason for that is that AI output is so often wrong. If AI output truly were as good as human subject-matter-expert answers in 99.9% of cases, we'd mostly be worrying about whether anyone deserves rep from this hypothetical gold-mine of high-quality answers. Or whether SO should continue to exist other than to collect FAQs in a polished way since formulating the right question is non-obvious. Commented Jun 5, 2023 at 15:21
  • 3
    @AlexCeleste: But we're very far from that hypothetical scenario, with AI explanations for performance effects being at least partly wrong in every one of maybe a dozen or two answers I've seen, often completely missing the point of the question. My comment before yours was phrased that way to make it clear that this isn't just protectionism (not wanting AI-users to steal our rep?), to explain the situation to someone who doesn't seem to realize that AI output is often garbage. That's the key fact that makes a ban necessary to keep out the pollution of bad answers. Commented Jun 5, 2023 at 15:24
  • 3
    @AlexCeleste: Where there's room in theory to be flexible is when a human has used ChatGPT to get a useful idea, but has the time and ability to understand what the code (if any) does and knows enough to spot any non-obvious pitfalls like SQL injection vulnerabilities or corner-case failures it might have. And has carefully read the text that explains the what's going on, and knows enough about the subject-matter to be sure it's right. Commented Jun 5, 2023 at 15:27
  • 1
    @AlexCeleste: So a good human-checked or partly-rewritten answer based on ChatGPT output is no less effort than writing an answer from scratch, and easier to miss errors. The benefit for some people would just be ChatGPT's "polished" writing style to convey their ideas. In theory I have no problem allowing such "honest" use of ChatGPT. But there's no way to tell whether someone actually had the expertise to check an answer, or whether they put in the time to read carefully. Maybe there's a possibility for saying "this part from ChatGPT, I think it's right but don't upvote w.o checking" Commented Jun 5, 2023 at 15:33
17

Speaking more specifically about Stack Overflow here, but it may generalize.

The value of the site relies on the notion that user votes inherently reflect quality and accuracy of information (to whatever degree it is so).

At the same time, the reality of user behaviour is that we rely on the inherent quality and accuracy of information to inform our voting (again, to whatever degree). This is because in many cases, it is not possible, or just not practical to fully vet and validate the posts we see. After a modest mental investment failing to find any deficiencies in a post, it may feel justified to vote it up, putting at least some stock in the good-faith effort of the poster and their reputation.

An example of of the previous point that has happened to me multiple times in the past: I answer a question that is confidently within my area of knowledge, and I write an answer with a short code snippet. The answer receives a few up-votes. Later, someone comments on my answer or edits it to correct a syntax mistake that had been there the whole time. The code in its original state would not even pass the compiler. Nonetheless, multiple users "validated" the answer before the mistake was caught.

The issue I see here is that an AI model's ability to produce content that looks like it is of the quality we expect from genuine human posts (thus attracting up-votes, assuming voter behaviour stays the same) is not necessarily the same as the model's ability to produce content that is actually of the quality we expect from genuine human posts.

Therefore, as the proportion of AI-generated content of the site increases, I'm guessing that voter behaviour will need to change (particularly, more effort will need to be expended), or else the value and meaning of votes will diminish.

5
  • 1
    I've even had answers with obvious syntax errors accepted without any comment.
    – tripleee
    Commented Jun 5, 2023 at 4:12
  • 1
    "voter behaviour will need to change (particularly, more effort will need to be expended)" I think that would have been good all the time. Even already in the past, not being careful with votes could have resulted in degrading quality. Now the question is if people do really change or if the voting will always remain that shallow from most of the users. I think that this behavior is unlikely to change. On the other hand AI content scanners are unlikely to become much better. I conclude that we are all doomed, unless AI becomes better, which might take a couple of years or more. Commented Jun 5, 2023 at 6:43
  • 3
    @tripleee I'd accept such an answer if it put me on the way to the solution to my problem, and possibly wouldn't comment if that'd be too much work (not easy to write code in a comment after all)
    – jwenting
    Commented Jun 5, 2023 at 9:30
  • 3
    Even just "it worked after I fixed some errors" would help draw attention to the fact that there is a problem.
    – tripleee
    Commented Jun 5, 2023 at 10:45
  • 4
    I had this with an AI-written answer yesterday - it looked great, well-laid out, some helpful code snippets and I was grateful to have an answer quickly so I upvoted it. Then when I tried to actually use some of the code, it was based on references to a property that simply doesn't exist, but I couldn't downvote it without the answer being edited. Fortunately when I pointed out that there was a problem with it they responded with a classic GPT "my bad..." and edited to a different incorrect answer, so I could downvote, whereupon they deleted the answer. The system is being gamed, currently.
    – glenatron
    Commented Jun 8, 2023 at 10:16
10

If posting AI-generated answers is not permitted, but doing so has NO consequences for those that post them... then the policy disallowing AI-generated answers is utterly pointless.

And I'd advocate that we (the users) should cease any attempts to enforce it. If you spot any answer that is obviously AI generated? JUST IGNORE IT.

Because an AI only need 5s to generate another such answer, and the user who posts the answer only needs 2s to copy-paste another question and the generated answer, so investing ANY of your time is an obvious waste of effort.

If the StackOverflow/StackExchange staff does not wish to sanction users who post AI generated answers, then why should we care to clean up the mess said users create - let the site drown in low-quality, low-accuracy AI-generated non-answers. Perhaps they need a reminder to appreciate the volunteer work all of us put in.

4
  • This strategy will inevitably lead to filling up answers section with junk, impossible to differentiate from valid answers without proficiency in questions topic, and as a result were low number of users will have a chance to get something useful. And that kinda defeats the purpose of this site.
    – markalex
    Commented Jun 5, 2023 at 10:46
  • 3
    In my opinion, from this post is exactly one step to "If SE wants to fill up their sites with junk, let's not use their sites." But I hope, current situation can be solved less radically.
    – markalex
    Commented Jun 5, 2023 at 10:48
  • 1
    Well, is question is WHAT can we do. We already loudly argued that this policy of not-sanctioning users who post AI generated content is short-sighted at best, downright counterproductive at worst. But chances are that SE staff will - like with many other problematic issues - continue to ignore the userbase. And attempting to bail out water out of the boat - while the captain orders to turn a blind eye towards those who drill holes in it - is both frustrating and futile.
    – CharonX
    Commented Jun 5, 2023 at 11:15
  • 4
    I will not ignore chatGPT answers when I see them. I will at least downvote them, and possibly more than that. Commented Jun 5, 2023 at 14:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .