133

Update

The A/B test went live the morning of 21 Sept 2023 as did the new AI policy Help Center page. If you have any feedback or questions during this test, feel free to use an answer to this question to do so.

Summary

We are planning to run an A/B test for two weeks on Stack Overflow in late September to draw attention to the site-specific policy about the use of AI-Generated content when creating answers on the site.


As part of the agreements made during negotiations, we took the concerns raised during that discussion and worked on a design that we plan to A/B test here on Stack Overflow to see the effect it has on answering. We shared a preliminary design and plan with moderators a couple of weeks ago and are moving forward with sharing the plans for this project here on MSO. While the design is in development to be built, we still have time before we start the test to make adjustments to the text in the notice.

The Test Design

We went through a few variations for placement and functionality to get to the design we're proposing below. For the duration of the test, we'll remove the "new contributor" banner (example) on all answer fields for both test groups with the control group seeing nothing and the test group seeing the design below as they use the same space. Based on the outcomes of this test, we'll determine next steps for the "new contributor" banner.

We are planning to show this to all user types in the test group in the "Your Answer" area of the question page. Here's what the design we're going to test looks like:

The editor showing the banner design as described in the following paragraph.

Directly under the formatting bar, there's a blue bar with text that reads "Reminder: AI-generated answers are not allowed on Stack Overflow." It includes a "Learn more" link to the Help Center page about AI-generated content.

For the duration of the experiment we don't plan to make the banner dismissable as we want to know that the test group was shown the banner each time they posted an answer.

The Testing Phase

To test this design, we plan to do a two-week A/B test so that we can compare the new design with the current one and see if people in the new group have a reduced likelihood of posting AI-generated answers. We are currently only planning to run this test on Stack Overflow as it is one of the sites driving this change and the higher volume should help us assess the impact more quickly but are open to testing on or shipping to other sites once the initial tests are completed.

The metrics we plan to review include:

  • Reduction in posts containing AI-generated content as determined by the number of validated flags and mod messages sent to users in the test group.
  • Answer rate from test group drops at a rate consistent with the above metric with some allowance for cases that may go unflagged.

We will wait a week after the test ends before reviewing the data to allow community members time to flag and mods to handle those flags.

What we need from you

While we're interested in your thoughts about this test, we're planning to start with the design you see here. Based on the results, we'll determine the next steps based on whether we're seeing an impact in the test group.

As noted, the current copy is a suggestion. We've considered a few alternatives but would be interested in if you have any particular suggestions you feel should be included.

Some notes for your consideration - since this will be shown to everyone in the test, the bulk of people seeing this banner likely have no plan to post AI-generated content. Going into detail about policy and reasoning should be saved for the Help Center - which we are planning to update before this test begins.

  • The text should be short.
  • The text should be informative.
  • The text should be neutral.
  • We only want to link to the Help Center page.

If you have any questions, please let us know - we really appreciate your help and look forward to working collaboratively to find a solution that meets our goals. Once this test is live, we'll update this post to keep you informed.

42
  • 23
    "We will wait a week after the test ends before reviewing the data to allow community members time to flag and mods to handle those flags.' I'm really not sure a week is going to be long enough. I have tens of flags for potential AI generated content with 1 pre-dating the strike. The team have a significant back log to go through, so it's going to take them time. A week to work through the flags isn't going to give meaningful results when, presumably, older flags are going to be being worked on more often. Unless for the entire duration, and a week afterward, focus will be on new flags?
    – Thom A
    Sep 7 at 21:28
  • 6
    All great points and I definitely understand the timing of when we review the data can be quite impactful to the results we see - I don't necessarily think we need all of the flags to be handled to accurately gauge the outcomes, though. Depending on how many flags there are and how many get handled, we may be able to project the percentage of flags that would be validated to the remaining flags from the test period. It wouldn't be precise but I'm under the impression that's a pretty standard way of reviewing data.
    – Catija StaffMod
    Sep 7 at 21:49
  • 13
    Some new users seem to have problems to express themselves (read/write) in English. I think some might not understand the term AI-generated standalone. I'd change it to Remeber: AI.generated answers (like from ChatGPT)..... ChatGPT everyone knows... Just an idea...
    – Vickel
    Sep 7 at 21:56
  • 24
    I think the phrasing of the warning strikes a good balance between 1) being brief enough to not annoy legitimate users and 2) clearly articulating the policy.
    – Nick ODell
    Sep 7 at 22:05
  • 8
    Thanks - @Vickel I see your point. I do think we want to avoid naming specific companies in the UI directly but we can certainly include a non-exhaustive list in the Help Center information. I think we're open to alternative terminology if there's better options. :)
    – Catija StaffMod
    Sep 7 at 22:05
  • 5
    @Vickel Would you mind putting a bit more detail into your concerns and making them into an answer? I think it's worth keeping in mind and I'm worried they'll get lost in the comments.
    – Catija StaffMod
    Sep 7 at 22:26
  • 22
    @philipxy Describing images in text is generally considered a good practice, both for users of accessibility services and those who may be behind a firewall that blocks Imgur. While alt text is one way to do it, it can be nicer if it's worked into the normal text as well.
    – Ryan M Mod
    Sep 7 at 22:44
  • 19
    @philipxy I mean - I did intentionally not use a freehand red circle because I was trying to avoid that exact response - amusingly, I was doing the calculus of whether I'd get more points for the freehand circle or lose them for "having" to circle the banner to make it more obvious to people. Oh well! The description is not actually for people who can see the images, though - I opted to explain it in the post rather than hiding the image description in the alt text.
    – Catija StaffMod
    Sep 7 at 22:45
  • 7
    @philipxy so, here's what I would have said if you had taken a more constructive tack and maybe asked about our design decisions rather than just calling them "bad" - as noted in the question, we did go through several designs and the feedback from mods did indicate some concern about it being easily overlooked. The goal of this test is to see whether something subtle but insistent can change behavior rather than immediately jumping to options that might be more overt but potentially more likely to cause a reduction in good answers. If it seems to go unnoticed, we can try something else.
    – Catija StaffMod
    Sep 8 at 1:43
  • 6
    @user13267 this text is for SO only at this point and will only be enabled on sites that expressly ask for it because they have policy that prohibits this content. The banner/notification was requested by mods and SO users repeatedly because the policy prohibiting the AI-generated content was easily missed and causing a negative impact on curators and moderators. My understanding is that some of the people who are mod messaged about this reply that they were unaware of the policy and some even apologize and state they will stop. As such, user education is needed, at least in the short term.
    – Catija StaffMod
    Sep 8 at 1:52
  • 7
    We generally require an indication from the community that such changes are deemed necessary - usually in the form of a meta post, @user13267. We can not perpetually have a banner on the entire site to notify users about this policy. Mods don't want to keep the policy featured on meta and that bulletin isn't visible on all viewports anyway - that's why we're doing this at all. It's a test. As has been stated, if there's no difference in behavior between the control and test groups, that's useful information. But having a banner up six months ago doesn't mean today's posters saw it.
    – Catija StaffMod
    Sep 8 at 2:14
  • 6
    @PetəíŕdTheLinuxWizard that's how we generally run A/B tests, yes. I have no clue what algorithm we use or why it might matter.
    – Catija StaffMod
    Sep 8 at 5:36
  • 6
    @user13267 "Now that there is a standard (heuristics) to fall back on when actioning a post for (possibly) being AI, why still this debate of allowed and not allowed?" Because we want to prevent users posting AI posts in the first place and reducing the workload. All AI is banned because it is not feasible determining the accuracy of every AI answer. This is not the only reason, but before we can even discuss others, we would need the solution for that one. SO community voted for banning AI posts with significant majority meta.stackoverflow.com/questions/421831 Sep 8 at 6:54
  • 6
    @user13267 You have every right to state your opinion, and just the same I have the right to state mine. I am not dragging you into pointless discussions, it is you who are dragging everyone else. Sep 8 at 7:59
  • 10
    Progress! Really thankful for all the hardwork that the people behind the scenes put in here, both on the community end and the corporate end. Thank you.
    – Travis J
    Sep 8 at 19:06

3 Answers 3

67

As noted, the current copy is a suggestion. We've considered a few alternatives but would be interested in if you have any particular suggestions you feel should be included.

as suggested by @Catija and to sum up my comments in the OP:

  • Some new users seem to have problems to express themselves (read/write) in English. I think some might not understand the term AI-generated standalone. I'd change it to:

    Remember: AI-generated answers (like from ChatGPT)....

    because ChatGPT everyone knows...

I've observed the lack of language knowledge many times during my many years here on SO. Communication with users who are not native/2nd language speakers is a major problem.

  • Remember AI in German is KI (Künstliche Intelligenz), in Spanish+Portuguese it is IA (Inteligência Artificial), I have no idea how it could sound in languages I don't speak. People are used to their "local" abbreviation of Artificial Intelligence and might simply miss (in the sense of overreading/ignoring it) or misunderstand/don't understand it in English.
10
  • 27
    I think this is an excellent point, particularly since we have evidence from the company that suggests questions that have historically been deleted as being AI generated are more likely to originate from countries where English is not the primary language. We should do everything we can to make the banner understandable for users whose first language is not English. Sep 7 at 22:45
  • 15
    Maybe writing it out would help? "Reminder: Answers generated by artificial intelligence (AI) are not allowed on Stack Overflow."
    – wjandrea
    Sep 8 at 4:09
  • 1
    FWIW, in French it's also IA and it seems to be the equivalent of II in Russian. To get foreign terms for things, I like to go to the Wikipedia page, look at the list of other languages, and hover over them to see the article title.
    – wjandrea
    Sep 8 at 4:22
  • 4
    The style guide says "Write for a 7th grade reading level.". For example, "Don’t complicate your ideas with business or insider jargon. Only use jargon when you have reason to believe it will improve user understanding.". I don't know if it applies here or not. Sep 8 at 15:02
  • Unless you’re Norwegian, and hear «AI» much more than «KI»… :( Damn, can my fellow people learn how to use their own language? Sep 8 at 16:39
  • Yep indeed, 'AI'="IA" also in French (= "Intelligence Artificielle")... // Typo in the Answer, ... quickly corrected...
    – chivracq
    Sep 9 at 2:26
  • It should be "Machine Learning" instead. AI is the general term - we had AI before ML - ChatGPT is specifically using ML. Sep 9 at 7:15
  • 2
    @wjandrea "Reminder: Answers generated by machine learning models are not allowed on Stack Overflow." Sep 9 at 7:21
  • Since SO doesn't want to mention any specific product names in the banner (understandably), what about saying "GPT"? It does "resemble" ChatGPT and it is simply a NN model.
    – M--
    Sep 13 at 15:36
  • Is "Reminder: Answers generated by artificial intelligence (AI) language models such as ChatGPT are not allowed on Stack Overflow." too long?
    – Someone
    Sep 13 at 20:23
1

The term used should be "language model" - because it is more accurate.

In this vein I suggest the message be:

Reminder: Answers from ChatGPT or other language models are not allowed.

And maybe like a short wiki describing what hallucinations are and for that including the case where a lawyer made a fool of himself using ChatGPT.

10
  • 24
    I think that "machine learning" is potentially too technical a term, and people may not understand that LLMs are "machine learning". I would believe that they're more likely to understand that they are "artificial intelligence". (also, in the event that someone decides to generate answers by some sort of non-ML "AI", then...that also seems bad)
    – Ryan M Mod
    Sep 9 at 7:26
  • @RyanM Yeah well I suggested an alternative variant - implying the actual term - and citing an compelling case. Sep 9 at 7:31
  • 3
    I don't think we should use that alternative, either, as we would also not permit answers from non-ChatGPT AIs.
    – Ryan M Mod
    Sep 9 at 7:48
  • 1
    @RyanM Updated my answer - and used "language models" instead. Sep 9 at 7:51
  • 3
    "Language models" is perfect :D
    – Cerbrus
    Sep 9 at 7:51
  • 2
    The term used in the community policy is AI, specifically "generative AI" but in this context I assume "generated" implies "generative", though this isn't my field. On the other hand, the official policy only covers GPT.
    – wjandrea
    Sep 9 at 13:40
  • 2
    A rose by any other name. Why does the term matter since no AI generated content is allowed - the purpose of the banner isn't to provide some technically precise definition. If you can get some video game to generate SO questions then that's very likely off-topic and against policy too. There exists no situation where something AI-related can generate a valid SO question, so there's no need to split hairs over it.
    – Lundin
    Sep 12 at 13:37
  • 1
    Instead of getting mired in the technical details of what term is both broadly-understood and fully accurate (goals that are often at odds), especially because answers generated by any future or past artificial agent are presumably not allowed, regardless of the technical details, why not focus on what we do want? "All answers must be written by a human" or something similar.
    – lmhamilt
    Sep 12 at 20:56
  • @lmhamilt Well so we are technically accurate and not confusing. - besides I think the term "Language models" is a good fit. Sep 12 at 23:27
  • I work with AI professionally and I wouldn't describe ChatGPT as a "language model", at least not colloquially (but as a technical description, sure). AI and ML are the much more recognised terms (although "language model" has been gaining popularity). AI is more recognised than ML, and while AI is broad, it's also specific enough: I don't think anyone will try to post a self-driving-car-generated answer any time soon, but the distinction between pasting text as is from ChatGPT versus writing an answer based on the output of some model should be clarified under the "Learn more" link.
    – NotThatGuy
    Sep 14 at 13:51
-5

Who will test it? Will you select a random people? If so, what algorithm? –

that's how we generally run A/B tests, yes. I have no clue what algorithm we use or why it might matter.

Running a test for a hypothesis (that users shown the banner will post less AI generated answers) surely needs a better selection method than random? My thoughts would be to include the reputation of users asking question in test data, and assign members of the test population to A and B groups more or less evenly based on some random property of their profile data - say the odd/evenness of a checksum of their email address (not itself included in test results of course).

Test results might then be able to reveal segments of the community more likely than others to post AI generated comment and how likely they are to respond to information in a banner.

And if it doesn't consume too many resources would it not be possible to include everyone posting an answer in the survey?

2
  • 17
    No, to test a hypotheses in a statically defensible way, you need to make truly random selection, otherwise whatever bias you have in your selection might be an explanation for the results you are seeing.
    – MegaIng
    Sep 11 at 15:22
  • @MegaIng I agree and that while I was trying to clear up what the process for selection would be, it was poorly worded for the thoughts behind the post. Forgetting about checksums and email addresses, if user numbers are issued sequentially I would think odd/evenness of user numbers is random - unless my user number ending in "2" is part of a non random process. If it's even/oddness is random, it could be used to determine if questioners are shown the AI banned banner or not. I would hope that SO database records could then provide data for analyzing any number of hypotheses in the future.
    – traktor
    Sep 11 at 23:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .