69

I just want to point out this recent paper on arXiv: Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow, by Maria del Rio-Chanona, Nadzeya Laurentsyeva, Johannes Wachs, submitted 14 Jul 2023.

Here's its abstract:

Large language models like ChatGPT efficiently provide users with information about various topics, presenting a potential substitute for searching the web and asking people for help online. But since users interact privately with the model, these models may drastically reduce the amount of publicly available human-generated data and knowledge resources. This substitution can present a significant problem in securing training data for future models. In this work, we investigate how the release of ChatGPT changed human-generated open data on the web by analyzing the activity on Stack Overflow, the leading online Q&A platform for computer programming. We find that relative to its Russian and Chinese counterparts, where access to ChatGPT is limited, and to similar forums for mathematics, where ChatGPT is less capable, activity on Stack Overflow significantly decreased. A difference-in-differences model estimates a 16% decrease in weekly posts on Stack Overflow. This effect increases in magnitude over time, and is larger for posts related to the most widely used programming languages. Posts made after ChatGPT get similar voting scores than before, suggesting that ChatGPT is not merely displacing duplicate or low-quality content. These results suggest that more users are adopting large language models to answer questions and they are better substitutes for Stack Overflow for languages for which they have more training data. Using models like ChatGPT may be more efficient for solving certain programming problems, but its widespread adoption and the resulting shift away from public exchange on the web will limit the open data people and models can learn from in the future.

I'm fairly sure the users here would be highly interested in it. Perhaps you'd like to share your thoughts on it.


I gave it a quick read, and these observations seem worth highlighting here:

The authors observe a substantial drop in Stack Overflow posts after the launch of ChatGPT, which they didn't find at other sites (e.g. math.SE):

enter image description here

They estimate "a 16% decrease in weekly posts on Stack Overflow" which subsequently worsened to 25%:

By the end of April 2023, the estimated effect stabilizes at around 25%. Interestingly, ChatGPT use, in general, peaked around this time.

and observe that some tags , , and were more affected than others (the authors also mention JavaScript).

Regarding voting, they write:

Indeed the relative stability of voting behavior suggests that the quality of posts on Stack Overflow has not meaningfully changed after the introduction of ChatGPT.

One main discussion point is that GenAI-written posts make publicly available Q&A less useful for training GenAI. Moreover, while private use of GenAI (such as users asking GenAI the questions directly instead of asking on Stack Overflow) will offset this GenAI-training problem, this will allow major GenAIs to monopolize knowledge that would otherwise be freely available.

6
  • 24
    "Posts made after ChatGPT get similar voting scores than before, suggesting that ChatGPT is not merely displacing duplicate or low-quality content…" Seems naïve to assume that such posts are always downvoted. We might not even be able to use close vote data as a measure, since (even ignoring the strike) it could just show that reviewers have always been spread too thin.
    – Laurel
    Commented Jul 19, 2023 at 1:25
  • 4
    What is "Server fault (Chinese)" ?
    – user13267
    Commented Jul 19, 2023 at 1:50
  • 9
    @user13267 It's Segmentfault, and not a Stack Exchange site. Commented Jul 19, 2023 at 1:59
  • 17
    The main deficiency is that they don't consider closure or deletion when looking at question quality, only votes, which makes that analysis pretty much nonsensical. I also could not find whether they consider deleted posts at all, and since data from other sites are retrieved by scraping, they can't for all sites. Would be interesting to see their scripts and be able to check how it's performed exactly and if it's robust to changes, but they just say "soon", which is weird for an academic paper. They also don't split on questions/answers per question which is a pity
    – Erik A
    Commented Jul 19, 2023 at 7:20
  • 16
    i'm a regular responder on pyspark tag, and i can see the number of posts have reduced compared to last year (same period). i feel the quality of the problems has gotten better as not many askers are interested in asking for code help (e.g., write my code). a lot of it has become understanding and infra related.
    – samkart
    Commented Jul 19, 2023 at 8:14
  • 34
    It's almost as if all the GenAI hype tricked people into doing what they should have been doing all along -- some research rather than directly dumping trivial duplicates on SO. Seems like a win, at least from the standpoint of the original SO mission.
    – Dan Mašek
    Commented Jul 19, 2023 at 14:56

8 Answers 8

63

But since users interact privately with the model, these models may drastically reduce the amount of publicly available human-generated data and knowledge resources.

People that no longer use Stack Overflow and use Gen AI instead can only do that because their problems are duplicates, that we already answered zillion times on Stack Overflow and across the Internet.

Plenty of Stack Overflow traffic consists of useless duplicates, that don't contribute to the knowledge base. Removing those will not reduce existing knowledge.

The only problem it may pose to LLM is that duplicate data can increase statistical significance for certain solutions, so Gen AI could be less reliable in the future if the only metrics it uses for training is vast number of duplicate solutions to the same problem.

But, that is potentially a problem for Gen AI (not unsolvable one), and is not a problem for Stack Overflow or other sites in the network. Yes, it may reduce some of the traffic, but plenty of that was "bad" traffic anyway.

Gen AI is not suitable for solving programming problems, regardless of what people think and given enough time, more and more people will realize this - they just need to get burned by the AI few times before that.

36
  • 26
    Good points here. The best way to use AI to improve Stack Overflow is... to let people use it off site, realize their questions are trivial, and thus have them stop adding to the garbage heap. The downside of this, of course, is that it validates the hordes of clueless people who think this proves something negative about Stack Overflow, or demonstrates the "intelligence" of large language models. At least, in their own minds. Commented Jul 19, 2023 at 9:00
  • 8
    "…they just need to get burned by the AI few times before that." Ah, but it's not going to be them who gets burned but us, the people who will be unwittingly using the software people like that make.
    – Laurel
    Commented Jul 19, 2023 at 13:09
  • 1
    @Laurel Most likely. But plenty of people also have some supervisors or teachers and will get stuck in some AI crap sooner or later.
    – Dalija Prasnikar Mod
    Commented Jul 19, 2023 at 13:25
  • 5
    Guys, people have been saying the same thing about Stack Overflow and "it's mindless copy-paste culture". Get with the change. GenAI is a tool, you can use it or misuse it.
    – Alex
    Commented Jul 20, 2023 at 6:59
  • 6
    @Alex If you are using Gen AI for programming then you are misusing it. It is a mindless chat bot.
    – Dalija Prasnikar Mod
    Commented Jul 20, 2023 at 10:19
  • 6
    @KarlKnechtel It's always believed me that the people most willing to believe in intelligence of LLMs, are the same people who lack sufficient intelligence to make that judgement.
    – Ian Kemp
    Commented Jul 20, 2023 at 18:47
  • 5
    "Gen AI is not suitable for solving programming problems" Fully agreed. AI is more like a search engine, not like an engineer. But I don't understand why this is a bad thing. Retrieving an existing solution is an important and useful task. One can make money with this. Also this only describes the current state of AI. Who knows what will happen in say a few years. Commented Jul 20, 2023 at 21:52
  • 3
    @KevinKrumwiede New users are supposed to search first before dumping their question on SO. If new user in established language (that has a documentation, books, tutorials, and plenty of answers on SO) needs to ask question, the chances are it will be useless duplicate. Ability to figure things out on your own is the most important trait for any developer. And if you are that kind of newbie that you really need some handholding, then you need a teacher or tutor, not SO.
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 6:52
  • 3
    If the new users are not willing to put some effort, then they are not the kind of users we want or need in order to maintain knowledgebase for the future.
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 6:53
  • 5
    @DalijaPrasnikar Other mindless tools I use for solving programming tasks: A whiteboard. Google. A text editor. An IDE. Pen and paper. Colleagues (more or less mindless). A keyboard and a mouse. Sometimes even SO. Documentation. Who has ever needed to argue about the inherent "intelligence" in our tools? Chat GPT is like an awesome personalized search engine. I've been programming for almost 20 years, unlike you (?) I do not expect to be able to copy/paste random code that I find regardless if it's on a old blog in the internet, a post on SO or from a tool like Chat GPT.
    – Alex
    Commented Jul 21, 2023 at 7:43
  • 2
    Mandatory reading for anyone thinking Gen AI can be used as a tool github.com/mdn/yari/issues/9208
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 8:52
  • 2
    @NoDataDumpNoContribution I specifically said Gen AI for reason as this is kind of AI we are talking about here (and it won't be any better as far as hallucinations are concerned). Also, I don't want to talk about the future as I don't have a crystal ball. If and when there is something significantly new and different we can discuss about it then.
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 9:06
  • 2
    @DalijaPrasnikar Sorry but I can't interpret your comments other that you hate AI and want none of it. Have you actually tried using it? I know it brings a lot of value for me and for my other colleagues, you can not argue about that. Who would ever consider "using chat GTP as a tool" as copy/pasting its answers without processing it? That's just silly, you are drawing strawmen on the wall. Just like the people hating on stack overflow 10 years ago for it being a copy/paste resource for bad programmers.
    – Alex
    Commented Jul 21, 2023 at 14:07
  • 2
    @Alex I don't hate AI. I know what it can do and what it cannot do. Who would copy/paste answers without processing - just look around here. Mods have deleted thousands posts by people copy pasting answers directly from AI. also if you say you will not copy/paste AI without processing it, you are also misusing it. Because you cannot trust a word it spews out, including the code. You have fallen into the AI trap and you think you know better.
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 14:28
  • 2
    @KevinKrumwiede I am sorry if I misunderstood, but I cannot follow your logic. You are basically saying that we should ruin the site for people that find it useful, to accommodate people that think it is a joke, because some day they just might become worthwhile contributors.
    – Dalija Prasnikar Mod
    Commented Jul 21, 2023 at 19:16
36

Phew, what a read...

As far as I can tell, this is mostly (~10 pages) almost raw data and technicalities, with a bit (~3 pages, much of which is "other people said") of loosely attached discussion. Some of the references are weird – such as talking about ChatGPT with a ref from 2012 – or plain outdated – the duplicate ratio practically must have changed since 2013 – but, well, *reviewer mode deactivated*, who cares amirite?

So... I'm not sure if this brings anything new to the table, for us, here, on SO. The main data is "people post less since ChatGPT" and, well, d'uh. We have been through this a lot, to the point that SE Inc has made decisions based on that and the only thing folks can definitely agree on is that people post less since ChatGPT.
So we kinda knew that.

There is a quality metric used that is basically "votes per posts". Yes, the reviewer in me has questions – where did all the total non-relative voters go, why is no-votes-on-average a significant indicator for quality – but really they are just proxies for "meh". There isn't enough substance there to actually say things either way.

The breakdown by tags is interesting, but, well, *adjust monocle*, is it really surprising that ChatGPT impact correlates with popularity? Learning corpus and ratio of beginner problems *waves hands* and all that. Maybe, maybe not, it's not like the numbers tell us that.

So... yeah. ChatGPT and things like it exist now and are noticeable. 🤷‍♂️

3
  • The breakdown by tags is the most interesting. SO is a community of communities, do we need a policy per tag?.
    – MT1
    Commented Jul 21, 2023 at 10:02
  • @user10186832 I don’t think that makes sense. Tags are assigned by users and can be edited, so we would allow people to choose their own rules but without any guarantee that they are consistent. Commented Jul 21, 2023 at 10:03
  • 1
    most of the tags in the list are not that volatile. Tags like python and c# are essentially fixed.
    – MT1
    Commented Jul 21, 2023 at 10:07
26

One main discussion point is that GenAI-written posts make publicly available Q&A less useful for training GenAI.

Of course. GenAI currently relies on previously generated knowledge. It processes that and uses it to infer answers. If the question pertains to something that was discussed in the past, e.g. an already released Java version or questions on how to deal with a certain type of tire puncture on your bicycle, GenAI might be able to combine knowledge from various sources and produce a somewhat reasonable answer.

With current low-quality questions on Stack Overflow ("How to get started on X", "Debug mah codez pls" etc), GenAI might very well be able to get reasonably close to the correct answer. However, when the newest version of Android is released (bugs guaranteed), GenAI won't know how to solve those. It has not been trained on content related to that version and will only be able to post general debugging tips or hallucinate things which might or might not work.

Monopolising knowledge, therefore, is unlikely to happen with the current GenAIs in my opinion. It simply needs a training dataset on a certain topic to be able to answer within said topic.


To come back to the quote, yes, GenAI written posts will make publicly available Q/A less relevant for training, given it generated that knowledge in the first place. It's more regurgitation of the same knowledge base, not a new knowledge base.

1
  • "only be able to post general debugging tips" To be fair that is already something valuable. If people would add the information what came out of the general debugging iteration in their questions, we might have less work answering them. Commented Jul 20, 2023 at 21:46
21

My reaction to skimming this is that it's not really telling us anything new.

Yes, interaction with the site has been down. It's been more rapidly so with the availability of ChatGPT. But we know that.

Yes, people are able to find solutions through GenAI, and this is largely attributed to their training on the large sets of already publicly available data. As in, Stack Overflow's data was always (and should always remain) publicly available, and so training on that data was an inevitability.

What it gleefully omits is the stuff that's really the bread and butter of the problem here.

If site interaction is going down, and has been steadily decreasing since about 2016, what other factors could there have been that contributed to this increase? From 2016-2020 there wasn't a whole lot of extra...stuff...that was added to Stack Overflow that increased its utility for people looking for an answer to their question, and so naturally, it makes sense that fewer people are coming to the site to get their questions answered. It implies they're finding it somewhere else.

In 2020, the giant spike was during lockdown. Folks didn't have anything else better to do, so they stayed indoors and learned to code, or brushed up on it. On paper, Stack Overflow has always been celebrated as this great place to learn how to code, but in practice it's more a hodgepodge of incomplete solutions to relatively specific programming domains, so while I personally think the learning potential is limited, the sheer volume of information on the site still has some propensity to teach someone.

But after that initial spike, with restrictions loosening and maybe even some people deciding to pivot careers, again there's not a lot of improvements to this whole "finding answers" service of Stack Overflow's that saw much improvement between then and ChatGPT, so interaction trended low again.

Then ChatGPT hits the mainstream, and people just throw whatever they want at it. And it gives them an answer. And people find that their itch is scratched. They asked a question and something gave them an answer. No one's talking about the quality of that answer or the practicality of that answer, but the numbers don't tell you that - they just say that people got an answer somewhere else.

...If you're surprised by this, I wonder, have you been under a rock for the last decade?

While I believe it's an interesting piece, I feel strongly that the study overly conflates two very distinct things into one cause-and-effect, which isn't wholly false, but isn't wholly true, either. For years it's been easier to find answers to my questions anywhere except Stack Overflow, but it's also because the questions I have are so much more precise than Stack Overflow could offer solutions for at scale. Not everyone's going to be an expert in Spring Boot and can volunteer their time to help me troubleshoot why I'm not able to get this bean wired in properly. For the longest time, that wasn't really a problem. The value of Stack Overflow was always to provide quality answers, but the questions that were asked couldn't be as broad as say, building a whole web service.

That wasn't an issue until leadership started seeing its lunch being eaten by ChatGPT. But I'm just here to say, the plate was cleaned well before ChatGPT hit the mainstream. It's just the most obvious person that has crumbs around its mouth, is all.

Using Stack Overflow to get answers has been in decline for years. Other sites and resources exist now that do a better job of answering the broad and tricky questions, the very kind of questions that Stack Overflow pivots itself away from answering. Then, ChatGPT comes along and picks up that slack. Does it really matter in that case then if they get their answer from AI? I sure don't think that Stack Overflow needs to change how it decides what questions to answer.

11
  • 2
    "it makes sense that fewer people are coming to the site to get their questions answered. It implies they're finding it somewhere else." Or, and hear me out, it means they're finding it on SO without interacting with the site, as is intended. Shocking! There are more questions already answered on the site, so there are fewer new questions to ask! Commented Jul 20, 2023 at 15:43
  • @CrisLuengo: There's an angle to that, but I don't believe it as strongly. The questions that are useful and the answers that are pertinent to people are quite old, and the technologies that people may have asked about could have easily evolved and matured in that time frame. Angular is pretty infamous during this period as well as the advancements of React and Vue, along with Android's ever shifting ecosystem. So yes, there could have been value in what's already here. But I still think that there was better value elsewhere in the style people needed it.
    – Makoto
    Commented Jul 20, 2023 at 16:02
  • 3
    Maybe it depends on what technologies you use. I find an answer to 95% of my programming-related questions on Stack Overflow. And they are not always 10-year-old questions. Sometimes the question I have had been posted in the last few years. For example, C++ evolves at a pace where occasionally there will be good (generic, broadly useful) new questions posted, but most of those were posted >10 years ago. Commented Jul 20, 2023 at 16:30
  • @CrisLuengo: Sure, if you're writing in languages, libraries or frameworks that are that old, I could see this as a strong argument. Not a lot of things really move that quickly when it comes to C/C++ so you'd be more likely to find a lot of valuable things on Stack Overflow about it. But if you're dealing with something like JavaScript/TypeScript, Java (these days!), Kotlin, Python, Ruby, etc, you're moving much, much faster.
    – Makoto
    Commented Jul 20, 2023 at 16:32
  • I think Python moves slower than C++. At least when it comes to the language itself. Commented Jul 20, 2023 at 16:33
  • 2
    @CrisLuengo: Hard disagree. I still remember the 2 to 3 fiasco, and the impact of it. There's also a lot of movement in terms of the GIL as well. It's been ported to different runtimes as well (not just CPython, for instance), and those are still in active development. It's not break-neck pace, but it's definitely not glacial. :D
    – Makoto
    Commented Jul 20, 2023 at 16:38
  • 2
    Yeah, I know. C++ is not glacial either. We get a new standard every 3 years, with significant new features added. C++ today is not what is was in 2011, which was a totally different language from what it was before then. Commented Jul 20, 2023 at 16:46
  • ChatGPT has accelerated an existing trend, yes. My personal take is that it probably does so preferentially in a way that benefits the site, and that this trend is inherently beneficial already. I just wrote an answer to that effect; hopefully the reasoning is clear. Commented Jul 20, 2023 at 20:23
  • "Other sites and resources exist now that do a better job of answering the broad and tricky questions" Just to give readers some insights here, would it be possible to name a few such popular resources that do a better job at this questions? Commented Jul 20, 2023 at 21:58
  • 1
    @NoDataDumpNoContribution: I've gotten a lot of value out of Baeldung in recent months with Spring Boot questions, as I look to keep my Spring applications up to date or to solve different problems. Where I work I also get a free subscription to O'Reilly Books/tutorials which have helped a ton with getting up to speed on k8s and Ansible. Of course, there's also Jeff Geerling's books on Ansible that have helped there too.
    – Makoto
    Commented Jul 20, 2023 at 22:42
  • @Makoto Comprehensive guides so to speak. I never liked that books take so long to digest them. One needs to skim read them for useful information usually unless they are the really good ones. Commented Jul 21, 2023 at 6:34
18

This paper was also discussed on Reddit here. I read through the comments there, and some points I thought were worth mentioning:

  • Many Redditors commented about interactions they perceive as negative at Stack Overflow and that they found ChatGPT friendlier, such as:

    ChatGPT isn’t always correct but I actually learn along the way, I’ve learned more in a week than I ever did in years using SOF for help. I also don’t have to worry about being told I’m an idiot while trying to learn.

  • They commented about how ChatGPT can give faster responses:

    Isn’t the absolute worst thing about stack overflow is having to wait for humans to respond to your issue? If an application can advise you in 30 seconds then shouldn’t we be rejoicing?

  • Some users realized they are indeed using Stack Overflow less due to LLMs:

    I just realized that I used to visit nearly every day but haven't been in months since I started using LLM's for the same thing.

  • One user pointed out how they're, as a result of ChatGPT, using search engines less:

    I am not surprised, since I got a ChatGPT subscription I barely ever use a search engine and that used to be the main entry point to StackOverflow.

  • And I thought this comment was interesting:

    I have moderated question on SO and 99% of them are repeat questions by people who don’t know or care to search - LLM may not be a bad thing here as it essentially is a pre-search for info we didn’t know we needed

30
  • 8
    Shooting the messenger much, lol?
    – CodeCaster
    Commented Jul 19, 2023 at 5:33
  • 12
    You added the magic words "interactions they perceive as negative at Stack Overflow and that they found ChatGPT friendlier" that attracts downvoters like flies
    – user13267
    Commented Jul 19, 2023 at 5:57
  • 11
    Those don’t look like thoughts on the paper, rather like a mix of general and ChatGPT specific thoughts on SO. FWIW there are some comments actually addressing the paper in that thread. Commented Jul 19, 2023 at 5:58
  • 2
    It sounds like some of them could benefit from actually using the search engine. The only time I could find myself in need of something like ChatGPT would be for an edge case for which I need a specific solution, in which case someone more knowledgeable would be more likely to provide help. But then again I was told I never do things the way the others do it.
    – Clockwork
    Commented Jul 19, 2023 at 7:19
  • 11
    @CodeCaster when I saw this answer, I immediately got the impression that the Q&A were a Trojan horse being used to plant more pro-ChatGPT propaganda on Meta. Commented Jul 19, 2023 at 9:02
  • 8
    @AugustoVasques: Learning along the way is inevitable. Do you genuinely believe that competent folks already know everything they need to know to do their job, given constant technological change? Commented Jul 19, 2023 at 12:10
  • 3
    @KarlKnechtel Agreed--OP doesn't have a SO account as far as I can see, so there does seem to be an agenda here. As discussed in one of their previous posts, I wish they'd be a bit more transparent about why they're doing this on a site they (presumably) have no history or stake in whatsoever.
    – ggorlen
    Commented Jul 19, 2023 at 15:36
  • 2
    cgpt is the next iteration of the blind leading the blind.
    – Kevin B
    Commented Jul 19, 2023 at 15:44
  • 5
    @zcoop98 My first blush, knee jerk assumption with zero statistical backing is it's the different sorts of questions were asking. The beginner stuff its generally covered to death here on SO and all over the Internet, so the model can find dozens of examples of the problem in its data set and compose a decent answer from them. The kind of stuff the experienced programmer is looking to solve is not so well covered. Finding any documentation on what I'm typically having trouble with is a gift. Without good examples in its dataset all the model has to fall back on is dumb luck and outright lies. Commented Jul 19, 2023 at 20:11
  • 4
    And I surmise that the lack of good documentation on whatever I'm struggling with is exactly why I'm struggling with it. All of us who are primarily on the answering side of SO learned long ago that the key to success is RTFM. Commented Jul 19, 2023 at 20:14
  • 5
    Makes sense from the perspective of Reddit, it doesn't have the same goals as Stack Overflow so people there are more inclined to want schooling to be a thing - which is fine. That is a common misconception of Stack Overflow, that it is just like any other site. And therefore hostile and toxic because things which are normal everywhere else, such as discussion and opinions, are rejected here. I also very much agree with that last comment - ChatGPT is not competition or a replacement. It is a means to reduce junk input on Stack Overflow if people use it to outsource their work and homework.
    – Gimby
    Commented Jul 20, 2023 at 11:21
  • 3
    @KevinB For many simple problems, there's nothing wrong with ChatGPT's answers. I'd side more towards what user4581301 said, that ChatGPT just fails on harder problems.
    – Passer By
    Commented Jul 20, 2023 at 11:56
  • 2
    @zcoop98 I think there's also some significant background differences at play. I think a lot of the "ChatGPT is useless" crowd are trained to be skeptical towards literally everything, which is a pretty good attitude for an engineer or scientist. It surprises them to no end that that's not the default mode of behaviour for most of the population.
    – Passer By
    Commented Jul 20, 2023 at 12:02
  • 6
    @PasserBy "For many simple problems, there's nothing wrong with ChatGPT's answers." The emphasis is the problem. Given what GPT is and how it generates responses, the responses can only be trusted to look correct. That's all it is designed for. Tailoring it's usage toward users who, by definition, are unable to tell good advice from bad for the subject matter is obviously far worse than any outcome that could come from publicly asking the question and getting answers from other people. Regardless of how many cherrypicked successes are brought forward.
    – Kevin B
    Commented Jul 20, 2023 at 14:31
  • 2
    @KevinB " is obviously far worse than" depends crucially on the actual success rate of ChatGPT. There are other scientific papers about that and Franck Dernoncourt tried to ask that a few times on SE but was mostly downvoted. If beginners are better off with AI or with SO, I really don't know. My gut feeling would be to use as many different sources as possible including GenAI. Just try for yourself and learn from the experience. Commented Jul 20, 2023 at 21:42
17

One question of the researchers is essentially: Who will (unwillingly) train the next LLM bullshit-bingo-machine if everybody uses the current one instead of posting material that could be ingested by LLMs?

This is one of the arguments that non-coders can easily understand and which may be valuable in the public understanding of how much false hope and irrational hype is happening at the moment. Of course, the researchers’ framing may be a different one (digital public goods), but it doesn't make that question any worse.

In that regard I wanted to add another possibility / hypothesis for lower interaction:

Some people may feel unhappy with the idea that their volunteer work in helping others is used by private companies with cynical names, to produce answers that are only randomly correct, to fill pockets of shareholders, against or at least without the consent of said user.

Or shorter: they don't want their public help be used as privatized profit.

And possibly making this worse - the (still standing?) decision of SE to forbid moderators to use the detection tool might give such users the impression that they have zero protection against such practices, at least here.

Because if a user has this notion, they might get the idea that perhaps some of the questions they've recently answered were just ChatGPT induced stimuli, made to evoke a reply which would feed the LLM. Essentially, this could lead to loss of trust in user:user interaction, which is the worst thing that can happen for a user’s willingness to participate.

Would it be a realistic fear? I'm not sure - but it wouldn't appear as a totally crazy conclusion. And since I'm bringing this up - it was certainly one of the first questions that came to my mind.

I know this couldn't possibly explain such sharp declines, but I could see that some people may play it through in their mind and come to the conclusion that without these issues being fixed, they don't feel at home here anymore.

Allow me to explain this - certainly not what I want to happen, and not what I necessarily predict will happen; but a danger I see (and not just here):

Once we lose trust in the other side being a human (and for that to happen you don't need 50% chance of meeting a bot, I'd assume 10% is more than enough poison), it's just not the same. I'm pretty sure we lose interest, because, even if anonymous like here, the knowledge that we help somebody else - or they help us - is what forms these connections. No matter how nicely formulated, if we know we're talking to a computer, we lose that minimum level of affection (the basic human affection we have even for those we may not like). And in the worst case, if we didn't know before, we may feel tricked and betrayed.

In that sense, I really cannot understand why the negotiations about this mod tool is taking such a long time. Stack Overflow cannot be replaced by ChatGPT, unless the company falls into the hype trap and does it themselves.

4
  • 4
    Re "Once we lose trust in the other side being a human": Here is an example. Commented Jul 20, 2023 at 18:59
  • 4
    I have zero interest in "talking to a computer" regardless of whether I realize it. For years if not decades now we've had automated call handlers that do voice recognition and try to parse what you're saying to pull a canned answer out of a database or dispatch you to the right department. Those are already in my mind an unspeakable evil. I don't even like leaving a message on answering machines, honestly. Commented Jul 20, 2023 at 19:24
  • "if everybody uses the current one instead of posting material that could be ingested by LLMs" To be fair the claim is currently that a substantial amount of new content is AI generated not everything. There are still thousands of real human people out there on SO asking and answering questions as well as voting on them. Future training material would be compromised but not void of human content. Whatever the impact on the efficiently of AI will be, I've no idea. But some days I think I'm not much more than a very complex machine. Commented Jul 20, 2023 at 22:06
  • Automated call handlers... Meanwhile we have the unemployed starving in the streets. Unemployed who could be employed to take verbal abuse from irate customers. Me, I'm more comfortable swearing at a computer than a person. Commented Jul 20, 2023 at 23:52
13

Other answers have hinted at some things I'm going to say here, but I'm going to present an argument I'm worried might be controversial: It's a net good thing for Stack Overflow if the rate of incoming questions declines this much.

First off: we know there just isn't that much new to ask about; in fact we already know that the existing machinery has been extraordinarily inefficient in re-asking things. New questions that aren't duplicates are extraordinarily unlikely to fit as intended within the existing framework, because overwhelmingly they represent personalized requests for help rather than work orders.

I've long been an advocate for questions starting in a closed state and needing to prove themselves worthy of opening, as things currently stand. It's just too unlikely in 2023 that a new question is really valuable. Just for perspective: there are something like three times as many open questions on Stack Overflow, where the scope is specific to programmers, as there are articles on Wikipedia, where the scope is effectively as broad as it could possibly be.

Time not spent on new questions is time that can be spent on curating the existing library: identifying the best possible duplicates and refactoring accordingly - asking only a few, specific questions as artificial canonicals motivated by the patterns that we can now see in what people tend to ask.

Aside from that, there's good reason to believe that the questions being preemptively filtered out (by people asking ChatGPT and being satisfied with the results) are disproportionately weighted towards lower quality questions. Why?

  1. Lower quality questions inherently come from people who lack either the skill, patience or interest to do the necessary legwork to ask a high quality question - attempting to analyze the problem, isolate problematic code or work out precise specifications, create MREs, etc.

    People like that will naturally be disproportionately tempted to try ChatGPT, i.e. the tool with the interface "ask a question and see what answer you get". They don't read error messages because the compiler (or runtime) doesn't speak in natural language but in canned phrases that don't reflect understanding. They don't use search engines because they can't type multiple paragraphs or copy-paste code into a single query but instead have to actually think about what they're asking, and then potentially look at multiple search result titles and think about whether they're likely to be relevant, then go there and scan through a bunch more text (without knowing how much ahead of time) and think even more. Imagine!

    Just think about it. When was the last time you saw text like "I tried asking ChatGPT about it but the response wasn't useful" to an actually good question? In the past, people would say the same thing about search engine results, and they'd say it on questions that turned out to be high quality as well - they'd just give a better explanation of what search results they got and why they weren't helpful. But that doesn't happen with ChatGPT. People who ask good questions seem much more likely to understand why ChatGPT can't actually help except insofar as it acts as a sophisticated search engine (converting a question into a query for its database of training data, then synthesizing bogus search results out of concepts).

  2. As much as we are willing to accept "easy" questions and give them a thorough treatment if asked properly and if they help build the library, it can't be ignored that lower quality questions tend to concern easier subject matter. Sometimes that's because of some weird idiosyncratic misunderstanding of a fundamental concept ("fundamental" does not imply either "easy" or "simple", but many of these concepts incidentally are not hard to grasp); a lot of the time it's something answered by documentation, or a question of applying a common technique. Some of those can become high-quality questions (it's just that they're overwhelmingly actually an issue for people who won't ask a high-quality question); others really just can't.

    ChatGPT doesn't care about any of that. It doesn't care if your question is a duplicate (although I suppose the planet cares about the electricity usage as it re-generates an answer where a cached one would have been just as good). And the funny thing is, while it's generating an "answer" for an easy question, it's more likely to output something correct, or at least useful, despite the lack of actual comprehension.

    Why? Just think about how the training data works. If the Internet is full of people asking "How do I foo the bar?" in a million different rephrasings, and being told to use a widget, it becomes much easier for a LLM to predict that the next three tokens after text that asks about fooing a bar are use-a-widget. This seems to entail, in a meaningful sense, some kind of representation of the concept of fooing and of bars and widgets. It doesn't, to be clear, entail any kind of conscious experience of those concepts, or of logical reasoning, or an understanding of how widgets solve the problem (although it does entail an encoding of information about what people typically say when trying to explain that). But none of these philosophical questions actually matter to the observed behaviour of the "ChatGPT + programmer with a question" system. The simpler fooing a bar is, the more likely it is that the flood of "use a widget" answers in the training data are correct, in a statistical sense. Therefore, the chance that the output from ChatGPT is "use a widget" (or other prose to that effect) rather than "use a sprocket" goes up.


So, everyone wins when people with simple questions try using ChatGPT first - people who want Stack Overflow to be a high-quality library can spend more time on quality rather than dealing with quantity; people who are too lazy to use a search engine get seduced into effectively doing it anyway; people looking for a tutor get pre-filtered in case the concepts don't actually require expertise to explain; people who simply need a rubber duck get an extraordinarily sophisticated one; and Stack Exchange, Inc. doesn't actually need to pay for API usage.

Unless, of course, the answer they get actually is wrong, which will happen a lot. And except, of course, for the planet. But at least we aren't to blame for it.

Oh, and also there are the people who will walk away with false impressions of how "intelligent" ChatGPT is, but we can't do anything about that anyway. It's all the same crowd that thinks a site that provides a question-asking interface and doesn't let you just ask whatever you want, for your own benefit, is inherently "toxic".

Oh, and I guess the company won't be pleased about a lower rate of page views when ChatGPT synthesizes their (really our) content into a new form on the fly rather than linking to it. If someone else is profiting off of something I wrote, despite that it's CC licensed and I can just as easily rehost it myself, I don't know that I really care who is making the profit. The company should have thought about that when it first got wind that OpenAI was crawling the web for training data.

12
  • 2
    I read the first paragraph, thought "I'll bet this is Karl", and scrolled down and revelled in my genius. In more seriousness, I think I agree with the main conclusion/point here, but just don't really resonate with all of the ideas.
    – starball
    Commented Jul 20, 2023 at 20:52
  • 3
    I think my opinion, writing style and formatting are all fairly distinctive, honestly, and I don't really feel like masking any of them. Commented Jul 20, 2023 at 21:18
  • 1
    "everyone wins" There might be some user groups that actually liked answering duplicate questions for the up-teenth time just to get some points, so maybe not everyone wins but most. And in case AI gets better somehow in the future, this calculation could actually change. Curators (editores, voters) might remain in high demand but answers (especially not too much experty experts) maybe a bit less. Commented Jul 20, 2023 at 22:18
  • @NoDataDumpNoContribution "liked answering duplicate questions"... there will still be plenty of existing questions to post duplicate answers on. Doesn't really seem to change what they're already doing anyway...
    – Dan Mašek
    Commented Jul 20, 2023 at 22:31
  • 1
    "There might be some user groups that actually liked answering duplicate questions for the up-teenth time just to get some points" - the more I think about it, the weaker "everyone" seems to be; but the change is still clearly positive. I'm... not particularly sympathetic towards the group you're talking about, anyway. ;) Commented Jul 20, 2023 at 23:07
  • Search engines are also in an interesting state of change. Remember how fast Google killed all comers by presenting clean, ad-free results that appeared to have a high degree of focus rather than a long list of keywords embedded in the site? Now that Google's effectively hiding the ads in the search results and there's a large cottage industry around making your page show up in ANY google search where do we-the-users sit? Commented Jul 20, 2023 at 23:26
  • I heard the kids somehow use Tiktok as a search engine now, or something like that anyway. *polishes cane* Commented Jul 20, 2023 at 23:35
  • 1
    Why I remember having to trudge all the way to Delphi to get answers... Uphill. Both ways. Commented Jul 20, 2023 at 23:48
  • 1
    @user4581301, the thing that made Google work so well early on was the existence of "links pages": webpages that existed entirely to provide curated lists of high-quality sites on a topic. Those disappeared because early Google was able to provide a list of high-quality sites on any topic.
    – Mark
    Commented Jul 21, 2023 at 0:16
  • 7
    Re "might be controversial": Only for the company's KPIs. 23,822,634 questions is way too many. The massive duplication effectively renders Stack Overflow useless as a research tool (too inefficient). In most cases, there is not even an attempt to find the duplicate (potential answerers don't have any interest in it). Commented Jul 21, 2023 at 9:58
  • 3
    @NoDataDumpNoContribution: In some cases, it might also be driven by payment behind the scenes. For instance, by the paid homework industry. Or they may be part of the spam organisations (e.g., a sort of contractors recruited through Upwork or similar) to facilitate the spammers getting a foothold. For example, they get a commission on Upwork to generate a certain amount of reputation points (transferred / laundered to the spammers or CV stuffers through the bounty system). Commented Jul 21, 2023 at 10:14
  • 1
    cont' - Paid homework industry example (the URL has been obfuscated to remove any link juice - remove all instances of the letter "z" - see e.g. near "How can I submit my homework?") Commented Jul 21, 2023 at 10:18
4

..private use of GenAI (such as users asking GenAI the questions directly instead of asking on Stack Overflow) will offset this GenAI-training problem, this will allow major GenAIs to monopolize knowledge that would otherwise be freely available.

Let me just focus on this part. It seems a bit like a very broad conclusion and probably not backed up by a simple drop in traffic to Stack Overflow, but still I agree with it. It's a broad tendency and will be a central topic for all AI impacts in the future that likely only the big players will profit from AI while many others will suffer.

AIs take big training data sets from everywhere and finally (somehow) make money from that. But there is no compensation that I'm aware of for those who created these training data. It's a bit similar to Stack Overflow Inc who get all the advertisement revenues and got bought for two billion dollars while presenting content they didn't create themselves only bigger. AIs present converted content they nevertheless didn't create in the first place on a very large scale.

Less duplicate questions asked are a good thing, less traffic is already more critical because it translates into less income for the company threatening further lay-offs and might also reduce motivation of experts or curators. One motivation to answer questions is to help future visitors. If there are much less future visitors, why bothering to write answers.

Are we happy with just providing the training data for future AI applications? Or how would a fair relation between content creators and AIs look like? That's what I'm asking myself mostly.

1
  • 1
    As a first-order effect, fewer crap questions sounds like a big win. As a second-order effect, actual experts will probably reduce their output, but that may well also translate into higher-quality answers on average. SOfT as the main source of income for the company benefits from less traffic on the public sites, so in the end this could play out nicely for the company, too.
    – tripleee
    Commented Jul 21, 2023 at 8:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .