71

Is this why the company wanted to effectively unban AI-generated content? Because AI-generated code explanations will be auto-dumpable now?

From WWC23 - Stack Overflow: Community and AI (Mainstage) at t=25:55, at the section where Prashanth is speaking about the explain code feature of the planned VS Code extension:

Let's say that you had a piece of code you wanted to really get an explanation for the code and you're able to then do that through this mechanism now. You'll get back sort of a short summary of what you're trying to find out or get an explanation for and you're also going to get references from again our data set on where this explanation really was rooted in, right. So this is a really really powerful way for you again to trust and attribute answers from the public community and within stack overflow for teams, and if you think by the way that this was helpful you can create a Q&A, or you can create an article in this case, it's going to suggest tags automatically and you can post it into stack overflow for teams or public stack overflow.

This isn't actually happening as described in the talk, right? The VS Code plugin's explain code feature is not actually going to enable people to dump code explanations from an AI as Q&A pairs to SO, right? You're not going to deploy and promote a feature which blatantly violates site policy, right?

And even if this weren't against site policy, how would it square with community consensus on how code explanation questions should be scoped and formulated? Ex.

I think this would have a pretty severe negative impact on the quality of the knowledge-base over time with content that the poster has dubious understanding of, diluting the value of the knowledge base's content, and filling the internet with AI-generated content that future AI systems will learn bad information from.

And, y'know, having a bunch of your OverflowAI users dumping such Q&A pairs and then getting their questions downvoted and closed for being too broad, and that contributing toward / possibly triggering a question and/or answer ban... isn't really good for your image.

And even if considering all that, you're still going to do this, please at least follow the OpenAI policies for sharing and publication (unlike what you did with your other experiments).

13
  • 1
  • 10
    Ouch... I missed that part with creating content on public Stack Overflow. I don't have enough facepalms. Jul 27 at 19:04
  • 6
    "So this is a really really powerful way for you again to trust and attribute answers from the public community" - ????? Jul 27 at 19:15
  • 5
    I personally, based on my own experience with LLM output, will not trust LLM output for the considerable future. So if the future, is that the company is going to allow and promote the creation of this content, then that output will not be (appreciated, trusted, or considered helpful). Jul 27 at 19:21
  • 29
    It's worth mentioning that MDN tried LLM-powered code explanations on their web dev docs sites sites and it did not go well, I think they disabled it after a week or so of bizarre misinformation and angry users Jul 27 at 19:34
  • 1
    @user56reinstatemonica8 that's not what their own feedback showed though! even gpt4 said their results were great.
    – Kevin B
    Jul 27 at 19:34
  • 4
    About 20 years ago I worked as code monkey for a group of Comp Sci PH/D candidates working on a system that would parse student code and supply meaningful feedback. The candidates were mostly worried about making the engine parse code into ASTs fast enough to provide real-time feedback. Me, I was wondering how they were planning on building the database needed to interpret the ASTs generated from the student code and who was going to supply the interpretations of often-insane student code. Jul 27 at 20:33
  • 5
    I made an attempt at copyediting it: "Let's say you have a piece of code you want to get an explanation for. Through this mechanism, you'll get a short summary of what you're trying to learn, including references from our data set on where this summary came from. This is a way to trust and attribute answers from the public community and within Stack Overflow for Teams. If you think the result was helpful, you can create a Q&A pair, or an article; the mechanism will suggest tags automatically, and you can post it into Stack Overflow for Teams or public Stack Overflow."
    – TylerH
    Jul 28 at 13:38
  • 2
    I’m voting to close this question because, upon further reading, it is a speculative question about future, unpublished site features Jul 30 at 16:08
  • 3
    @TomWenseleers It's phrased as speculation to express disbelief. The transcript of Prashanth's talk is quite clear. Also, asking about the details of planned features being off-topic on meta is news to me, and I don't understand why it would be.
    – starball
    Jul 30 at 20:49
  • 1
    @starball Well it was new to me too, but my question - very similar to yours - meta.stackoverflow.com/questions/425775/… - was closed for that exact reason... Jul 30 at 21:51
  • @KevinB is it really a "code building" tool? It supposedly explains code. Also, how are they not letting us benefit from it? What would you propose? How would this be useful to you?
    – starball
    Aug 1 at 17:18
  • @KevinB but... it's supposed to be a VS Code extension? Are we talking about the same thing?
    – starball
    Aug 1 at 17:29

2 Answers 2

50

I wish the company could again trust the public community and the consensus (read common sense) and just stop pushing for genAI content to be posted on their (our) network.

  • Improving search bar with "AI"? Fine, we can work on that. As long as it does not lead to questions drafted by "AI".

  • Improving question titles (not the body) with "AI"? Sure, it didn't go well the first time, but we can test that further.

  • Letting "AI" hallucinations on the network? No, please no.

  • Dumping unverified "AI" garbage on us? Heck no.

P. S. As Shog9 said (on discord), "First law of holes applies":

If you find yourself in a hole, stop digging

And if you are interested in getting out of there, read his post on MSE.


Why "AI" in quotations? Because LLMs have a long way before they can be called intelligent.

15
  • 1
    i mean... your first two bullets do exactly what 3 and 4 are against.
    – Kevin B
    Jul 27 at 19:33
  • 2
    @KevinB I can see the issues with the 2nd one (hence testing it further). But how you can make a connection between a search bar and content posted on the site?
    – M--
    Jul 27 at 19:35
  • A key part of that feature is it turns into a prompt -> answer dialog with a chatbot that will then generate a question for you if you aren't satisfied.
    – Kevin B
    Jul 27 at 19:41
  • @KevinB where did you get that? I must've missed that part :/
    – M--
    Jul 27 at 19:43
  • 1
    Image #4 on the announcement and the paragraph above it meta.stackoverflow.com/questions/425766/… (the first three images look like 1...)
    – Kevin B
    Jul 27 at 19:44
  • @KevinB I don't see that in the images nor the text: i.stack.imgur.com/f0sOy.jpg
    – M--
    Jul 27 at 19:46
  • This in particular: 'From here, they’ll see another AI-summarized solution, and as they build their knowledge and possible solutions, they can get more specific in their query, adding in their code or what they’ve tried and making highly personalized search results. These follow-up questions and iterative search experience will help enable users to better understand how the content applies to their context." which is then followed by *"if an instant answer is not available, the user will then engage in a conversation to help them craft a high-quality question"
    – Kevin B
    Jul 27 at 19:49
  • This process in particular will be able to hallucinate new novel answers that don't exist in the questions returned by the search, particularly once the user is allowed to submit their code. Additionally, this process concludes by generating a question for the user automagically, again, likely to be hallucinated in the same way the formatting assistant did.
    – Kevin B
    Jul 27 at 19:50
  • 2
    @KevinB I believe you are not reading it objectively and actually reading between the lines; what I read is that they will be prompted into using ask wizard or something like that, not that they will get their question written for them by AI. P.S. putting aside that I said we can work on that feature, not that we'd take whatever we are given.
    – M--
    Jul 27 at 19:52
  • There's an image floating around from the presentation of a generated question, complete with banner stating something along the lines of "Here's your question ready to be published!" with feedback controls, but I need to step away and couldn't find it quickly.
    – Kevin B
    Jul 27 at 19:54
  • 2
    @KevinB well I haven't seen that. But that is totally different from what you stated above by quoting and referencing the announcement. P.S. Ask Wizard says something along those lines when you complete writing your question, if I am not mistaken.
    – M--
    Jul 27 at 19:56
  • 2
    I agree on the first bullet. AI searching can be really useful. Users asking a question usually implies they have limited knowledge on that topic, and hence don't know the exact keywords they need to look for. AI would definitely help.
    – Ricky Mo
    Jul 28 at 6:26
  • 1
    @M--ßţřịƙïñĝ this is specifically what i was referring to: i.stack.imgur.com/CYo8U.png -- it's the end result after going through searching, chatting with the chat bot, and then providing your code. It's sourced from the announcement video
    – Kevin B
    Jul 28 at 19:01
  • @KevinB seen that on discord. Hoping that it was left out of the announcement for a good reason (retracting), not to deliberately misrepresent the feature.
    – M--
    Jul 28 at 19:04
  • I'm going with it was left out because they won't be ready to release that part in the alpha phase, but who knows
    – Kevin B
    Jul 28 at 19:06
-26

I think you'll find genAI-assisted questions would receive a warmer welcome at the language sites---LLMs are, after all, good at (multiple) languages. Hallucinations are rare(r), and more obvious.

Many users of these sites are not native English speakers, which leads to lots of editing; maybe genAI could reduce that workload. Some questions need the OP to add in their attempts or research, point out where they're confused, or explain why Google Translate doesn't answer their question. Some questions are "hit and run" translation requests, which are kind of on-topic but don't attract experts. Maybe genAI can prevent these.

Here's my first AI-assisted questions at Chinese.SE. Here the LLM is functioning as a kind of secretary or typist---it's my question; the LLM wrote down a first draft, and after some back and forth where I gave the AI more details I wanted to include, I edited the result to a final draft and posted it.

(I have a list of questions I want to ask [I jot them down while studying], but I don't have time to convert them to actual questions. Maybe I can finally ask them with genAI's help.)

The main issues I've encountered with AI-assisted question: (a) the LLM making up motivations for asking the question; and (b) the LLM adding in inappropriate phrasing for Stack Exchange (e.g., every sentence is a new question, extended "hellos" and "thank yous", asking for opinions). I tried to get it to add in dictionary references, but that didn't work (it'd need to be done manually). It's also not so great at the Stack Exchange markdown.

Oh, and sometimes the AI actually answers my question in the process of writing.

11
  • 24
    I think this answer is better suited to a similar question on SE's meta... What happens on other SE sites is not very relevant on SO...
    – Cerbrus
    Jul 28 at 5:37
  • 3
    You're right, but... Stack Exchange have said what happens on SO is going to eventually happen at other sites. And if the feature is not popular here, "why not try another site?" might be a reasonable answer. Jul 28 at 5:47
  • 5
    "Some questions are "hit and run" translation requests, which are kind of on-topic but don't attract experts." In theory, japanese.SE doesn't allow those. Jul 28 at 6:37
  • 7
    Rebecca, developing these AI tools takes considerable resources (wages + time). SO is the core of the network: it gets the vast bulk of the traffic (& hence advertising revenue) and the reputation of SO is what enables SE Inc to sell Teams. If these AI tools work on SO, they may also be extended to other sites, but they simply can't afford to develop them for the minor sites and not SO.
    – PM 2Ring
    Jul 28 at 6:51
  • I mean they could pioneer these tools somewhere where they'd be more welcome (and are easier to moderate when they go wrong), then bring them over to SO after the issues that arise have been resolved. Things are still fairly experimental. I'm just suggesting the possibility, since the earlier answer is like "we don't want genAI". (Oh, and Chinese.SE is not minor to me.) Jul 28 at 7:09
  • 12
    Stuff that works on language sites doesn't necessarily work on SO, and vice versa. I don't think the two types of site are comparable at all, and I really don't think implementation experience is transferable like that.
    – Cerbrus
    Jul 28 at 7:11
  • 5
    @RebeccaJ.Stones Imagine the disaster if this is developed on some language site where it might be more welcome and also could work better simply because LLM are chatbots and they they apply this on technical or other sites where it completely falls apart. Jul 28 at 7:12
  • 7
    And even with language I have experienced situations where Gen AI translation was totally off the mark as it completely missed the context. I could only catch it because I know both languages well and it was a test. This is rare enough, and it is not as important for asking questions on language sites as others will catch that something is off, but it can be a problem if you allow AI generated answers. It is likely that knowledgeable people would be able to catch errors because sites have low traffic, but imagine what could happen if traffic grows to the point it cannot be handled anymore. Jul 28 at 7:18
  • 7
    Once you cannot successfully moderate it will be very hard to turn back and prohibit what was previously allowed. If you need proof, just look at SO where at beginning some kinds of questions were allowed, but not anymore, and we still get frequent complaints from people asking similar questions wondering why they are closed and downvoted pointing to existence of such now off topic popular old questions. Same goes for answers. Once you allow AI it is done deal. Jul 28 at 7:22
  • 5
    @RebeccaJ "they could pioneer these tools somewhere where they'd be more welcome (and are easier to moderate when they go wrong)" Sure, and they have tested some stuff at smaller sites in the past, for those reasons. But that's probably not a useful strategy for these AI tools. They have to work on the scale of SO, which currently has 23.8 million questions, Chinese.SE has ~11,000.
    – PM 2Ring
    Jul 29 at 17:04
  • 4
    (cont) Also, searching & summarising from a natural language site is a slightly easier task for a language model than searching a mixture of technical natural language and multiple coding languages, where a LLM has a higher risk of hallucinating authoritative-sounding nonsense.
    – PM 2Ring
    Jul 29 at 17:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .