-52

I have signed up for an account at OpenAI and asked ChatGPT a programming question, namely: How to list all files in an Android device recursively with Java

And in an instant, it generated an answer in less than a second and the answer was correct. Given that most users post to Stack Overflow and don't even get an answer to their question, this could be a relief to most users who experience this and would discourage anyone plagiarizing code from OpenAI and pasting it here to gain reputation. But technology is advancing and we are becoming more and more independent of human skill to find solutions to programming challenges. Some high reputation contributors ignore questions deliberately but openchat gpt came to save the day. The speed and the accuracy of OpenAI responses are what gives it an edge over Stack Overflow and then users won't get banned or question limit imposed on users based on time intervals. Your thoughts and views on this matter are welcome, could openchat GPT be the end of Stack Overflow?

39
  • 39
    We are talking about the same AI that thinks that in the time you aged 64 years, your sister aged 70 years right?
    – Thom A
    Dec 29, 2022 at 8:48
  • 23
    "and the answer was correct" .. why, because it worked? How do you know it is not the worst approach which also works? Or is that unimportant as long as does what you want it to do?
    – Tom
    Dec 29, 2022 at 8:57
  • 14
    Some of the horrendously wrong and unoptimised solutions I've seen from it would dispute that, @PubliusFlaviusTiberium .
    – Thom A
    Dec 29, 2022 at 9:01
  • 12
    @PubliusFlaviusTiberium You can believe that, yes, but expect do be disappointed.
    – Tom
    Dec 29, 2022 at 9:03
  • 4
    If it can't do basic maths, @PubliusFlaviusTiberium , I'm not going to trust it with anything. Especially as I can't read or write Java and thus have no way of telling if the words it spits out are even close to correct.
    – Thom A
    Dec 29, 2022 at 9:05
  • 10
    How about you give it a question that isn't already answered multiple times on the internet? I've played with it for quite some time and have seen it imagine things and generate faulty code, unless I give it very specific directions. With their current capability these tools are only good as a wing-man / assistant for a programmer. Dec 29, 2022 at 9:06
  • 4
    @PubliusFlaviusTiberium Not specific in the sense of the problem but specific in the sense of the code I want to be generated. I've defined the problem specifically enough to it that a good developer will be able to write code for it. If you feel that it is generating stellar code and replacing SO you haven't played around with it enough. Dec 29, 2022 at 9:18
  • 7
    Note as well, that ChatGPT was out of date even when it was launched. It was only "fed" data up to some point in 2021; it does not have any knowledge of new features and progress made since then (so it knows certainly knows nothing of updates bad to languages/products made in 2022, and we're about to enter 2023). For some technologies, this lack of "bleeding edge" fucntionality is an easy downfall for ChatGPT.
    – Thom A
    Dec 29, 2022 at 9:26
  • 25
    I am forever jealous. Why do other people find places to work where they actually need only code which an AI can produce? I always need to do things that require three weeks of work, thousands of lines of code added/changed/deleted of which one week of head banging because documentation too assumes everyone is building hello world and little more :/ Not to mention the gruelling process of writing automated tests to verify that it works because testing in production tends to be a no-no and test environments are usually warzones held together with duct tape.
    – Gimby
    Dec 29, 2022 at 10:15
  • 8
    “… and it would explain like a human being. That's where the money is.” - Trying to explain a Java algorithm like that would also make a horrible Stack Overflow answer. One I would probably downvote. As a programmer who has seen a couple dozen answers generated by OpenGPT I can tell you that OpenGPT has a pretty poor track record (literally 100% of the answers I have seen have been low quality incomplete garbage) In fact I haven’t seen worse answers to a programming question anywhere from anyone in my nearly 3 decades of programming. ChartGPT is a garbage tool to answer programming questions Dec 29, 2022 at 10:40
  • 9
    Another example of stupid arithmetic (and lack of logic) by ChatGPT: i.stack.imgur.com/Z4uFC.jpg
    – PM 2Ring
    Dec 29, 2022 at 11:21
  • 21
    That's not an arithmetic failure, @PM2Ring, it's a logic failure. Same as answering "1 month" to the question "if it takes 1 woman 9 months to have a baby, how long will it take 9 women to have a baby?"
    – Cody Gray Mod
    Dec 29, 2022 at 11:23
  • 7
    What is "OpenChat GPT"? An Android app? Do you mean OpenAI's ChatGPT? OpenChat is allegedly a chat application. Have they incorporated ChatGPT into it? Can you elaborate? Dec 29, 2022 at 18:56
  • 5
    @President James K. Polk: The Stack Overflow Q&A as a training set was already dubious (unless they have applied some filters, like a vote total threshold or a 'lateness' threshold). Many (or most?) of the "try this" answers (code dumps) are likely completely bogus and/or plagiarised code (bogus in the context). Dec 29, 2022 at 22:50
  • 7
    @CodyGray ChatGPT might be a blessing in disguise for the site, if new and very inexperienced developers can get good and fast answers to their problems, which repeatedly flood the site, this should leave more room for the more intriguing head-scratching variety, problems that require ingenuity, creativity and a wealth of experience. Or am I being unrealistic and optimistic?
    – Mari-Lou A
    Dec 30, 2022 at 10:53

14 Answers 14

24

I have had the same experience as the OP.

ChatGPT has saved me quite literally hours of searching for solutions online. While I agree that ChatGPT does not accurately solve all of my problems, it has been extremely useful in aiding me with objectives like, learning react, (finally) installing and leveraging docker, and building applications using websockets and it offers far more of a head start than browsing SO. I sense that ChatGPT is facing the same pessimism that all novel technologies face, and unjustly so. It is an incredible tool that we should all be excited about and I humbly advise my colleagues here to learn to work with ChatGPT.

9
  • 9
    "does not accurately solve" That is, like, exactly the problem. None of the other things are SO's goal. I sense that SO is facing the same pessimism that all specialised tools face, and unjustly so. It is an incredible tool that we should all be excited about and I humbly advise my colleagues here to learn to work with SO. Feb 9 at 7:56
  • 4
    nailed three technical projects with chat GPT and am pretty sure if I posted here the questions would go unanswered Feb 9 at 11:04
  • 3
    The point is that once an answer is generated from ChatGPT, its still on you to conform it to your project. The same is true of SO, however ChatGPT will get you to the final solution much faster and with far less effort.
    – Daniel
    Feb 9 at 16:56
  • Chat GPT has rendered most of the StackExchange platforms suddenly completely obsolete.
    – Cesare
    May 7 at 21:43
  • 6
    @Cesare If you believe that ChatGPT now fullfills all your needs over SE, I’m happy for you. :) You’re free to use ChatGPT instead of SE sites if you so wish. May 8 at 0:23
  • 1
    Thank you!! :-) already doing that. The main difference is that when asking a question to chat GPT you get an answer. When asking a question to SE you get replies like "please provide a minimum example, or an example of what you have done so far"
    – Cesare
    May 8 at 9:44
  • agree. It points you in the right direction a lot quicker
    – Andy
    May 9 at 15:08
  • 4
    @Cesare And that for a reason. The humans on SO realize that to help you, all the necessary information to get at an answer, must be provided. Else help cannot be given. ChatGPT doesn’t think. It just generates some text that suits whatever input you give it. You can’t rely on it, and if information is missing, then surely the answer it gives can’t be accurate. But if it does help you, that means somebody else already had the same question, or you could’ve found the answer through some research yourself; so your question shouldn’t be put on SO anyway. May 11 at 8:55
  • 1
    I think this answer deviates from the discussion a bit. The OP asked, "Is this the end for SO?" with a not-so-subtle insinuation that ChatGPT content should be allowed here. Just so we're clear, the fact that many people find ChatGPT useful for programming is not sufficient justification for allowing such content on SO. May 23 at 20:49
69

I've yet to see an answer from ChatGPT that was accurate. Speed? Sure, the robot wins. I don't come to Stack Overflow because I need the fastest answer. I come to Stack Overflow because I need to know the correct answer, from a trustworthy expert who knows what they're doing. (And, better yet, an answer that has been vetted by other experts via our voting system.) Your mileage may vary. Do whatever works for you.

11
  • 8
    Even accuracy, I just asked it how to list all audio files in my PC with python and it generated an answer answer glob patterns and os modules, this AI is amazing. it's nothing the world has ever seen before. Dec 29, 2022 at 7:56
  • 26
    A bot that can copy-paste code snippets from elsewhere really doesn't amaze me all that much. You know what's really ironic? If what you say about it is true, then we don't need human programmers at all. Just have the bot do it. It won't just be Stack Overflow that is obsolete; it'll be everyone. I think we have a few years before that happens.
    – Cody Gray Mod
    Dec 29, 2022 at 7:57
  • 4
    even if its copy pasting, the answer is correct and in an instant. My post would spend days or hours without a response here on SO , I feel better now as a programmer. But it's true automation will render all of us jobless in some few years Dec 29, 2022 at 8:01
  • 34
    Dunno about you, but no aspect of my job involves copy-pasting code. I have to design things that work reliably.
    – Cody Gray Mod
    Dec 29, 2022 at 8:07
  • 3
    but future users come to SO to copy paste ? Dec 29, 2022 at 8:17
  • 33
    Then those users will have the same problem with Stack Overflow as they do with ChatGPT, @PubliusFlaviusTiberium , blindly copying code without validating and altering it for their own needs first, and then wondering why it isn't working.
    – Thom A
    Dec 29, 2022 at 8:54
  • while some people(including myself) might simplify a programmer task is to "copypaste". It's hardly the case.
    – Alvi15
    Dec 29, 2022 at 9:26
  • 9
    “It’s nothing the world has ever seen before.” - A Google engineer thought the same thing about their AI project, they thought it was alive, even Google knew better and fired that engineer. ChartGPT is garbage in my opinion. Just to be clear it’s also not AI. Dec 29, 2022 at 10:42
  • 1
    I think for alot of people it comes down to the fact that most people on SO are pretty condescending, The Voting system isnt always accurate you tend to get more upvotes just for taking a bit more time to explain or write some prettier code. I think overall most just see as a way to avoid being drenched on for asking a question that to the individual is strange and difficult but might be simplistic in the eyes of the community thus becoming a burden. I think both have their place, However im going to have to go with the AI here Its less snarky and tends to explain better Sorry SO
    – Azurry
    Feb 16 at 5:42
  • 2
    @Azurry What's "inaccurate" about getting more upvotes for taking a bit more time to explain or write prettier code? Doesn't that make the answer more useful to more people in the future? That's exactly what upvotes are for. And then you complain that SO answers don't explain as well as the AI? This is a contradiction! Yes, if you're looking for a help desk or a personalized, individual answer or someone to, um, chat with, then ChatGPT is probably better than Stack Overflow. That's fine; that's not what Stack Overflow is trying to be.
    – Cody Gray Mod
    Feb 16 at 6:28
  • 3
    now we can be stupid faster
    – starball
    Mar 14 at 6:30
49

No, at least not with this technology.

A lot of people don't understand that ChatGPT is a language model. Its goal is to smash words together to create a plausible-sounding explanation, not creating a correct one. So most of the time the answer can look correct at first glance, but it is blatantly wrong upon close inspection. This can be seen as it made up a return value.

In contrast, a high-upvoted Stack Overflow answer is written by a subject-matter expert and reviewed by people, so it is correct most of the time.

5
  • 2
    did you ask it a programming question? it works Dec 29, 2022 at 8:04
  • 19
    @PubliusFlaviusTiberium Have you seen 99% of its' blatantly wrong answers floating around the site? 1% of the time what it came up to is right by pure chance. I could do that too. Dec 29, 2022 at 8:10
  • 2
    but you lose right at the speed? Dec 29, 2022 at 8:18
  • 34
    @PubliusFlaviusTiberium I don't need speedy and wrong answers. People come to Stack Overflow to copy the correct answer. Dec 29, 2022 at 8:19
  • 30
    I can calculate 8912384*12333 faster than a calculator.... the answer is 400. Being fast doesn't mean much.
    – Alvi15
    Dec 29, 2022 at 9:27
39

Well, when I entered How to list all files in an Android device recursively with Java into Google, I got links to several Stack Overflow Q&A, also in less than a second. The first one has 7 different answers, and the second one has 33 different answers. People have already voted on whether the proposed solutions were helpful to them or not; the top answer on one of them has 442 upvotes and 1 downvote, so I already know that hundreds of humans have reviewed the answer for correctness.

You got one ChatGPT answer that no one else had reviewed but that just happened to be correct (and that's not always the case); in the same amount of time, I got 40 answers to choose from (all of them reviewed by other humans before, in many cases hundreds of times).

Imagine what would've happened if ChatGPT's answer hadn't been correct; what would you do then - ask it the same question and hope for a different answer? Or just post a Stack Overflow question instead?

You can downvote a bad Stack Overflow answer or comment asking for clarification, but there's no recourse for a bad ChatGPT answer.

Also,

Some high reputation contributors ignore questions deliberately...

Given that most high-rep users got that way in the first place mostly by answering questions, this claim is close to self-defeating. Personally, I have 609 answers (vs. 61 questions), so I guess I didn't ignore at least 609 questions. Jon Skeet didn't ignore 35,534 questions. Gordon Linoff didn't ignore 85,121 questions. VonC didn't ignore 29,184 questions. I could keep going with this, but the point is the vast majority of people become high-rep by answering lots of questions.

By way of contrast, you have asked 61 questions (50 of which have at least one answer and 34 have an accepted answer) and have only written 9 answers, so there is a certain irony in you accusing us of deliberately ignoring questions.

7
  • 1
    you checked my profile lol 🤣 , I will borrow the answer attitude and try to provide more answers to questions than asking my own Dec 29, 2022 at 18:50
  • 3
    I do ignore a lot of questions. After closing them as duplicates, lacking information, the result of typos or similar trivial problems. Not many good, original questions, but that's just the nature of the beast. In C++ there will be a burst of good questions around the release of a new Standard revision, then a few years of mostly same-old, same-old because damn near all of the good, old questions have been asked and answered. And that's exactly what should happen. Dec 31, 2022 at 9:49
  • @user4581301 "After closing them as duplicates, lacking information, the result of typos or similar trivial problems". Closing question because of typos and trivial problems is why SO kinda get's labeled as toxic. Please don't close questions because of simply a typo. <3
    – brat
    Apr 11 at 11:03
  • @brat It's site policy to close such questions, so it would be unethical for us not to vote to close them. They're highly unlikely to be helpful to anyone other than the OP, so they create a distraction from more substantial issues. If you disagree, feel free to write a Meta post challenging that policy. Apr 11 at 13:04
  • 1
    @brat Depends on the nature of typo. The vast majority of the time a comment like, "You defined FooBar and implemented foobar" or "you mixed up i and j in the for loop[,]" is enough to get the asker back on the right path and isn't going to help anyone else. Writing a formal answer and keeping the question visible is just adding noise to the search results. Close it and let the roomba do its job. If it's something subtle and sneaky or there's some secondary value-add that's more important than the question asked, considering answer that. Apr 11 at 15:30
  • @user4581301 Oh. That kind of typos. Fair enough, my bad. I thought you meant spelling mistakes like this wone. Up voting your answer for clearing that up.
    – brat
    Apr 11 at 15:33
  • 2
    @brat Yeah, if it's just typos in the post itself, then it should be edited (not closed). If a post is salvageable through editing, it should be. Apr 11 at 16:00
33

ChatGPT is not the end of Stack Overflow. SO isn't designed to give answers to everyone who asks. The goal is to offer quality answers so that people don't have to ask a question. We trust that people who answer are experts who check their answers. The answers have value because they explain some programming concept in an easy and correct way. Speed is irrelevant as the correct answer can be posted at any time even years later.

ChatGPT is able to produce an answer that isn't necessarily correct or best practice. It might work for you but it's not coming from a programming expert.

In the long run ChatGPT may help Stack Overflow if it can provide correct answers to simple questions. This could help avoid duplicates of common questions from being asked.

5
  • 3
    Yeah and would you encourage users to try chaptGPT first then if the solution doesn't work only then post to SO? Dec 29, 2022 at 8:57
  • 2
    I wouldn't wait for a whole for an answer . I would rather try the incorrect one and learn something Dec 29, 2022 at 8:57
  • 17
    @PubliusFlaviusTiberium so you're saying you're incapable of "Trial and Error", and you need AI to help you with that? I am not sure whether the end of SO should be your main concern...
    – Alvi15
    Dec 29, 2022 at 9:28
  • 8
    I would not encourage use of ChatGPT as it is now. Maybe a couple of years into the future. But chances are that if you can ask ChatGPT, this topic may already be covered on Stack Overflow.
    – Dharman Mod
    Dec 29, 2022 at 9:49
  • 11
    “Yeah and would you encourage users to try chaptGPT first then if the solution doesn't work only then post to SO?” - Ask it how to program that hasn’t been done a million times by freshman college Students in their Introduction to Programming class. Every example you have provided you think is amazing is something any experienced programmer can do in their sleep. Dec 29, 2022 at 10:45
31

Perhaps some future AI program will be able to reliably answer SO questions, but ChatGPT certainly cannot do it. And even if you gave it ten times as much training data, it still couldn't do it, no matter how good that training data was.

GPT-3, the large language model underlying ChatGPT, is certainly an impressive piece of language processing software. But that's all it is. ChatGPT is not intended to be an artificial intelligence, although it may give some people the initial impression that it has some intelligence.

ChatGPT generates plausible text, consistent with its training data and input, but it doesn't know what it's talking about, and it has no way of representing or evaluating the truth of its utterances. Yes, it can say true things, but it can also say complete nonsense, and it can't tell the difference.

ChatGPT does not attempt to make truthful or even logical statements. Its job is to create "completions" of the text you feed it. It's ultimately doing the same kind of tasks as the spelling-checker and word suggestion feature supplied with modern phone keyboards, except that it's working with deeper and broader linguistic structures.

It has no problems making grammatically correct utterances with no spelling errors, and those utterances are linguistically consistent with the input you feed it and its training data. But it's only operating on linguistic structure, it is oblivious to the underlying logic and truth value of the statements it makes. It may emit text which is logically valid and true, but that's just a fortunate side-effect of the linguistic structure and its training data. In extended passages, it can easily say things that are logically inconsistent. And it has no way to evaluate how relevant its utterances are to the input data, apart from the linguistic consistency. So when it says true and useful things that's just a lucky coincidence.

ChatGPT often makes stupid errors of logic and silly arithmetic errors. OTOH, we have theorem-proving AI that can handle complex logic, and sophisticated mathematical systems that can do arithmetic, algebra, and calculus. In the future, we will surely have natural language software that can handle logic and mathematics, but the current incarnation of ChatGPT won't magically gain those abilities merely by throwing more training data at it.


Stephen Wolfram has written some excellent articles about how ChatGPT works, its limitations, and the potential benefit of combining it with Wolfram | Alpha. Please see What Is ChatGPT Doing … and Why Does It Work? and Wolfram|Alpha as the Way to Bring Computational Knowledge Superpowers to ChatGPT.

4
  • yes with the example you provided in a comment. I think it's beyond perfect but with time it will be improved as the research organization that built it are constantly studying it to discover ways on making it almost perfect which when happens will make SO obsolete Dec 29, 2022 at 13:51
  • 1
    @PubliusFlaviusTiberium OK, but that hasn't happened yet and doesn't appear to be anywhere near happening.
    – ggorlen
    Dec 29, 2022 at 19:08
  • 2
    I'm sorry, @Publius, but I can't tell what "yes with the example you provided in a comment" is supposed to mean, and the rest of your comment isn't very clear, either. Perhaps you disagree with my claim that ChatGPT will not become capable of creating reliable SO answers merely by improving the quality and quantity of its training data, or you simply don't understand my argument regarding that claim.
    – PM 2Ring
    Dec 29, 2022 at 21:37
  • @PM2Ring, the logical fallacies of Open chat gpt, I asked it the same question. if it takes 2 cars hours to travel from DC to Washington how long will it take 4 cars and it replies with 4 hours Dec 30, 2022 at 5:24
17

I've been using ChatGPT to answer programming questions for weeks. Unless I've been manipulating it poorly, quite simply, ChatGPT takes a great deal of prodding and pleading to get a complete, correct response from, when a correct response is achievable at all. It's a tool, basically a jacked-up autocomplete that's wrong, say, more than half the time.

For example, here's some code it wrote for an extremely trivial programming task, finding the minimum element in an enumerable in Elixir:

def find_smallest(numbers) do
  Enum.min(numbers)
end

Fine and well. I asked it to translate this code to Dart and it provided this wrong answer[1]:

To solve the problem of finding the smallest number in a list in Dart, you can use the min function from the dart:math library.

Here is an example of how you could implement the find_smallest function in Dart:

import 'dart:math';

int findSmallest(List<int> numbers) {
  return numbers.min();
}

You can then use the expect function from the package:test library to write test cases for the function.

[bunch of test cases that the function fails on]

This is the sort of plausible-looking ChatGPT-based answer you'd see flooding Stack Overflow before the ban.

I'm not a Dart programmer, so after this failed, I looked it up on SO to find the human solution. This isn't an isolated incident. It's the typical workflow using ChatGPT at the present time.

There's no doubt that it's an amazing piece of technology that's correct some of the time. It's helped me quickly solve certain problems over the past few weeks. I don't doubt that it solved the problem you mentioned for you. When successful, it's magic. The problem is, it reveals itself as unreliable once you put it through the ringer on a variety of problems. I'm not talking about 80% accuracy, from what I've seen, more like 40% accuracy.

But technology is advancing and we are becoming more and more independent of human skill to find solutions to programming challenges

Not really. Human skill is just as important as ever. Nowadays, one needs to be extremely judicious about separating fact from fiction and separating plausible-looking solutions from truly correct ones.

Unless large strides are made soon, ChatGPT is nowhere near replacing Stack Overflow or programmers in general any more than self-driving mode will be replacing human drivers.


[1]: I'm aware that asking the same question again, differently or at a later time may give the correct answer for this problem. That doesn't change the fact that it fails catastrophically like this often, on simple tasks.

1
  • 6
    In this comment I linked to a ChatGPT answer posted by a 12k+ member (before the ban). The answer text seems to make sense, and the code looks good, but they're rubbish. The loop should use continue to skip duplicates, but instead it uses break, so the loop ends when it finds the first dupe, and hence the output array is truncated. But statistically, break is much more common than continue...
    – PM 2Ring
    Dec 29, 2022 at 22:04
12

Your thoughts and views on this matter are welcome...

Speed yes, accuracy no. One example is not enough to draw any conclusion. Please conduct a more thorough investigation.

The speed of the response generation of the service you used is indeed impressive, although I wouldn't say it's necessarily much faster than using a search engine like Google and clicking on the presented link to Stack Overflow, for example.

Accuracy is unfortunately not yet that good. In same cases it gave acceptable results, but in others not, when I tested it. I cannot recommend it yet.

Why is your and my experience so vastly different? Maybe because you drew a conclusion from only a very small amount of examples (N=1?). That cannot work. There hasn't been done enough example generation and evaluation for the claim to be supported by the evidence.

It's futile to discuss the implications of something we do not know with sufficient certainty yet. We need to study this service more. I'm quite sure that scientists do that already.

Just as a side comment. ChatGPT is competition (capable or incapable) for this platform, but for a knowledge seeker it's just another (useful or not useful) source of information. How useful? The jury is still out.

10
  • 1
    your the first person to acknowledge GPT as a competition to stackoverflow. Let's see how this will affect stackoverflow traffic. Will SO still get 5000 questions a day or will this drop? Dec 31, 2022 at 10:23
  • 1
    @PubliusFlaviusTiberium I'm not the first one really but also not the last one. The future is difficult to predict. I know some people who use ChatGPT for programming but they might not have asked a question here anyway. Let's just wait and see. Not only the number of questions might drop maybe also the number of visits (and this would be the real loss for this platform).
    – Trilarion
    Dec 31, 2022 at 10:58
  • but i wouldn't want SO to be knocked off its perch by that site, SO can get back at them by adding new features like animations and more themes to make the site even more exciting Dec 31, 2022 at 11:02
  • 3
    @PubliusFlaviusTiberium What will happen is not necessarily what you want. I don't think there is any imminent risk for SO (more for Google maybe) because SO shines with expert content and high quality content is much more difficult to create than average quality content. On the other hand if that happens I would embrace it. That's progress. I also do not want to go back to write letters instead of sending text messages just because that keeps the postman busy...
    – Trilarion
    Dec 31, 2022 at 11:19
  • @evolution. making life easier and easier with technology results in other technologies being obsolete. Dec 31, 2022 at 11:25
  • ("animations" may refer to the slow type out of text, like on ancient 110 baud terminals, telex terminals, and old films (typewriter-like).) Dec 31, 2022 at 15:46
  • 1
    Cathode ray tube television was really cool for a long time until say 20 years ago.
    – Trilarion
    Dec 31, 2022 at 16:28
  • I don't know why you would think that a timed display of each character on a post (aka, simulating typing) would be an important addition to the site. It's certainly enticing from a certain perspective, but it doesn't do anything aside from looking weird or fancy depending on where you stand. @PubliusFlaviusTiberium Jan 3 at 8:58
  • 1
    A scene from the film (at 02 min 14 secs). May 12 at 15:33
  • And here (at 01 min 21 secs). May 12 at 15:58
0

ChatGPT will be useful to StackOverflow as the number of poor questions will dramaticaly decrease as they will be redirected to ChatGPT.

The ChatGPT user will be puzzled by a non-working solution and will came here to ask a more intense and minded question.

-2

Meanwhile I read frequently in the questions statements like this: "Even ChatGPT was not able to answer my question, so I try it here on SO". Typically it applies to rather basic questions but these questions are not necessarily "bad" or "poor" questions, they are just normal.

So I don't think ChatGPT can replace StackOverflow in the near future.

I asked "What the result of 968416854 + 8694165478653436"

First answer: 8694165479621854 - which is wrong!

The proof (really hard to believe):

enter image description here

2
  • 4
    If you're bad at asking questions then nothing can help you. Asking questions isn't hard, asking good questions is. Any question can be answered, only good questions get a useful answer. That is why we have help center articles for how to ask, that is why Jon Skeet blogged about the perfect question, that is why we have the ask wizard, staging ground. The technology isn't the problem. The demand for instant gratification without any effort within 140 characters or a 15 seconds tiktok is the norm today. You can't beat laziness.
    – rene
    May 15 at 7:19
  • No technology is perfect just like the human beings who built the AI. No programmer is perfect. Being bad is good for learning and progress, we learn through mistakes. May 15 at 7:59
-9

Some dudes are shooting the messenger because they don't like the message.

Here is an answer from ChatGPT itself:

While ChatGPT is a powerful tool for answering a wide range of questions, Stack Overflow still serves a valuable purpose as a community-driven platform for technical Q&A. Rather than replacing Stack Overflow, ChatGPT could be used in conjunction with the site to provide additional support and guidance for users.

One possibility would be to integrate ChatGPT's capabilities into Stack Overflow's existing interface, allowing users to ask questions in a conversational format and receive instant responses. This could potentially help users who are struggling to formulate their questions or who are looking for guidance on how to approach a particular problem.

However, it's important to consider the potential drawbacks of such an approach, such as the risk of information overload or the possibility of inaccurate or incomplete responses. Ultimately, any changes to Stack Overflow should be carefully evaluated to ensure that they align with the site's goals of providing accurate, high-quality technical information to users

2
  • 6
    "Some dudes are shooting the messenger because they don't like the message." The downvotes here are on the answer, i.e. the message. The messenger hasn't been shot at all here. However, it might help the answer if the "messenger" would provide some reasoning to why they requested and delivered a message from an uninvolved writer for recipients who could have asked the writer themselves if they cared. Feb 14 at 15:50
  • I agree with you @MisterMiyagi. Your argument makes sense to me.
    – zell
    Feb 15 at 11:43
-9

ChatGPT largely is Stack Overflow, albeit trimmed and polished to match the user's request. So the ironic thing is, if everyone started using ChatGPT instead of SO, a significant source of ChatGPT's source of knowledge would dry up.

What to do? Stack Overflow should include provision for people to post their questions and answers from ChatGPT. These could be commented upon, and voted up or down as with any post or its replies

Doing this would: 1) benefit the community (instead of remaining private), 2) provide corroboration, correction, and refinement for the poster, and 3) give ChatGPT more knowledge to digest. In this way, we harness the incredible power of ChatGPT, help it improve, and keep the real source of knowledge --which comes from humans-- from drying up.

14
  • 6
    It's already been clearly demonstrated that this can't work, due to what would be an immense requirement for subject-matter experts to review the content and the disparity between the volume of such evaluations and people evaluating the answers solely on if it sounds good, rather than it actually being good. As has been mentioned many times, the nature of the AI generated content and how most people perform evaluations results in the AI content often being upvoted even when disastrously wrong and/or internally inconsistent.
    – Makyen Mod
    Apr 12 at 22:15
  • However many times something is mentioned has no connection to its truth..unless you're some kind of follower-no-matter-what. That out of the way, people are missing the fact that this is how intelligence itself works. It's how we humans work. The brain is a prediction machine. Check out the work of Jeff Hawkins who pioneered the neuroscience of intelligence and whose work is surely responsible for the boon in AI today. Takeaway: we have to learn to live with the machines; not fight them. So strange to see "programmers" to rebel against the greatest programming feat of humanity. Apr 12 at 23:14
  • 3
    Sigh Of course the number of times something is repeated doesn't affect its accuracy (if it affects "truth" is a philosophical question). What the fact that it's been mentioned many times and that you don't address the issue in your answer (not even to say you just don't believe the people who have said such things, who very likely have a much better view of what's going on than you do) tends to indicate that you've not actually taken a detailed look at the issue and/or don't have much background in the area and are just stating your personal, not well researched, opinion.
    – Makyen Mod
    Apr 13 at 0:06
  • 3
    What you need to address with such a suggestion is A) how to resolve the orders of magnitude difference between the level of participation from subject-matter experts which such a plan requires to do what you're claiming, and
    – Makyen Mod
    Apr 13 at 0:15
  • 5
    B) the disparity between the much lower volume of evaluations by experts (votes/comments) vs evaluation by people who are basing their evaluation primarily upon if the text reads/sounds good, rather than if it's actually correct. As things stand, there are substantially more people evaluating things based on if it reads/sounds good (something at which current AI generated content excels), rather than if it's correct (something which current AI generated content doesn't even attempt to do).
    – Makyen Mod
    Apr 13 at 0:15
  • 4
    And, while there are some people who are against the concept of AI generation of content, the primary issue here on Stack Overflow is that the current capability of AI generated content is crap from the point of view of providing accurate answers, largely because the current generation of chat bots make absolutely zero attempt at being correct. What they do is string text together that their model predicts goes well together. That's great, and a really good accomplishment, for a chat bot. It's terrible as something to consistently provide an actual, correct answer.
    – Makyen Mod
    Apr 13 at 0:21
  • 1
    "ChatGPT largely is Stack Overflow" Not really. Yes, SO did form a significant part of the GPT training data on coding-related matters, but so did Github, Wikipedia, Reddit, and various other sources in the Common Crawl dataset. However, even if you train GPT from scratch on nothing but high quality curated perfectly correct data it can still produce incorrect output because it's just operating on syntax. It has no capability to perform logical analysis or to verify the validity of its utterances.
    – PM 2Ring
    Apr 13 at 2:04
  • I have asked ChapGPT 3 things today. 1) Is disconnecting the negative terminal of a battery a valid way to test an alternator?, 2) How do you setup a 2-dimensional array in PHP?, and 3) How do you return 2 dimensional arrays from php to javascript? It gave me lucid. to-the-point, and (I checked) accurate answers in all 3 cases (all snark-free!). It's on me to verify, but the same is true for whatever I read here on S.O. This comment I am writing now, along with what others have said, will end up in the bowels of ChatGPT. The machine is part of our lives. There is no going back. Apr 13 at 2:21
  • 2
    ChatGPT is pretty good at answering questions that it has already seen the answer to, yes (AKA: duplicate questions). That makes a generative AI like this a potentially decent way of searching for existing answers. The problem is that it doesn't know which things it doesn't know, and so it won't tell you when it's completely made things up out of thin air. Most humans at least try to only answer questions they think they know the answer to, and they can cite sources to support their answer. If you ask ChatGPT to cite sources, it makes up links that don't exist.
    – Ryan M Mod
    Apr 13 at 2:29
  • Thank you, Ryan M. That is exactly what I was saying in my answer. If people foolishly abandon public forums like S.O., we will be killing the goose that lays the golden eggs that ChatGPT needs in order to work. Given that ChatGPT is so convenient, people will be tempted to do just that. And that's why I would encourage socially-minded people to not just take their GPT answers and run, but to put them up on places like S.O. for public discussion. That will be good for the posters and for the machine. It would --hopefully-- prevent the sort of "forum desertion" that could tragically occur. Apr 13 at 2:36
  • I don't think you would care given your arguments, but maybe reading a related post on the subject might be interesting: meta.stackoverflow.com/a/422019/12349101 Apr 13 at 2:39
  • 1
    Given that GPT is primarily good at answering questions that have already been answered (quite likely on Stack Overflow), I'm not sure I see how putting GPT answers on Stack Overflow would increase the total amount of knowledge contained here. I'd also expect that a lot of the other issues it's good at spotting are the sorts of "debug my code for me" questions that also aren't particularly useful for future users.
    – Ryan M Mod
    Apr 13 at 2:40
  • Everyone is (rightly) saying that ChatGPT can spit nonsense that sounds intelligible. It's for cases like this that it would be helpful to see public correction. I'd go one step further, the people at OpenAI seem like public spirited types. Maybe Q&A's from their sessions could automatically be put out on public forums (like S.O., if it consented). We have to keep humans in the loop because if people start using ChatGPT for everything and stop publicly talking to one another, the information source will dry up. (Maybe we could get the NSA to start feeding it data...Whoa, just kidding) Apr 13 at 3:03
  • "However many times something is mentioned has no connection to its truth.." That also applies to claims - implicit or explicit - that SO has the curation capacities to handle content primarily created by ChatGPT or similar models. In contrast, many of the active curators on SO have voiced their unwillingness to waste their time on such content, and on many other occasions that they are already at their limits - something they can actually accurately describe. Apr 13 at 4:59
-10

I also verified today that the chat bot cannot beat human experienced responses to programming questions, here is how I verified this, I asked chat GPT a very technical question and it kept displaying the same text over and over again and the solution posted by chat GPT either compiles and refuses to work or the code does not compile. I reached to a developer on quora asked him the same question and his response worked and was straight to the point.

0
-11

This has got to be the most amazing piece of technology ever created. I tried to beat a Leet code challenge that needed me to get the difference of two arrays where the return object should be of type IList<IList<int>> and the first index should contain distinct elements in array1 that are not in array two while the second index should contain distinct elements of array 2 that are not in array one. I tried to use this code which failed some test cases

  public IList<IList<int>> FindDifference(int[] nums1, int[] nums2)
    {
        IList<IList<int>> list = new List<IList<int>>();
        //get all the distinct elements that are in nums1 and not in nums2
        list.Add(nums1.ToList().Where(x=>nums2.Contains(x)==false).ToList());
        list.Add(nums2.ToList().Where(x=>nums1.Contains(x)==false).ToList());       
        return list;
    }

I pasted the method to chat GPT and told it that my LINQ code failed some test cases. It provided an optimization with this code.

  public IList<IList<int>> FindDifference(int[] nums1, int[] nums2)
    {
        IList<IList<int>> list = new List<IList<int>>();
        list.Add(nums1.Except(nums2).ToList());
        list.Add(nums2.Except(nums1).ToList());
        return list;
    }

Thank You Open AI for this, you have reduced over dependence on human skill which saves time. My learning curve just got steeper ever since I started using this piece of technology in my day to day routine. Aim to be better than the best guy working at Open AI on this project because, this is gonna dominate the AI field forever. Aim to be better than the programming minds at Open AI pioneering this advancement. One Leap at Open AI is a giant leap in AI advancement for humanity. I thought for a moment there that if there is no programming challenge that chat GPT can't solve, this thing could be a programming god. A rookie programmer armed with GPT 4.0 can beat an experienced programmer at any task, credit to chat GPT. Wherever Microsoft throws money for research, you know something big is gonna go down. Next step is to let AI control computer hardware and do the programming.

11
  • 2
    It's fine that you like GPT, and it's fine that GPT helps with simple programming tasks like this. You're not making any coherent points on the core topic of the discussion: What does this mean for us (SO)? yesterday
  • @General Grievance, It means that you should embrace the tool, remove all down votes against this post and accept that chat GPT is better. Improve the site to keep up with the competition, I laughed when I saw that government agencies such as the FBI are stakeholders on this site. If they are here to spy, good. If they are here to hire, it's sad. yesterday
  • 4
    I might be missing something here. A group of common, publicly available questions with publicly available solutions, using common, publicly available approaches is practically what ChatGPT has been made for. This is a bit like praising a pocket calculator for being good at calculating. Googling gets you almost the same solution and the same approach - though both resources would have taught you quite better approaches as well. yesterday
  • The NSA, the CIA, NASA are all stakeholders on a privileged online Q and A site? If it were not for hiring then I would be cracking my ribs right now. yesterday
  • @MisterMiyagi, try googling and see if you you are lucky with any real SE challenge. Like building and training a neural network in. NET. You will find solutions online of course lol. And by online do you mean SO where similar questions are ignored? yesterday
  • 4
    You seriously need to stop derailing your own discussions... It's really not a good look. yesterday
  • 2
    @PassionateSE Your point being? The very example you have cited isn't a real SE challenge. If you have a case of ChatGPT providing a ground-breaking answer to a question that would be suitable for SO and not extremely similar to its learning corpus, that would be a good example and point. yesterday
  • @Passionate SE You mention in your profile bio that you need to "take care of my reputation today and make sure am getting better". It does not really seem like you care about your reputation.
    – Nexevis
    yesterday
  • @Nexevis, do you mean that I will never be elected as a moderator if m meta posts have lots of negative scores? 23 hours ago
  • This seems contradictory with your earlier answer. What point are you trying to make here, exactly?
    – Cody Gray Mod
    6 hours ago
  • @Cody Gray, My earlier answer wasn't based on an actual programming challenge experience. 6 hours ago

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .