-105

ChatGPT is a tool. We should adopt all useful tools. This site is in the best position to do this adoption. I propose that before the question is published the OP gets the opportunity to read the answer from the AI. The OP might already accept this answer. Or they might realize that the question needs refining. Or they might be struck with inspiration and find the answer by themselves based on an incorrect answer from the AI. These are all good outcomes.

Adopting the AI by the site has nothing to do with banning the use of AI to provide answers. These two policies can coexist.

42
  • 50
    Or they might waste their time reading and trying a likely completely useless answer. That's bad both for them and for our reputation as a site with high quality answers. Dec 9, 2022 at 10:14
  • 17
    We should adopt all useful tools... Just no. Tools should only be adopted if there's a clear benefit of having it incorporated (instead of having users be able to use it without integration), if it aligns with the purpose of the site, and if it makes business sense (it's free in a trial phase now, plus even free tools cost dev time = money).
    – Erik A
    Dec 9, 2022 at 10:17
  • 59
    You are probably not aware how wrong can the results be. Prevent SQL injections. In the browser. The code and answer are very confidently complete BS. Yes, some answers it gives are decent. But if a person is asking, they probably can't really differentiate. One answer I saw is where it used a language feature which does not exist. It was a proposal but ultimately withdrawn in 2015. Yet, it generated an answer confidently stating it's the way to solve a particular problem.
    – VLAZ
    Dec 9, 2022 at 10:18
  • 5
    If we don't adopt this AI or the next we will get replaced, just like how SO replaced Experts Exchange. Dec 9, 2022 at 10:20
  • 30
    I don't think there's any evidence that not incorporating AI will get us replaced, but even if that happens and we do get replaced by a superior product with AI, then that's the companies problem, not ours. We just get to move to the better product. Experts exchange getting replaced by SO very much was a good thing, imho.
    – Erik A
    Dec 9, 2022 at 10:25
  • 28
    AI is just a word made for marketing noise. The real word some people like to use is Neural Network, or even Statistical Learning. AI does not know, does not understand and is in no way intelligent, even if the result sometimes say otherwise. Yes, "AI" tools can be useful, but as others better said, this isn't aligning with the focus of SO/SE for high-quality answers Dec 9, 2022 at 10:26
  • 2
    While OP is composing the answer a list of existing answers is presented to them. How is this list useful to me? I already googled. No clear benefit from my standpoint. So, why do we like this list, and don't like something similar, but also entirely different? OP can ignore the AI answer just as well. But it also might provide genuine help. It may happen. Dec 9, 2022 at 10:30
  • 3
    The problem is answerers are not using it as a tool.. They are just blindly copying from ai response and pasting
    – Suraj Rao
    Dec 9, 2022 at 10:32
  • 22
    ChatGPT may return different and contradicting answers when retried multiple times with the same prompt (question). How many times the asker have to retry it? Or well, why does it have to be done on SO?
    – Andrew T.
    Dec 9, 2022 at 10:38
  • 5
    @VLAZ Hey, it's not SQL injection if you provide an endpoint to execute arbitrary SQL against your database but process parameters separately. Then it's just SQL. So it's technically correct = the best kind of correct unless you don't want your database leaked/erased 🤡
    – Erik A
    Dec 9, 2022 at 11:22
  • 8
    "We should adopt all useful tools." - You can. You are very free to use the tool BEFORE you dump a question on Stack Overflow, and maybe save yourself the time. This is an alternative to Stack Overflow, not an extension.
    – Gimby
    Dec 9, 2022 at 12:37
  • 3
    “I propose that before the question is published the OP gets the opportunity to read the answer from the AI.” - No; I have literally read hundreds of answers generated by ChartGPT, and they all have, been incredibly incorrect. ChartGPT in general generates content that isn’t helpful. “If we don't adopt this AI or the next we will get replaced, just like how SO replaced” - We don’t have to adapt this particular AI, given in my experience, the AI generates low quality answers. ChartGPT isn’t even a real AI. Dec 9, 2022 at 12:46
  • 14
    I'm not gonna bother contesting the closure in either direction, but please don't vote to delete this question. We've had two duplicates today alone. With the banner up and many people repeatedly having this idea, leaving this as a dupe target saves us of having this discussion over and over for when this inevitably shows up again very soon
    – Zoe is on strike Mod
    Dec 9, 2022 at 21:02
  • 3
    Presenting users who are having trouble interpreting code with an answer that likely looks correct but could be wrong/dangerous without an opportunity for an expert to review said answer is dangerous. It can reinforce bad practices, suggest vulnerabilities, create an X/Y situation, etc
    – Kevin B
    Feb 3, 2023 at 15:46
  • 3
    Sure, it's a tool. It's absolutely the wrong tool for the job.
    – iono
    Feb 25, 2023 at 18:50

1 Answer 1

63

I'm gonna throw my non-existing hat in the ring and make an answer, to hopefully clear misunderstanding/confusion (or future ones).

I'm gonna reuse and quote what you said so as to better explain:

ChatGPT is a tool. We should adopt all useful tools.

Right, I agree it's a tool, and while its usefulness is questionable (eg: it depends on how it is used, the probability of getting a useful answer/output, etc) that does not have anything to do with what SO and SE stand for.

This site thrives for high-quality answers (and questions), by valuable contributors. If you have as others pointed out, to fully check, understand, and retry (eg: using the Try again button, or repasting the same input) multiple times until you get what you "seek", this isn't what I call high-quality answer, but more like "throwing something until it sticks".

That's not even the tip of the iceberg in terms of cons, but I'll get to that in a second.

This site is in the best position to do this adoption. I propose that before the question is published the OP gets the opportunity to read the answer from the AI.

And then what? They'll get confused when a website's main goal is to provide high-quality answers and questions end up giving you a completely wrong one, with no one to blame but a Statistical Learning algorithm?

Worse case, they repost that as a question while mentioning they don't understand why the "AI" gave them a wrong answer. Multiply that by 10k per day, and that's the kind of thing that will flood the site.

If you have a problem and a "tool" gave you another set of problems that you don't even know how to solve, then I think your main problem is depending on the "tool" in the first place.

OP might already accept this answer. Or they might realize that the question needs refining. Or they might be struck with inspiration and find the answer by themselves based on incorrect answer from AI. These are all good outcomes.

Having inspiration is fine. For example, I found that just by writing an MRE, I usually always find the answer to my question while I write it. That's also why I don't always post questions these days.

But when you depend on something that does not even understand context, then clearly, you'll end up having the wrong mindset more often than not to solve this by yourself. We all used/tried a search engine to find inspiration at least once for a problem, but this is worse than that because it doesn't have:

  1. Accountability (yes, you can blame whoever trained/made it, but they'll end up blaming the dataset or the "AI" since as mentioned, it's just marketing noise).

  2. Attribution: It doesn't even show you credit/attribute it to the original writer of whatever it's writing. Sure, if you believe that it's intelligent, it might make sense to think it just made that itself, but that's again, not how this works. Everything in any GPT3 (and by extension, most "AI" tools) are just mutated input that was used when training it. It's technically more complex than that, but, when you get result like this, you really start to notice a pattern here (this is for copilot, but it's also using gpt3 like chatgpt is, albeit differently, so the same apply here).

Don't even get me started on the plagiarism implication this brings, as that would be way out of scope for this post I think.

  1. Understanding. As others pointed out, it often does not actually correctly state explanations for things. Excerpt one, two. I have more, but sadly there is a limit of character, so you get the idea. If someone has a problem because they don't know or understand how to do something, and are clueless, but totally hopeful that this "thing" will give them what they want, will totally become dependent on this, and as a result, become extremely surprised whenever it's wrong or someone else contradicts what's written there. It also does not understand context even if the result might align with the input.

Adopting the AI by the site has nothing to do with banning the use of AI to provide answers. These two policies can coexist.

Sadly, they cannot coexist, especially because you cannot trust that people will think logically when using this, and not trust it/try to understand it instead.

As I said in the comments, "AI" is just a term/word for marketing. The real term would be Neural Network, or more correctly Statistical Learning. AI does not know, does not understand and is in no way intelligent, even if the result sometimes says otherwise.

That doesn't mean it's not useful, but the TLDR is: This doesn't align with the goal of SO/SE.

18
  • 32
    If you have a problem and a "tool" gave you another set of problems.. Ah.. the XY problem became X-AI problem
    – Suraj Rao
    Dec 9, 2022 at 11:35
  • 3
    maybe X-WHY will work too, given as I mentioned, will lead to more people asking why this doesn't work and "btw this is from chatgpt :D" Dec 9, 2022 at 11:49
  • 1
    @SurajRao: ...a "tool" gave you another set of problems... I imagine a big box full of shoes and glass bottles...
    – honk
    Dec 9, 2022 at 11:58
  • 3
    "Worse case, they repost that as a question while mentioning they don't understand why the "AI" gave them a wrong answer." I have already seen this happening. Feb 9, 2023 at 23:44
  • 2
    The fundamental problem here is that a some of ChatGPT's model comes from StackOverflow. When OpenAI updates their model post-2021, it's going to have to figure out how to filter out a lot of the incorrect information it generated.
    – Aaron_H
    Feb 17, 2023 at 22:36
  • 1
    You know, it is a hard question is phylosophy, that a being emulating intelligence is actually intelligent or not. Behind the question, there is a not admitted but somehow... heard "side question", do we have a soul. We can see this thing as a machine language processor capable to process the best google hit to an answer. We can see it as a living, thinking being. I am not a phylosopher, and I have (strong) beliefs in the matter, but I am not an expert. I can say, what do I feel. I feel fear. I feel fear that everything destructs what was the life and technology for me today.
    – peterh
    Mar 30, 2023 at 23:31
  • @Aaron_H Probably people talking with him, will correct him. If it is an enough good AI, it will understand the believable proofs that he tought it badly, and it updates his model.
    – peterh
    Mar 30, 2023 at 23:33
  • 1
    @peterh I know, and I at least agree that this is a complicated debate. It can be condensed to a simple Chinese room vs Turing test, but putting that aside, Once you know how it works (eg: statistical learning, etc) and even without taking into account the concept of soul/special human feature into account, you can at least tell there is nothing to fear, aside from what people do with them, rather than the "tool" themselves. Mar 31, 2023 at 0:06
  • (cont2). I do agree that intelligence is a difficult concept to fully and entirely explain in details. What separate us from XYZ? I'm not an expert either, but I do believe it is in part the understanding of context. Statistical algorithm need to bruteforce using random trick such as learning rate and weights, and etc to even "grasp" the smallest of "pattern" that we understand, but we do not need to do that. Mar 31, 2023 at 0:08
  • (cont 3) If someone goes back in time and show off a mini size computer, everyone would think it is "intelligent", but to most people, even people agreeing on the whole "AI is real" concept, would most likely not think of a single silicon chip as intelligent. But to people less used, and perhaps, not as knowledgeable on the matter, maybe. Mar 31, 2023 at 0:10
  • (last cont) I said it earlier and I'll say it again, AI is just a hype and the real things it represents isn't what it tries to convey. It's fine if people still like to think or believe whatever they want, regardless if they know how things actually work or not. That still doesn't make their state any less truthful, though. Mar 31, 2023 at 0:14
  • @NordineLotfi Considering the history, what do you think, what will people do with them?
    – peterh
    Mar 31, 2023 at 0:30
  • @NordineLotfi Now AI can overthink the best chess players of the world. Next time AI will be able to overthink all the politicians of the world. We won't see about it anything, as we can not see any special in a chess game between two 3000+ ELO machines. They step, step, and once one of them mates. Not even the world chess champion would understand, why they stepped on this way. By analogy, we would see that so that one of the countries becomes suddenly surprisingly successful in the world politics, close to become a "world master".
    – peterh
    Mar 31, 2023 at 0:34
  • 1
    Let us continue this discussion in chat. Mar 31, 2023 at 0:39
  • 2
    Chinese room: the Chinese room argument. "a digital computer executing a program cannot have a "mind", "understanding", or "consciousness", regardless of how intelligently or human-like the program may make the computer behave." May 12, 2023 at 14:32

Not the answer you're looking for? Browse other questions tagged .