I'm gonna throw my non-existing hat in the ring and make an answer, to hopefully clear misunderstanding/confusion (or future ones).
I'm gonna reuse and quote what you said so as to better explain:
ChatGPT is a tool. We should adopt all useful tools.
Right, I agree it's a tool, and while its usefulness is questionable (eg: it depends on how it is used, the probability of getting a useful answer/output, etc) that does not have anything to do with what SO and SE stand for.
This site thrives for high-quality answers (and questions), by valuable contributors. If you have as others pointed out, to fully check, understand, and retry (eg: using the Try again
button, or repasting the same input) multiple times until you get what you "seek", this isn't what I call high-quality answer, but more like "throwing something until it sticks".
That's not even the tip of the iceberg in terms of cons, but I'll get to that in a second.
This site is in the best position to do this adoption. I propose that before the question is published the OP gets the opportunity to read the answer from the AI.
And then what? They'll get confused when a website's main goal is to provide high-quality answers and questions end up giving you a completely wrong one, with no one to blame but a Statistical Learning algorithm?
Worse case, they repost that as a question while mentioning they don't understand why the "AI" gave them a wrong answer. Multiply that by 10k per day, and that's the kind of thing that will flood the site.
If you have a problem and a "tool" gave you another set of problems that you don't even know how to solve, then I think your main problem is depending on the "tool" in the first place.
OP might already accept this answer. Or they might realize that the question needs refining. Or they might be struck with inspiration and find the answer by themselves based on incorrect answer from AI. These are all good outcomes.
Having inspiration is fine. For example, I found that just by writing an MRE, I usually always find the answer to my question while I write it. That's also why I don't always post questions these days.
But when you depend on something that does not even understand context, then clearly, you'll end up having the wrong mindset more often than not to solve this by yourself. We all used/tried a search engine to find inspiration at least once for a problem, but this is worse than that because it doesn't have:
Accountability (yes, you can blame whoever trained/made it, but they'll end up blaming the dataset or the "AI" since as mentioned, it's just marketing noise).
Attribution: It doesn't even show you credit/attribute it to the original writer of whatever it's writing. Sure, if you believe that it's intelligent, it might make sense to think it just made that itself, but that's again, not how this works.
Everything in any GPT3 (and by extension, most "AI" tools) are just mutated input that was used when training it. It's technically more complex than that, but, when you get result like this, you really start to notice a pattern here (this is for copilot
, but it's also using gpt3 like chatgpt is, albeit differently, so the same apply here).
Don't even get me started on the plagiarism implication this brings, as that would be way out of scope for this post I think.
- Understanding. As others pointed out, it often does not actually correctly state explanations for things. Excerpt one, two. I have more, but sadly there is a limit of character, so you get the idea. If someone has a problem because they don't know or understand how to do something, and are clueless, but totally hopeful that this "thing" will give them what they want, will totally become dependent on this, and as a result, become extremely surprised whenever it's wrong or someone else contradicts what's written there. It also does not understand context even if the result might align with the input.
Adopting the AI by the site has nothing to do with banning the use of AI to provide answers. These two policies can coexist.
Sadly, they cannot coexist, especially because you cannot trust that people will think logically when using this, and not trust it/try to understand it instead.
As I said in the comments, "AI" is just a term/word for marketing. The real term would be Neural Network, or more correctly Statistical Learning. AI does not know, does not understand and is in no way intelligent, even if the result sometimes says otherwise.
That doesn't mean it's not useful, but the TLDR is: This doesn't align with the goal of SO/SE.