Timeline for ChatGPT seems to be better than Stack Overflow, both in speed and accuracy—what does this mean for us?
Current License: CC BY-SA 4.0
15 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Apr 13, 2023 at 4:59 | comment | added | user5349916 | "However many times something is mentioned has no connection to its truth.." That also applies to claims - implicit or explicit - that SO has the curation capacities to handle content primarily created by ChatGPT or similar models. In contrast, many of the active curators on SO have voiced their unwillingness to waste their time on such content, and on many other occasions that they are already at their limits - something they can actually accurately describe. | |
Apr 13, 2023 at 3:03 | comment | added | user15436681 | Everyone is (rightly) saying that ChatGPT can spit nonsense that sounds intelligible. It's for cases like this that it would be helpful to see public correction. I'd go one step further, the people at OpenAI seem like public spirited types. Maybe Q&A's from their sessions could automatically be put out on public forums (like S.O., if it consented). We have to keep humans in the loop because if people start using ChatGPT for everything and stop publicly talking to one another, the information source will dry up. (Maybe we could get the NSA to start feeding it data...Whoa, just kidding) | |
Apr 13, 2023 at 2:40 | comment | added | Ryan M Mod | Given that GPT is primarily good at answering questions that have already been answered (quite likely on Stack Overflow), I'm not sure I see how putting GPT answers on Stack Overflow would increase the total amount of knowledge contained here. I'd also expect that a lot of the other issues it's good at spotting are the sorts of "debug my code for me" questions that also aren't particularly useful for future users. | |
Apr 13, 2023 at 2:39 | comment | added | Nordine Lotfi | I don't think you would care given your arguments, but maybe reading a related post on the subject might be interesting: meta.stackoverflow.com/a/422019/12349101 | |
Apr 13, 2023 at 2:36 | comment | added | user15436681 | Thank you, Ryan M. That is exactly what I was saying in my answer. If people foolishly abandon public forums like S.O., we will be killing the goose that lays the golden eggs that ChatGPT needs in order to work. Given that ChatGPT is so convenient, people will be tempted to do just that. And that's why I would encourage socially-minded people to not just take their GPT answers and run, but to put them up on places like S.O. for public discussion. That will be good for the posters and for the machine. It would --hopefully-- prevent the sort of "forum desertion" that could tragically occur. | |
Apr 13, 2023 at 2:29 | comment | added | Ryan M Mod | ChatGPT is pretty good at answering questions that it has already seen the answer to, yes (AKA: duplicate questions). That makes a generative AI like this a potentially decent way of searching for existing answers. The problem is that it doesn't know which things it doesn't know, and so it won't tell you when it's completely made things up out of thin air. Most humans at least try to only answer questions they think they know the answer to, and they can cite sources to support their answer. If you ask ChatGPT to cite sources, it makes up links that don't exist. | |
Apr 13, 2023 at 2:21 | comment | added | user15436681 | I have asked ChapGPT 3 things today. 1) Is disconnecting the negative terminal of a battery a valid way to test an alternator?, 2) How do you setup a 2-dimensional array in PHP?, and 3) How do you return 2 dimensional arrays from php to javascript? It gave me lucid. to-the-point, and (I checked) accurate answers in all 3 cases (all snark-free!). It's on me to verify, but the same is true for whatever I read here on S.O. This comment I am writing now, along with what others have said, will end up in the bowels of ChatGPT. The machine is part of our lives. There is no going back. | |
Apr 13, 2023 at 2:04 | comment | added | PM 2Ring | "ChatGPT largely is Stack Overflow" Not really. Yes, SO did form a significant part of the GPT training data on coding-related matters, but so did Github, Wikipedia, Reddit, and various other sources in the Common Crawl dataset. However, even if you train GPT from scratch on nothing but high quality curated perfectly correct data it can still produce incorrect output because it's just operating on syntax. It has no capability to perform logical analysis or to verify the validity of its utterances. | |
Apr 13, 2023 at 0:21 | comment | added | Makyen Mod | And, while there are some people who are against the concept of AI generation of content, the primary issue here on Stack Overflow is that the current capability of AI generated content is crap from the point of view of providing accurate answers, largely because the current generation of chat bots make absolutely zero attempt at being correct. What they do is string text together that their model predicts goes well together. That's great, and a really good accomplishment, for a chat bot. It's terrible as something to consistently provide an actual, correct answer. | |
Apr 13, 2023 at 0:15 | comment | added | Makyen Mod | B) the disparity between the much lower volume of evaluations by experts (votes/comments) vs evaluation by people who are basing their evaluation primarily upon if the text reads/sounds good, rather than if it's actually correct. As things stand, there are substantially more people evaluating things based on if it reads/sounds good (something at which current AI generated content excels), rather than if it's correct (something which current AI generated content doesn't even attempt to do). | |
Apr 13, 2023 at 0:15 | comment | added | Makyen Mod | What you need to address with such a suggestion is A) how to resolve the orders of magnitude difference between the level of participation from subject-matter experts which such a plan requires to do what you're claiming, and | |
Apr 13, 2023 at 0:06 | comment | added | Makyen Mod | Sigh Of course the number of times something is repeated doesn't affect its accuracy (if it affects "truth" is a philosophical question). What the fact that it's been mentioned many times and that you don't address the issue in your answer (not even to say you just don't believe the people who have said such things, who very likely have a much better view of what's going on than you do) tends to indicate that you've not actually taken a detailed look at the issue and/or don't have much background in the area and are just stating your personal, not well researched, opinion. | |
Apr 12, 2023 at 23:14 | comment | added | user15436681 | However many times something is mentioned has no connection to its truth..unless you're some kind of follower-no-matter-what. That out of the way, people are missing the fact that this is how intelligence itself works. It's how we humans work. The brain is a prediction machine. Check out the work of Jeff Hawkins who pioneered the neuroscience of intelligence and whose work is surely responsible for the boon in AI today. Takeaway: we have to learn to live with the machines; not fight them. So strange to see "programmers" to rebel against the greatest programming feat of humanity. | |
Apr 12, 2023 at 22:15 | comment | added | Makyen Mod | It's already been clearly demonstrated that this can't work, due to what would be an immense requirement for subject-matter experts to review the content and the disparity between the volume of such evaluations and people evaluating the answers solely on if it sounds good, rather than it actually being good. As has been mentioned many times, the nature of the AI generated content and how most people perform evaluations results in the AI content often being upvoted even when disastrously wrong and/or internally inconsistent. | |
Apr 12, 2023 at 19:01 | history | answered | user15436681 | CC BY-SA 4.0 |