Timeline for Thoughts on the paper "Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow"?
Current License: CC BY-SA 4.0
31 events
when toggle format | what | by | license | comment | |
---|---|---|---|---|---|
Jul 20, 2023 at 22:10 | comment | added | NoDataDumpNoContribution | @KevinB I wanted to argue that as long as it knows most of what people want to know it's useful, but who am I to argue. Going off site is perfectly fine. | |
Jul 20, 2023 at 22:02 | comment | added | Kevin B | @NoDataDumpNoContribution No, it doesn't at all. If it can't be trusted to know what it doesn't know, it can't be trusted to present solutions to people who can't critique them without oversight. If users want that, they can go to an LLM themselves off site. | |
Jul 20, 2023 at 21:42 | comment | added | NoDataDumpNoContribution | @KevinB " is obviously far worse than" depends crucially on the actual success rate of ChatGPT. There are other scientific papers about that and Franck Dernoncourt tried to ask that a few times on SE but was mostly downvoted. If beginners are better off with AI or with SO, I really don't know. My gut feeling would be to use as many different sources as possible including GenAI. Just try for yourself and learn from the experience. | |
Jul 20, 2023 at 18:27 | comment | added | user4581301 | Also remember that If all you see is the stuff SO users asked about, you're almost exclusively seeing the fail cases, the ones where the robot crapped out something wrong if not hilariously wrong. The SO-centric view is going to be that robots are complete garbage. Just like all professors are morons teaching C++ like it's C, Java, or Fortran. | |
Jul 20, 2023 at 15:54 | comment | added | Cris Luengo | @Gimby I was already rejecting new stuff in 2012. :p | |
Jul 20, 2023 at 14:31 | comment | added | Kevin B | @PasserBy "For many simple problems, there's nothing wrong with ChatGPT's answers." The emphasis is the problem. Given what GPT is and how it generates responses, the responses can only be trusted to look correct. That's all it is designed for. Tailoring it's usage toward users who, by definition, are unable to tell good advice from bad for the subject matter is obviously far worse than any outcome that could come from publicly asking the question and getting answers from other people. Regardless of how many cherrypicked successes are brought forward. | |
Jul 20, 2023 at 12:05 | comment | added | Gimby | @PasserBy that, and there are also a fair few people who are not just skeptical but outright reject anything that deviates from how things worked back in 2012 :) | |
Jul 20, 2023 at 12:02 | comment | added | Passer By | @zcoop98 I think there's also some significant background differences at play. I think a lot of the "ChatGPT is useless" crowd are trained to be skeptical towards literally everything, which is a pretty good attitude for an engineer or scientist. It surprises them to no end that that's not the default mode of behaviour for most of the population. | |
Jul 20, 2023 at 11:56 | comment | added | Passer By | @KevinB For many simple problems, there's nothing wrong with ChatGPT's answers. I'd side more towards what user4581301 said, that ChatGPT just fails on harder problems. | |
Jul 20, 2023 at 11:21 | comment | added | Gimby | Makes sense from the perspective of Reddit, it doesn't have the same goals as Stack Overflow so people there are more inclined to want schooling to be a thing - which is fine. That is a common misconception of Stack Overflow, that it is just like any other site. And therefore hostile and toxic because things which are normal everywhere else, such as discussion and opinions, are rejected here. I also very much agree with that last comment - ChatGPT is not competition or a replacement. It is a means to reduce junk input on Stack Overflow if people use it to outsource their work and homework. | |
Jul 19, 2023 at 20:19 | comment | added | Kevin B | or the ability to find said manual (curse you flutter results on google) | |
Jul 19, 2023 at 20:14 | comment | added | user4581301 | And I surmise that the lack of good documentation on whatever I'm struggling with is exactly why I'm struggling with it. All of us who are primarily on the answering side of SO learned long ago that the key to success is RTFM. | |
Jul 19, 2023 at 20:12 | comment | added | Kevin B | I'd suspect it's more a case of... the people looking for said easy problems aren't able to recognize the inaccuracies or flaws in the solutions provided, rather than those easy solutions actually being accurate. | |
Jul 19, 2023 at 20:11 | comment | added | user4581301 | @zcoop98 My first blush, knee jerk assumption with zero statistical backing is it's the different sorts of questions were asking. The beginner stuff its generally covered to death here on SO and all over the Internet, so the model can find dozens of examples of the problem in its data set and compose a decent answer from them. The kind of stuff the experienced programmer is looking to solve is not so well covered. Finding any documentation on what I'm typically having trouble with is a gift. Without good examples in its dataset all the model has to fall back on is dumb luck and outright lies. | |
Jul 19, 2023 at 19:11 | comment | added | zcoop98 | I just... I don't know if this is strictly relevant or not, but... I find this dichotomy between beginners and many SO-regulars regarding perceived utility of LLMs absolutely fascinating. I have seen, with my own eyes, people (often beginners) successfully use and learn a whole lot with an LLM to bounce questions and queries off of, and yet the consistent consensus I see from folks here is essentially "LLMs are useless/ 99% wrong/ rarely helps". I just don't... why is that? Why is there such a drastic disparity in opinion? What's driving the us-vs-them-isms that are so common right now? | |
Jul 19, 2023 at 18:26 | comment | added | user4581301 | The answer to that can be brutally simple: Never be wrong. Dunning and Kruger offer a stunningly effective pathway to the art of never being wrong. | |
Jul 19, 2023 at 16:21 | comment | added | Augusto Vasques | The question is, how do you know it's time to learn along the way whether you don't know if you are wrong? | |
Jul 19, 2023 at 16:11 | comment | added | Augusto Vasques | @RobertHarvey, Learning along the way is reasonable once you are aware that you need to learn something new because you have reached your limit, just like taking one more step. However, it doesn't seem productive to live in uncertainty about whether or not you will need learn along the way with every line of generative code. | |
Jul 19, 2023 at 15:44 | comment | added | Kevin B | cgpt is the next iteration of the blind leading the blind. | |
Jul 19, 2023 at 15:40 | comment | added | ggorlen | As for ChatGPT, I'm glad they're finding it helpful, but in my experience it consistently fails on pretty much any non-trivial question, so I'm guessing their questions are straightforward, and the GPT is basically looking it up on its SO training data, which they could probably do themselves just as easily. Fine with me. None of these comments discuss the paper. | |
Jul 19, 2023 at 15:40 | comment | added | ggorlen | As for the main answer here, these comments are basically the same anywhere you look--youtube, hacker news, reddit, etc. These are people who I don't think understand the purpose of SO. We're here to curate a resource of every programming answer, not a help desk. It's against our COC to call people idiots, so either that's a fabrication or the hostile user was likely disciplined. I rarely see that sort of language being thrown out (last time I did was by a GPT spammer). | |
Jul 19, 2023 at 15:36 | comment | added | ggorlen | @KarlKnechtel Agreed--OP doesn't have a SO account as far as I can see, so there does seem to be an agenda here. As discussed in one of their previous posts, I wish they'd be a bit more transparent about why they're doing this on a site they (presumably) have no history or stake in whatsoever. | |
Jul 19, 2023 at 14:46 | comment | added | user4581301 | "I also don’t have to worry about being told I’m an idiot while trying to learn." But what if you ARE and idiot? That's important information to have, especially while learning. The first step in solving a problem is knowing you have a problem. | |
Jul 19, 2023 at 12:10 | comment | added | Robert Harvey | @AugustoVasques: Learning along the way is inevitable. Do you genuinely believe that competent folks already know everything they need to know to do their job, given constant technological change? | |
Jul 19, 2023 at 9:02 | comment | added | Karl Knechtel | @CodeCaster when I saw this answer, I immediately got the impression that the Q&A were a Trojan horse being used to plant more pro-ChatGPT propaganda on Meta. | |
Jul 19, 2023 at 7:34 | comment | added | Augusto Vasques | ...ChatGPT isn’t always correct but I actually learn along the way... perhaps, without deadlines, without a family to support, and by using other people's money, learning along the way could be an efficient methodology. | |
Jul 19, 2023 at 7:19 | comment | added | Clockwork | It sounds like some of them could benefit from actually using the search engine. The only time I could find myself in need of something like ChatGPT would be for an edge case for which I need a specific solution, in which case someone more knowledgeable would be more likely to provide help. But then again I was told I never do things the way the others do it. | |
Jul 19, 2023 at 5:58 | comment | added | user5349916 | Those don’t look like thoughts on the paper, rather like a mix of general and ChatGPT specific thoughts on SO. FWIW there are some comments actually addressing the paper in that thread. | |
Jul 19, 2023 at 5:57 | comment | added | user13267 | You added the magic words "interactions they perceive as negative at Stack Overflow and that they found ChatGPT friendlier" that attracts downvoters like flies | |
Jul 19, 2023 at 5:33 | comment | added | CodeCaster | Shooting the messenger much, lol? | |
Jul 19, 2023 at 3:03 | history | answered | Rebecca J. Stones | CC BY-SA 4.0 |