30

Disclaimer: Some of the following is based on speculation, and may well turn out to be wrong. This does not invalidate the necessity to discuss the core problem, namely, how to stabilize a currency ("reputation") in times where AI-generated content is a dead-simple, readily available tool to drain value.

I recently came across this question. While I intended to write up an answer, I didn't find time immediately, and only returned to it later. In the meantime, the following had happened:

  • The OP placed a bounty on the question
  • Boatloads of ChatGPT-generated answers were posted
  • One of those answers (that wasn't an answer to the question) was awarded the bounty
  • All (presumably) ChatGPT-generated answers were deleted by moderators, and their respective user accounts temporarily suspended

This generates two issues:

  1. Users can now steal by having reputation attributed to their user account that should go to ChatGPT instead (in the rare case where it actually produces a correct answer)
  2. In the best case scenario where ChatGPT-generated contributions are properly identified and deleted, value is drained out of the system (point in case being that the bounty has apparently not been refunded)

Moving forward, how should we handle bounties awarded to ChatGPT-generated content?

My personal opinion here is fairly simple: Permanently delete any user account that can be associated with a ChatGPT-generated contribution, and roll back any and all effects it had (including refunding bounties). Is that too harsh?

11
  • 13
    "point in case being that the bounty has apparently not been refunded" - bounties are non-refundable by design Commented Dec 25, 2022 at 12:16
  • 4
    Remember that "bounty on SE" works more as an advertisement instead of a real bounty, thus this also extends to other cases, including 1) having no answers, 2) having answers but not enough for an automatic award, 3) manually awarded to a delete-worthy answer (i.e. NAA, or in this case: ChatGPT answer); bounty reps are basically wasted.
    – Andrew T.
    Commented Dec 25, 2022 at 12:49
  • 5
    I agree, but some of the mods don't even want to suspend users posting ChatGPT answers, which is unfortunate Commented Dec 25, 2022 at 14:48
  • 5
    Yes, it's too harsh. I could go on a diatribe about how ChatGPT generates better answers than probably half the user base, but suffice it to say that I think the outrage significantly exceeds the crime. Commented Dec 25, 2022 at 17:20
  • 24
    "Permanently delete any user account that can be associated with a ChatGPT-generated contribution [...] Is that too harsh?" What if the answer only looks like it's ChatGPT-generated when it's not? That's happened before. The good thing with suspensions is that the suspended user can respond to the moderator who suspended them, and if the suspension is in error it can be undone. Account deletion on the other hand can't be undone. So before deleting someone's account you have to be completely sure beyond any doubt that they broke the rules. Commented Dec 25, 2022 at 17:28
  • 8
    value is drained out of the system (point in case being that the bounty has apparently not been refunded) - this is only true if you consider reputation points to have value. That way lies madness. I can create reputation points from thin air just by clicking buttons.
    – kaya3
    Commented Dec 25, 2022 at 21:45
  • 1
    I concur with @kaya3; rep has no value except for getting answers to your questions faster and being able to curate this site. So "unfairly" losing or gaining it is... really not a huge deal. And in this instance, the person who lost rep was the same person who chose an obviously wrong answer, so... no harm done.
    – Ian Kemp
    Commented Dec 25, 2022 at 23:28
  • @IanKemp It goes further: No answer is to be marked as correct as it maybe evades the question and the ChatGPT answers which instantly came first when the question was answered managed to garner at least 1 upvotes from suckers who think lots of paragraphs with code must be rewarded as hard work. Next: ChatGPT scammers get to put others to work using the stolen points ;) The points needs to be re-awarded, not lost.
    – oxygen
    Commented Dec 26, 2022 at 14:44
  • @oxygen A bountied question has never had bounty refunded before, even if the accepted answer was deleted. Having the accepted answer deleted because it's generated by ChatGPT is merely a variant on this and should not change this behaviour. The only guarantee with a bounty is that you lose some rep, not that something good happens.
    – Ian Kemp
    Commented Dec 26, 2022 at 17:44
  • @IanKemp Many things have not been done before. Re-awarding (not refunding) the bounty is common sense.
    – oxygen
    Commented Dec 26, 2022 at 17:59
  • 2
    @IanKemp Minor nitpick: Bounties which are canceled by moderators prior to being awarded are refunded to the user. Bounties which have been awarded are not refunded when the answer is deleted. When an answer is deleted, the answerer will keep the bounty if the answer qualifies to keep the reputation from upvotes and bounties (i.e. if it's > 60 days old when deleted and has score >= 3; no ChatGPT posts currently qualify, as all < 60 days). Removing the bounty rep if the answer so qualifies requires a CM escalation, which we would almost certainly do, as we currently do for plagiarized answers.
    – Makyen Mod
    Commented Dec 26, 2022 at 19:06

3 Answers 3

18

I don't think the combination of ChatGPT and bounties is actually a real problem here. They're currently kind of a useful honeypot for ChatGPT users, so most of the offending answers are identified and deleted pretty quickly. Occasionally one gets awarded a bounty (or half a bounty, automatically), but again that goes away on deletion.

I suspect the long term effect will be to make using bounties less attractive since it'll mainly be a way to pay imaginary internet points in exchange for reading 5 rapidly deleted ChatGPT answers. As someone who's generally sceptical of the value bounties bring to the site I don't think that's a big loss.

In summary: mass ChatGPT answers is probably a bad thing and needs moderation. Having them focused on bounties where they're easy to spot may be a good thing. Don't really see why we should treat ChatGPT answers on bounties differently to ChatGPT answers to other questions.

7
  • 5
    People with enough points for bounties may have acquired those points by contributing valuable material. When they actually want something back from the community by giving those points away as a prize they are dissapointed and then stop contributing. You should really have a positive attitude not force things into decay because you can’t see the value. Funnily enough ChatGPT violators know the value (unlike you) and will use those points to get people to work for them undeservingly - that’s the real tragedy.
    – oxygen
    Commented Dec 26, 2022 at 1:45
  • 6
    There's an elite squad of bounty hunter hunters that are looking for people who are using Chat GPT on bountied questions. If reputation is "imaginary internet points" then I don't know what to call their reward; it's less tangible. Their contributions are appreciated even if their names are obscure. Yet still they flag.
    – Laurel
    Commented Dec 26, 2022 at 3:05
  • 1
    "When they actually want something back from the community by giving those points away as a prize they are dissapointed and then stop contributing." Flushing out ChatGPT users is a valuable contribution to the site. Commented Dec 27, 2022 at 4:29
  • 6
    When you have a niche question with 0 answers the bounty system is one way of attracting potential answers. not flawless, but I do see the value.
    – JoSSte
    Commented Dec 27, 2022 at 8:35
  • 3
    @JoSSte There's definitely cases when bounties are useful (and that's the main one). I don't currently think that the use of ChatGPT and bounties is causing specific unsolvable problems though (beyond what ChatGPT alone is causing)
    – DavidW
    Commented Dec 27, 2022 at 8:56
  • So basically, your point is that the bounty will be refunded, even though it seemingly wasn't the case in OP's example?
    – Egor Hans
    Commented Feb 24, 2023 at 18:00
  • 1
    @EgorHans I hadn't really thought about refunded bounties. Still not sure if it's a real issue - a bounty doesn't guarantee a good answer so I don't really see why awarding a bounty to a ChatGPT answer earns you a refund. The only real issue is where a ChatGPT answer earns half a bounty due to votes preventing that happening to a real answer.
    – DavidW
    Commented Feb 24, 2023 at 19:24
7

Moving forward, how should we handle bounties awarded to ChatGPT-generated content?

This is a rather small edge case, but the approach once identified (that's probably the difficult part) is to follow the same protocol as all other violations. Suspend the account related to the ChatGPT-generated content in a manner consistent with other suspensions for similar actions, and move on.

The bounty itself was sent into the wind. Either an answer was given that would suffice and the bounty creator realizes that, or a situation arises where nothing is really worthwhile, and the system awards the reputation. In both these scenarios, the suspension works.

While a user might generate a little reputation from posting ChatGPT content in the case of bounties, the suspension to the account will far outrun their ability to actually gain anything of significance.

2
  • 1
    Once identified, the ChatGPT post will be deleted. If a bounty has been awarded to an answer, the answerer will loose the reputation from the bounty when the answer is deleted, unless the post qualifies to keep reputation (>60 days old and score >= 3). Thus, while they might have that bounty reputation for a while, it will go away (a CM escalation would be raised if manually removing it was required). It's possible the bounty would give them enough rep to get the association bonus. Once that's awarded, they get to keep the association bonus, as there's no methodology available for removing it.
    – Makyen Mod
    Commented Dec 26, 2022 at 9:19
  • 1
    Not all answers are born equal as ChatGPT answers come first and actually get upvotes. Good answers take time to show up and many times don’t get any attention anymore, so less upvotes. The people offering bounties forget or don’t want to award them out of some kind of spite if they’re not given fully copy pasteable code. The best thing to do is to re-award as the ChatGPT bounty hunters are exploiting the system, not following its rules.
    – oxygen
    Commented Dec 26, 2022 at 14:29
-16

Ok I don't really belong in this discussion as I am always on the side of the helped but nonetheless I just wanted to suggest that any AI-generated answers not be deleted altogether but just moved to a separate, still-accessible page of "Suspected AI-generated content" for consultation. This way, in case they DO contain something useful, which is going to happen more and more as it quickly improves, the poster is still helped and no time is wasted. After all, that is the main and, arguably, sole purpose of the website as it is the service it provides and that which generates visits and, therefore, revenue. More importantly there is absolute NO HARM WHATSOEVER in doing this, nor does it cost any resources.

A just compensation mechanism could then be thought of for the people who actually took the time from their busy schedules to give meaningful answers based on their hard-earned knowledge. Maybe any rewards could be split among all genuine contributors or anyone who contributed gets an increased reward for their next accepted answers. I'm sure you guys could think of something fair and fitting that could not be abused.

The inescapable reality is that there will be a drastically decreasing need for human help in the coming years not just here but everywhere. This is unavoidable and should NOT be seen as something bad at all. It's just technology replacing manual labor again, this time at a much grander scale. This is a great thing, an incredible thing, and needs to be accepted and used for good, certainly not resisted in any way.

I must say that I would find it very funny if Stack Overflow, one of the world's greatest bastions of programming, machine learning and all things computer-related, were to treat arguably the greatest fruit of its subject matter with contempt and prejudice instead of teaching the world by example as how to incorporate AI into existing solutions without there being any adverse effects for the humans that already participate in them.

7
  • 4
    AI answers are more often that not inaccurate or even outright wrong...
    – Cerbrus
    Commented Dec 27, 2022 at 18:34
  • Leave that for the user to decide. A chance of 1% is infinitely better than nothing. When you're desperate, anything helps. Furthermore, incorrect suggestions may serve to lead one to the right one. Lastly, they are going to improve exponentially fast as public interest in it is soaring and so will the amount of money involved. Also, I just successfully used ChatGPT to solve my beginner-level problems. Commented Dec 27, 2022 at 18:41
  • 6
    A chance of 1 percent is worse than nothing. It wastes everyone's time by leading people down the garden path with no resolution forthcoming. That said, I don't agree that the AI answers are wrong most of the time; quite to the contrary, I've found that ChatGPT can provide excellent results if properly motivated. Commented Dec 27, 2022 at 18:51
  • It certainly would be if people were forced to go through each and every answer but given that all that would be accomplished is that they would be given the opportunity to do so should they choose to, I would say that it most definitely is not worse than nothing for someone who has absolutely no candidate solutions on the table Commented Dec 27, 2022 at 20:36
  • 6
    I feel like this completely ignores the cost of effort needed to assess whether ChatGPT answers are correct. This entire meta-Q&A exists because such answers were found to be dangerously misleading, requiring significant time investment of expert knowledge to identify bad and potentially harmful advice. "there is absolute NO HARM WHATSOEVER" is just blatantly false if experts are tied up rating the content and non-experts cannot rely on the content. Whatever you think AI is capable of, ChatGPT is not that magic genie. Commented Dec 27, 2022 at 20:45
  • 3
    FWIW, this does not seem to be related to handling bounties at all. Temporary policy: ChatGPT is banned may be more appropriate to make such a suggestion, not that I would expect it to find more agreement there. Commented Dec 27, 2022 at 20:57
  • Thank you so much! I posted it there also. I don't really care about agreement to be honest, I posted it because I think it's important. If the people who matter at least take it into consideration, that's enough. Thanks Commented Dec 27, 2022 at 21:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .