Skip to main content
style
Source Link
mirekphd
  • 6.6k
  • 6
  • 9

I've seen this before in a similar context where automation was easier so it started earlier: auto-generated ("auto-ML") solutions (ML models predictions on the "public" data set) on Kaggle leaderboards. Auto ML models for easy (tabular) datasets ranked higher than most of my colleagues as early as 5-6 years ago, see e.g. Porto Seguro risk modeling competition, where H2O Driverless AI ranked top 28%.

Years have passed and insurance companies are still hiring mathematicians to do the job that was supposed to be automated long ago. So I'd be more open and let AI compete here to avoid a new form of discrimination.

Does AI pose new, unique problems? Yes it does. The main problem I see here is that SO users don't currently have the ability to withdraw an upvote after they notice the answer was a "trojan horse", only superficially right, but was hiding deeper problems. Maybe add a new flag "AI suspicion" that when raised enough times would release the answer lock and let users withdraw their upvote and even downvote.

Maybe users should have a twin account dedicated to AI-generated answers (and clearly marked as such), just like Kaggle lets you submit solutions under several nicknames (see e.g. human/team and AI-gen submissions posted by Navdeep Gill as himself or as H2O Driverless AI) ?

Due to the "trojan horse" risk it may be also necessary to withdraw / hide AI-gen answers from novice users (until certain reputation threshold [per tag]), to prevent them from accepting blindly answers that look right to the inexperienced programmers (likely suffering from the Dunning-Kruger effect, and hence disregarding deeper issues like edge cases, performance or scaling).

I've seen this before in a similar context where automation was easier so it started earlier: auto-generated ("auto-ML") solutions (ML models predictions on the "public" data set) on Kaggle leaderboards. Auto ML models for easy (tabular) datasets ranked higher than most of my colleagues as early as 5-6 years ago, see e.g. Porto Seguro risk modeling competition, where H2O Driverless AI ranked top 28%.

Years have passed and insurance companies are still hiring mathematicians to do the job that was supposed to be automated long ago. So I'd be more open and let AI compete here to avoid a new form of discrimination.

Does AI pose new, unique problems? Yes it does. The main problem I see here is that SO users don't currently have the ability to withdraw an upvote after they notice the answer was a "trojan horse", only superficially right, but was hiding deeper problems. Maybe add a new flag "AI suspicion" that when raised enough times would release the answer lock and let users withdraw their upvote and even downvote.

Maybe users should have a twin account dedicated to AI-generated answers (and clearly marked as such), just like Kaggle lets you submit solutions under several nicknames (see e.g. human/team and AI-gen submissions posted by Navdeep Gill as himself or as H2O Driverless AI) ?

I've seen this before in a similar context where automation was easier so it started earlier: auto-generated ("auto-ML") solutions (ML models predictions on the "public" data set) on Kaggle leaderboards. Auto ML models for easy (tabular) datasets ranked higher than most of my colleagues as early as 5-6 years ago, see e.g. Porto Seguro risk modeling competition, where H2O Driverless AI ranked top 28%.

Years have passed and insurance companies are still hiring mathematicians to do the job that was supposed to be automated long ago. So I'd be more open and let AI compete here to avoid a new form of discrimination.

Does AI pose new, unique problems? Yes it does. The main problem I see here is that SO users don't currently have the ability to withdraw an upvote after they notice the answer was a "trojan horse", only superficially right, but was hiding deeper problems. Maybe add a new flag "AI suspicion" that when raised enough times would release the answer lock and let users withdraw their upvote and even downvote.

Maybe users should have a twin account dedicated to AI-generated answers (and clearly marked as such), just like Kaggle lets you submit solutions under several nicknames (see e.g. human/team and AI-gen submissions posted by Navdeep Gill as himself or as H2O Driverless AI) ?

Due to the "trojan horse" risk it may be also necessary to withdraw / hide AI-gen answers from novice users (until certain reputation threshold [per tag]), to prevent them from accepting blindly answers that look right to the inexperienced programmers (likely suffering from the Dunning-Kruger effect, and hence disregarding deeper issues like edge cases, performance or scaling).

Source Link
mirekphd
  • 6.6k
  • 6
  • 9

I've seen this before in a similar context where automation was easier so it started earlier: auto-generated ("auto-ML") solutions (ML models predictions on the "public" data set) on Kaggle leaderboards. Auto ML models for easy (tabular) datasets ranked higher than most of my colleagues as early as 5-6 years ago, see e.g. Porto Seguro risk modeling competition, where H2O Driverless AI ranked top 28%.

Years have passed and insurance companies are still hiring mathematicians to do the job that was supposed to be automated long ago. So I'd be more open and let AI compete here to avoid a new form of discrimination.

Does AI pose new, unique problems? Yes it does. The main problem I see here is that SO users don't currently have the ability to withdraw an upvote after they notice the answer was a "trojan horse", only superficially right, but was hiding deeper problems. Maybe add a new flag "AI suspicion" that when raised enough times would release the answer lock and let users withdraw their upvote and even downvote.

Maybe users should have a twin account dedicated to AI-generated answers (and clearly marked as such), just like Kaggle lets you submit solutions under several nicknames (see e.g. human/team and AI-gen submissions posted by Navdeep Gill as himself or as H2O Driverless AI) ?