Skip to main content
Typo fix
Source Link
tripleee
  • 188.6k
  • 5
  • 69
  • 122

Speaking more specifically about StackoverflowStack Overflow here, but it may generalize.

The value of the site relies on the notion that user votes inherently reflect quality and accuracy of information (to whatever degree it is so).

At the same time, the reality of user behaviour is that we rely on the inherent quality and accuracy of information to inform our voting (again, to whatever degree). This is because in many cases, it is not possible, or just not practical to fully vet and validate the posts we see. After a modest mental investment failing to find any deficiencies in a post, it may feel justified to vote it up, putting at least some stock in the good-faith effort of the poster and their reputation.

An example of of the previous point that has happened to me multiple times in the past: I answer a question that is confidently within my area of knowledge, and I write an answer with a short code snippet. The answer receives a few up-votes. Later, someone comments on my answer or edits it to correct a syntax mistake that had been there the whole time. The code in it'sits original state would not even pass the compiler. Nonetheless, multiple users "validated" the answer before the mistake was caught.

The issue I see here is that an AI model's ability to produce content that looks like it is of the quality we expect from genuine human posts (thus attracting up-votes, assuming voter behaviour stays the same), is not necessarily the same as the model's ability to produce content that is actually of the quality we expect from genuine human posts.

Therefore, as the proportion of AI generated-generated content of the site increases, I'm guessing that voter behaviour will need to change (particularly, more effort will need to be expended), or else the value and meaning of votes will diminish.

Speaking more specifically about Stackoverflow here, but it may generalize.

The value of the site relies on the notion that user votes inherently reflect quality and accuracy of information (to whatever degree it is so).

At the same time, the reality of user behaviour is that we rely on the inherent quality and accuracy of information to inform our voting (again, to whatever degree). This is because in many cases, it is not possible, or just not practical to fully vet and validate the posts we see. After a modest mental investment failing to find any deficiencies in a post, it may feel justified to vote it up, putting at least some stock in the good-faith effort of the poster and their reputation.

An example of of the previous point that has happened to me multiple times in the past: I answer a question that is confidently within my area of knowledge, and I write an answer with a short code snippet. The answer receives a few up-votes. Later, someone comments on my answer or edits it to correct a syntax mistake that had been there the whole time. The code in it's original state would not even pass the compiler. Nonetheless, multiple users "validated" the answer before the mistake was caught.

The issue I see here is that an AI model's ability to produce content that looks like it is of the quality we expect from genuine human posts (thus attracting up-votes, assuming voter behaviour stays the same), is not necessarily the same as the model's ability to produce content that is actually of the quality we expect from genuine human posts.

Therefore, as the proportion of AI generated content of the site increases, I'm guessing that voter behaviour will need to change (particularly, more effort will need to be expended), or else the value and meaning of votes will diminish.

Speaking more specifically about Stack Overflow here, but it may generalize.

The value of the site relies on the notion that user votes inherently reflect quality and accuracy of information (to whatever degree it is so).

At the same time, the reality of user behaviour is that we rely on the inherent quality and accuracy of information to inform our voting (again, to whatever degree). This is because in many cases, it is not possible, or just not practical to fully vet and validate the posts we see. After a modest mental investment failing to find any deficiencies in a post, it may feel justified to vote it up, putting at least some stock in the good-faith effort of the poster and their reputation.

An example of of the previous point that has happened to me multiple times in the past: I answer a question that is confidently within my area of knowledge, and I write an answer with a short code snippet. The answer receives a few up-votes. Later, someone comments on my answer or edits it to correct a syntax mistake that had been there the whole time. The code in its original state would not even pass the compiler. Nonetheless, multiple users "validated" the answer before the mistake was caught.

The issue I see here is that an AI model's ability to produce content that looks like it is of the quality we expect from genuine human posts (thus attracting up-votes, assuming voter behaviour stays the same) is not necessarily the same as the model's ability to produce content that is actually of the quality we expect from genuine human posts.

Therefore, as the proportion of AI-generated content of the site increases, I'm guessing that voter behaviour will need to change (particularly, more effort will need to be expended), or else the value and meaning of votes will diminish.

Source Link

Speaking more specifically about Stackoverflow here, but it may generalize.

The value of the site relies on the notion that user votes inherently reflect quality and accuracy of information (to whatever degree it is so).

At the same time, the reality of user behaviour is that we rely on the inherent quality and accuracy of information to inform our voting (again, to whatever degree). This is because in many cases, it is not possible, or just not practical to fully vet and validate the posts we see. After a modest mental investment failing to find any deficiencies in a post, it may feel justified to vote it up, putting at least some stock in the good-faith effort of the poster and their reputation.

An example of of the previous point that has happened to me multiple times in the past: I answer a question that is confidently within my area of knowledge, and I write an answer with a short code snippet. The answer receives a few up-votes. Later, someone comments on my answer or edits it to correct a syntax mistake that had been there the whole time. The code in it's original state would not even pass the compiler. Nonetheless, multiple users "validated" the answer before the mistake was caught.

The issue I see here is that an AI model's ability to produce content that looks like it is of the quality we expect from genuine human posts (thus attracting up-votes, assuming voter behaviour stays the same), is not necessarily the same as the model's ability to produce content that is actually of the quality we expect from genuine human posts.

Therefore, as the proportion of AI generated content of the site increases, I'm guessing that voter behaviour will need to change (particularly, more effort will need to be expended), or else the value and meaning of votes will diminish.