-7

This is an attempt to invite a canonical answer to the question in the title.

Context:

Suggestion:

Asking the community to examine a disputed popular answer that meets the criteria described above should be acceptable, if done correctly.

Done correctly means:

  • Disclose your personal stance. Obviously, the only reason to even ask for scrutiny is a personal sense of something being amiss. That in itself cannot be a disqualifying factor.

  • Do not attempt to sway others or lobby for a particular action; providing supporting evidence is helpful, however.

  • Just ask for resolution in either direction - the community at large can only benefit:

    • If the answer turns out to be valid, it should receive more up-votes, and further comments should be added explaining why the dissenting comments are incorrect.

    • If the answer turns out to be invalid, down-votes (and up-votes on the dissenting comments) and/or additional explanatory comments can provide guidance to future readers.

    • Ideally, the scrutiny will cause the answer to be improved or fixed, or, if turns out there's nothing to salvage, to be deleted.

  • Needless to say, only users actually versed in the answer's specific subject-matter should get involved; no one should feel they have to do anything.

  • To reinforce the opening statement, a disputed answer should meet all the criteria above, so as to prevent ax-grinding and wasted effort.

    • If that requires a more formal definition of when inviting scrutiny is acceptable and how to ask for it, let's establish it here.

We all know about the meta effect, which is usually a byproduct of asking questions on meta.
This is an opportunity for employing it in a more transparent, straightforward way, in service to better answers by deliberate community consensus.

24
  • So if asking for scrutiny is allowed, would asking for attention on a low-viewed low-score canonicals that are good Q&As also be welcome?
    – Andrew Li
    Jul 6, 2017 at 2:23
  • 8
    When raising attention to a specific post, how can you be sure 'only users actually versed in the subject-matter get involve?' Once you've brought attention to it you can't guarantee that, which is most definitely a glaring issue with this process.
    – Taryn
    Jul 6, 2017 at 2:26
  • 1
    @bluefeet: It sounds like you have a concern about meta users behaving responsibly. Why is that?
    – mklement0
    Jul 6, 2017 at 2:29
  • 6
    @mklement0 Why isn't it a valid concern? I see people blindly voting based on a person's reputation or the majority's opinion all the time. Some don't even seem to known anything about the subject.
    – Andrew Li
    Jul 6, 2017 at 2:32
  • 8
    @mklement0 I have a concern about using Meta to pick on an answer when you could just comment, down vote, and walk away. Calling out specific posts is most likely going to bring on the Meta effect you mention in your post. I don't see how you are going to be sure only those with actual knowledge are involved.
    – Taryn
    Jul 6, 2017 at 2:32
  • 9
    @mklement0 I'm pretty sure those words didn't come out of my mouth. If that is how you interpret anything I've said, then you might want to take a walk away from your machine. You've been posting about this for hours and maybe you need to step away for a bit.
    – Taryn
    Jul 6, 2017 at 2:45
  • 2
    @bluefeet: My apologies: the comment about viewing users as unruly/clueless/malicious children was directed at AndrewLi's comment.
    – mklement0
    Jul 6, 2017 at 2:57
  • 4
    @mklement0 and comment upvotes should outweigh upvotes on an answer? If there truly was dissent that an answer was wrong, then a post should have more downvotes than upvotes. I personally don't feel asking on Meta for the community to weigh in on answers is the right way to go. It creates a witchhunt which is just wrong IMO.
    – Taryn
    Jul 6, 2017 at 3:00
  • 4
    @bluefeet: "If there truly was dissent that an answer was wrong, then a post should have more downvotes than upvotes." Not when the comment is several months or years late and the answer has already had hundreds of upvotes from people who were misled or didn't understand its implications. This answer had 100 upvotes before my answer came along. It has a major flaw that was pointed out in the comments, but because it appeared to be right by sheer coincidence, hardly anyone downvoted it. (1/2)
    – BoltClock
    Jul 6, 2017 at 3:19
  • 4
    @bluefeet: My answer has since outscored it more than threefold, but its own upvote count has more than doubled anyway because it remained the accepted answer for another three and a half years before the asker finally switched the accept mark to my answer. And still only 10 others have downvoted it so far. I'm happy my answer is accepted and pinned to the top, but for three and a half years, it wasn't, and it's any wonder it got pinned at all. (2/2)
    – BoltClock
    Jul 6, 2017 at 3:19
  • 2
    @BoltClock I get that. I've also done a ton of research on this exact issue. When I initially looked the overall percentage of answers like this is incredibly small. We've discussed internally many ways to deal with them and haven't found a solution for it. I personally do not agree with calling out specific answers on Meta.
    – Taryn
    Jul 6, 2017 at 3:26
  • 1
    @bluefeet: You must not like the [specific-question] tag either, then :P
    – BoltClock
    Jul 6, 2017 at 3:30
  • 7
    @bluefeet: Though I hope you understand that the percentage of such answers being incredibly small doesn't make it any less of a problem. Famous Question, Great Question and Great Answer are gold badges for a reason. Such Q&As command a ton of influence. I hope we'll be able to figure something out.
    – BoltClock
    Jul 6, 2017 at 3:35
  • 3
    @BoltClock I completely agree which is why I'm always open to hearing suggestions to fix a small problem but I don't think this is the solution to it.
    – Taryn
    Jul 6, 2017 at 3:36
  • 2
    @mklement0 I did a lot of querying and analysis of posts on SO. I'm saying that based on that, the overall number of posts which potentially could fall into a category like this is very small percentage in comparison to the total answers on the site. As I mentioned, we've discussed at length various ways to deal with these answers, I've even posted about some solutions on MSE 1, 2 but we've not come up with a solution that solves the problem without possibly creating more.
    – Taryn
    Jul 6, 2017 at 15:20

1 Answer 1

3

Let's clearly state the problem. The problem is that there are

answers that are (a) highly up-voted (and are possibly also accepted), (b) have not been obsoleted, (c) have several, highly up-voted comments that challenge the answer, and (d) these comments have neither been addressed nor has the question been revised

(I don't think OP's presence or current activity is relevant. I'm also not entirely sure if it being obsolete or not is important.)

This is an unfair situation. It is not fair to:

  • People looking for correct answers but who get disinformation.
  • Writers of correct answers who put effort into compiling excellent and well researched answers but remain buried.
  • The answerer who benefits from naive upvotes.

We have some options here.

Option: Do the status-quo "right thing" & wait it out.

The currently accepted right things to do are:

  • Downvote.
  • Write a clear and strongly worded critical comment that helps others know to downvote.
  • Write a clear demonstration in another answer that the information is incorrect.

I think this works in the long-run. However, this is dissatisfying because it can take years for the "right" answer to climb above the mediocre me-too answers, and it will not "unpin" a bad but pinned accepted answer.

Option: Bring it to Meta

Indirect attention usually has the right effect, as it did in the case of your recent issue with comments on a poor answer. However, you saw the effect when you tried to bring more direct attention to the subject matter - it wasn't well received on meta.

We could have a meta process to address these kinds of problems. We have a process now to burninate bad tags. We could create a similar process to "delete" or "down-mod" bad answers.

However, I think we're wary and skeptical of messages saying "this answer is wrong, please review, comment, and if you agree with me, downvote it." It kinda looks bad. We question your motives, especially if you have a stake in the matter (like a competing answer). And let's face it, we don't think that disinterested people are going to take up the cause of "someone is wrong on the Internet" - but maybe they will?

It's also risky. You could unfairly rile up the mob with pitchforks, or you could (also unfairly) look like you're trying to be a mob instigator.

If this situation is really as rare as some claim, perhaps that lowers some risks, but then maybe we'll be more likely to have false positives. But if it's more common, the risks of too many heated arguments are greater too.

How do we know who to trust? Someone with a gold tag badge? Someone with more than 100k rep? Someone with a good story, song, and dance? Someone with a link to documentation and a few million-dollar words and phrases?

The upside might be that with fairly unanimous consensus on meta, we could just outright delete otherwise pinned and top-sorted bad answers - and this could be a good thing.

However, by making an issue on meta, we're probably going to get lots of time spent on answers and comments on answers, and disputes amongst the reviewers on potentially subjective gray areas. There may be some positive effect. But it may not be worth the potential downside.

Regardless, I'd like to present a third option:

Option: Strengthen the signal over the noise at the post itself.

I have long argued that we need a hotness sort that uses vote recency to sort answers.

Here's my user description from meta stackexchange chat:

Why we need a hotness sort: What's a good answer is subjective, but a strong objective signal is a new answer slowly rising among crufty 8 year old answers.

This suggestion is different from the Reddit's "Best" sort, which does not consider the timing of votes (- fine for Reddit, since they lock voting on 6 month old posts - but that doesn't work for us.)

A time-based hotness sort would

  • use pre-existing mechanisms of voting and comments which can direct current voting.
  • emphasize current signal over the noise of time by using newer votes to rank.
  • "down-mod" bad answers by disregarding or deemphasizing old upvotes and emphasizing new downvotes.

We could make it a non-default sort option to start with - but for the benefit of less savvy answer seekers, I'd suggest we target making it the default.

The only downside that I can think of is that it may be difficult to implement and explain to users.

I would suggest the implementation details to group votes into quantiles by age, and sort by the most recent quantiles first (less recent would just be tiebreakers).

This is still not as fast or immediately satisfying as a meta intervention, but it would be an improvement over the current status-quo.

On a related note, we could also "unpin" accepted answers after some period of time. Note that self-accepts are already unpinned - so this is not suggesting actually "unaccepting" the accepted answer - just unpinning it.

New accepts on old questions could restart the clock.

Conclusion

The question was:

Is it appropriate to invite scrutiny of a highly up-voted, disputed answer?

I think it's appropriate for some definitions of appropriate, but I think and the consensus seems to be that it's too risky and too likely to devolve into rather unproductive arguments.

The current system is "working" (for some definitions of "working"). And I think it would "work" better with an improved sort and unpinning old accepteds, which I propose above.

8
  • 1
    There's another valid option for action - edit the answer. Include details from the more relevant comments. That makes for a better answer, fully fleshed. And is how the site was always supposed to work.
    – Oded
    Jul 6, 2017 at 8:33
  • 6
    @Oded: This only works if the comments don't debunk the answer outright. Otherwise you produce a self-contradicting answer, or at least an answer that's both right and wrong.
    – BoltClock
    Jul 6, 2017 at 11:48
  • @Oded: Another problematic aspect, aside from the on BoltClock mentions: If the OP is still active and you edit their answer, they may not take kindly to that, which is understandable. After all, if they're still active, they've made a decision to ignore the dissenting comments, and while we can speculate as to why and there are many explanations, disagreeing with the dissent is certainly one of them.
    – mklement0
    Jul 6, 2017 at 15:29
  • @AaronHall: Thank you for a well-reasoned, helpful answer. I like the idea of the "hotness sort", and I also like the idea of creting a process to "delete" or "down-mod" bad answers. I'll adress some minor points regarding the inviting-scrutiny-on-meta proposal below.
    – mklement0
    Jul 6, 2017 at 15:29
  • Re "I don't think the OP's presence is relevant" and "not sure if […] obsolete is important": In both cases it is an option to simply add an editor's note with a correction, which obviates the need for community involvement.
    – mklement0
    Jul 6, 2017 at 15:30
  • Re "If you have a stake in the matter": The sole reason I created my answer - many years later - was in an attempt to counteract the flawed accepted answer with a detailed rebuttal (more than can be done in a comment).
    – mklement0
    Jul 6, 2017 at 15:30
  • Re "disinterested": I'm very much interested, but not in a partisan way: I'm interested in fixing flawed answers that have wide exposure. The invitation to scrutiny is a way to resolve the disagreement one way or the other, and it is important to stress that aspect, even though a perceived problem is of necessity the starting point. I do get, however, that this is tricky business: I was trying to be extra careful in the wording of my original question that invited scrutiny of a specific answer, and I tried to be specific about the the criteria for when inviting scrutiny is acceptable here.
    – mklement0
    Jul 6, 2017 at 15:37
  • 1
    All that said, I like your proposals better, not least because of the reaction to my own attempts and it being seemingly impossible to convince others that there's no self-serving agenda.
    – mklement0
    Jul 6, 2017 at 15:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .