Let's clearly state the problem. The problem is that there are
answers that are (a) highly up-voted (and are possibly also accepted), (b) have not been obsoleted, (c) have several, highly up-voted comments that challenge the answer, and (d) these comments have neither been addressed nor has the question been revised
(I don't think OP's presence or current activity is relevant. I'm also not entirely sure if it being obsolete or not is important.)
This is an unfair situation. It is not fair to:
- People looking for correct answers but who get disinformation.
- Writers of correct answers who put effort into compiling excellent and well researched answers but remain buried.
- The answerer who benefits from naive upvotes.
We have some options here.
Option: Do the status-quo "right thing" & wait it out.
The currently accepted right things to do are:
- Downvote.
- Write a clear and strongly worded critical comment that helps others know to downvote.
- Write a clear demonstration in another answer that the information is incorrect.
I think this works in the long-run. However, this is dissatisfying because it can take years for the "right" answer to climb above the mediocre me-too answers, and it will not "unpin" a bad but pinned accepted answer.
Option: Bring it to Meta
Indirect attention usually has the right effect, as it did in the case of your recent issue with comments on a poor answer. However, you saw the effect when you tried to bring more direct attention to the subject matter - it wasn't well received on meta.
We could have a meta process to address these kinds of problems. We have a process now to burninate bad tags. We could create a similar process to "delete" or "down-mod" bad answers.
However, I think we're wary and skeptical of messages saying "this answer is wrong, please review, comment, and if you agree with me, downvote it." It kinda looks bad. We question your motives, especially if you have a stake in the matter (like a competing answer). And let's face it, we don't think that disinterested people are going to take up the cause of "someone is wrong on the Internet" - but maybe they will?
It's also risky. You could unfairly rile up the mob with pitchforks, or you could (also unfairly) look like you're trying to be a mob instigator.
If this situation is really as rare as some claim, perhaps that lowers some risks, but then maybe we'll be more likely to have false positives.
But if it's more common, the risks of too many heated arguments are greater too.
How do we know who to trust? Someone with a gold tag badge? Someone with more than 100k rep? Someone with a good story, song, and dance? Someone with a link to documentation and a few million-dollar words and phrases?
The upside might be that with fairly unanimous consensus on meta, we could just outright delete otherwise pinned and top-sorted bad answers - and this could be a good thing.
However, by making an issue on meta, we're probably going to get lots of time spent on answers and comments on answers, and disputes amongst the reviewers on potentially subjective gray areas. There may be some positive effect. But it may not be worth the potential downside.
Regardless, I'd like to present a third option:
Option: Strengthen the signal over the noise at the post itself.
I have long argued that we need a hotness sort that uses vote recency to sort answers.
Here's my user description from meta stackexchange chat:
Why we need a hotness sort: What's a good answer is subjective, but a strong objective signal is a new answer slowly rising among crufty 8 year old answers.
This suggestion is different from the Reddit's "Best" sort, which does not consider the timing of votes (- fine for Reddit, since they lock voting on 6 month old posts - but that doesn't work for us.)
A time-based hotness sort would
- use pre-existing mechanisms of voting and comments which can direct current voting.
- emphasize current signal over the noise of time by using newer votes to rank.
- "down-mod" bad answers by disregarding or deemphasizing old upvotes and emphasizing new downvotes.
We could make it a non-default sort option to start with - but for the benefit of less savvy answer seekers, I'd suggest we target making it the default.
The only downside that I can think of is that it may be difficult to implement and explain to users.
I would suggest the implementation details to group votes into quantiles by age, and sort by the most recent quantiles first (less recent would just be tiebreakers).
This is still not as fast or immediately satisfying as a meta intervention, but it would be an improvement over the current status-quo.
On a related note, we could also "unpin" accepted answers after some period of time. Note that self-accepts are already unpinned - so this is not suggesting actually "unaccepting" the accepted answer - just unpinning it.
New accepts on old questions could restart the clock.
Conclusion
The question was:
Is it appropriate to invite scrutiny of a highly up-voted, disputed answer?
I think it's appropriate for some definitions of appropriate, but I think and the consensus seems to be that it's too risky and too likely to devolve into rather unproductive arguments.
The current system is "working" (for some definitions of "working"). And I think it would "work" better with an improved sort and unpinning old accepteds, which I propose above.