68

I hope the following example will clarify my question.

Searching profiling code linux on Google, the first result I got was this very popular SO question. Reading the most upvoted answer, I noticed that there was a basic error about Bayesian statistics on the heading of the last column of both tables: f=>x should be replaced by f=>x|o=2/2. Interestingly, browsing among answer's comments, I noticed that at least two more users were aware of the error (and one of them is the original author, I suppose). Hence, I tried, twice, to perform the edit process submitting this simple replacement, but I got 4 out of 5 negative reviews (reported at the bottom).

In order to avoid a misunderstanding, let me clarify that I don't want to contest the reviews: given the reviewers' experience and the strong majority, this is very likely the best solution for SO. Nor I insist that my fix has to be accepted. The point is that 1 out of 5 of them gave importance to the bare correctness of the contents of the answer.

To summarize the paradox: There is Google top-ranked and largely upvoted SO question, whose 23-times edited and most upvoted answer contains an obvious error. Several people know it. The error is still there.

As a SO reader (rather than contributor), I am concerned.

Reviews:

1) This edit does not make the post even a little bit easier to read, easier to find, more accurate or more accessible. Changes are either completely superfluous or actively harm readability.

2)This edit deviates from the original intent of the post. Even edits that must make drastic changes should strive to preserve the goals of the post's owner.

3) This edit was intended to address the author of the post and makes no sense as an edit. It should have been written as a comment or an answer.

4)This edit was intended to address the author of the post and makes no sense as an edit. It should have been written as a comment or an answer.

Ps: due to my current low reputation, I cannot post comments.

17
  • 2
    You can always write your own answer. That way people have two answers to choose from and eventually the correct answer will "win" by vote count. Commented Feb 23, 2019 at 7:39
  • 44
    @RobertLongson: Writing an additional answer to fix a error in another answer doesn't sound feasible (there are already 11 answers + 20 deleted answers on that post). Reviewers should not reject edits that fix a problem which is already accepted by the author in comments.
    – BDL
    Commented Feb 23, 2019 at 7:51
  • But the original author start their comment with Um, yeah, I guess. That doesn't sound to me they are 100% convinced it is an issue that needs fixing?
    – rene
    Commented Feb 23, 2019 at 7:58
  • 3
    @rene: He stated Um, yeah, I guess. It's just the second-to-last column normalized to add up to 1, which is P(o=2/2&&f >= x) divided by P(o=2/2), you can verify yourself. Incidentally, this is exactly P(f >= x|o=2/2).
    – JtTest
    Commented Feb 23, 2019 at 8:20
  • 44
    you can verify yourself you're hugely over-estimating my capabilities.
    – rene
    Commented Feb 23, 2019 at 8:29
  • 1
    That post is CW, anyone with >100 reputation can edit it. There are more experts with rep >100 than experts with rep>2000, so ask somebody (via meta?) to edit it.
    – user202729
    Commented Feb 23, 2019 at 10:49
  • Mike Dunlavey is usually very responsive. You could try using comments first. Commented Feb 23, 2019 at 14:41
  • Incidentally, for me that answer has been the single most useful answer on the entire Stack Overflow site, leading to a 20-40 times speedup in the real world in three very different applications for where it mattered (direct effect on how long users had to wait, in one case reduced from minutes to seconds) - as usual the time was taken up in unexpected places. Controversial as it may be, the method works (and efficient at that). Commented Feb 23, 2019 at 14:50
  • 7
    @PeterMortensen: usefulness is not the subject. I am not a programmer and I found that answer very interesting, too. Is it correct under a computer-science point of view? I guess yes because I trust SO community's work. But, what happens if, for example, I used that tables to explain things or teach the same matter to someone else?
    – JtTest
    Commented Feb 24, 2019 at 6:01
  • 4
    @PeterMortensen except that new/lowrep users can not even leave a comment. So “you could try using comments first” is not a useful advice. That was one of my first Stackoverflow experiences too. You can not leave a comment and the edit got rejected.
    – Holger
    Commented Feb 25, 2019 at 9:08
  • 1
    @rene You're the best programming flower I've ever met, if it makes you feel any better.
    – TylerH
    Commented Feb 25, 2019 at 20:30
  • 4
    I just edited my answer to agree (I think) with your suggestion. I don't think it's a big point, because o=2/2 is pretty much stated throughout and in the leading text. But if it makes things any more clear, it's no problem. Commented Feb 26, 2019 at 1:14
  • 2
    Simply put, this isn't wikipedia. Posts belong to their authors... kind of. The preferred way of correcting factual/content errors is to poke the author and have them fix it themselves. And down vote blatantly incorrect answers. This particular answer was marked as community wiki though, so the author has given up ownership of it. Still, you'll need 2000 rep to be trusted to edit it.
    – Lundin
    Commented Feb 26, 2019 at 7:32
  • 1
    @rene "you're hugely over-estimating my capabilities" That's really the gaping hole in the review system for suggested edits; some reviewers want only major edits that solve world hunger and are worthy of disrupting their front page workflow, some reviewers are terrified by those exact edits and will reject on sight, some reviewers will hit skip like they're supposed to, and some reviewers will accept pretty much anything, and you'll never know who you're getting.
    – jrh
    Commented Feb 26, 2019 at 14:41
  • 1
    I wonder if it would be practical to have a checkbox when suggesting an edit saying "have a gold tag badge holder review this", and then the edit would only be shown to users that are familiar with the tag.
    – jrh
    Commented Feb 27, 2019 at 13:07

2 Answers 2

64

How much does correctness matter in SO answers?

Tremendously so. In fact, it is nearly paramount. The only thing that arguably matters more than correctness is helpfulness (which includes things like being relevant to the question, offering an explanation, anticipating and addressing subsequent problems, etc.), but even those two go hand-in-hand, as being wrong is rarely helpful.

In fact, the entire Stack Exchange system is designed around assessing the correctness and helpfulness of posts. That's why we have the vote system: so posts can be rated based on their usefulness and correctness by experts, and the best posts float to the top. It's also, as you correctly reasoned, why we have an editing system. It's even why we have comments: to allow people a mechanism to ask for clarification on and suggest improvements to posts, thus making them more correct and more helpful.

So, why was my edit rejected?

Honestly? Because the system does not trust you enough.

It is precisely because correctness is so important that we have to be cautious about the possibility of rogue edits that would introduce wrong information or otherwise reduce the helpfulness of an answer. The only thing worse than a wrong answer is an answer that used to be right being made wrong.

Notice that I used the word "system" up there. The system trusts users based on reputation. If you've contributed a significant quantity of correct, helpful information to this website (as scored by your peers), then you've earned enough reputation to unlock certain privileges, including the ability to directly edit posts.

Until the system has become convinced of your trustworthiness, you can only suggest edits, which have to be reviewed by other community members. That's what you did here, and your suggested edit was rejected during this review process.

Okay, so why was my edit rejected?

Because reviewers of suggested edits rightfully err on the side of caution, for the reasons enumerated above.

The suggested edits system works really well for enhancements to a post's formatting, grammar/spelling, and other things that anyone can accurately judge. It works a lot less well for enhancements to the technical nitty-gritty because that's a lot harder to judge.

For edits that tamper with the substance of a post, you can increase your odds of getting it approved by writing a clear, well-justified edit comment. You kinda dropped the ball on that on your first attempt; actually, it looks like you might have accidentally hit submit before you were finished typing. On your second attempt, you did a great job, writing:

Corrected last columns' headers of both tables (see comments of unutbu and Mike Dunlavey May 6 '10 at 16:29)

That describes both what change you made, and why you made that change.

Right, so, umm, why was my edit rejected?

Because the reviewers weren't sure, so they erred on the side of rejecting an edit that might have been doing damage to the correctness and helpfulness of the answer.

Honestly, I'm not sure that your edit is correct, either. Reading the comments you pointed to in the edit comment doesn't help much:

@Mike: Thank you for the example of Bayesian reasoning. In the last column of your tables, is P(f >= x) correct or do you really mean P(f >= x | o=2/2)?
unutbu May 6 '10 at 16:29

@~unutbu: Um, yeah, I guess. It's just the second-to-last column normalized to add up to 1.
Mike Dunlavey May 6 '10 at 18:33

Mike (the author) doesn't even appear to be sure. How could the reviewers possibly be sure, unless they were more knowledgeable than Mike about Bayesian statistics?

And so, while this isn't a perfect system, I'd rather have the reviewers err on the side of rejecting potentially harmful (i.e., incorrect) edits than have them approve them. Wouldn't you?

13
  • 3
    "cautious about the possibility of rogue edits" eh? I think the "system" is just paranoid of modifying anything that looks too much which isn't a detriment to the post, but it's a-ok in modifying too little which actually causes harm. We've discussed this in the past. That's why I prefer experts to review edits, even if we take more times to reach a conclusion. For me, having accurate reviews that would make the post better and never worse triumph anything else.
    – Braiam
    Commented Feb 23, 2019 at 13:15
  • 53
    I wonder why people reject such edits. Its not like you have to hit either reject or approve, you can skip. If you aren't sure, skip it.
    – Polygnome
    Commented Feb 23, 2019 at 14:49
  • Thank you for your thorough explanation. As I wrote, I never insist to make my edit accepted. But, I know that there is an error of basic statistics. So the reviewers should, too. Thus I am sure, since they are surely expert (otherwise they would have skip, as @Polygnome pointed out) that they will fix the problem.
    – JtTest
    Commented Feb 24, 2019 at 6:20
  • 19
    Wow, one of those reviewers "has approved 3187 edit suggestions and rejected 25192 edit suggestions". That cannot be right... This is someone mashing the reject button all day long? I think the problem is that the audits all require one to reject. There should be audits requiring accept. Commented Feb 24, 2019 at 6:49
  • 9
    @CrisLuengo I remember an FR about that. I suggested to make a post worse (unformatted, some typo on a common word, etc.) and use that as the original and put the actual version as suggested edit.
    – Braiam
    Commented Feb 24, 2019 at 18:13
  • 25
    "Because the reviewers weren't sure, so they erred on the side of rejecting" - I'm afraid not. They most certainly rejected because they actually think an edit that deviates from the author's original answer should not be approved, even if it corrects wrong information. And I've seen more obvious examples of this.
    – Marc.2377
    Commented Feb 24, 2019 at 18:49
  • 4
    @CrisLuengo: My own stats never got that imbalanced, but for quite a while I had a habit of skipping any suggestion I knew would get enough approves anyway, and saving my daily reviews for the edits that were more likely to be wrongly approved (or, occasionally, wrongly rejected). Commented Feb 25, 2019 at 2:54
  • 7
    @Polygnome you don’t get that gold badge for skipping reviews…
    – Holger
    Commented Feb 25, 2019 at 9:12
  • 5
    "The suggested edits system works really well for enhancements to a post's formatting" FWIW I gave up on the suggested edits system, I don't think it's good for anything at all and I personally found it to be almost a complete waste of time these days. Too many contradictory reviewer moderating strategies. Maybe things will settle out again later.
    – jrh
    Commented Feb 25, 2019 at 14:08
  • 2
    And here's what can happen if you don't. history.stackexchange.com/review/suggested-edits/38874
    – Joshua
    Commented Feb 26, 2019 at 0:05
  • 4
    I just re-inspected my answer, and edited it to agree with the suggestion. I don't think it's a very important point, but the suggestion does no harm. Commented Feb 26, 2019 at 1:16
  • 1
    SO really could use experts for review instead of experts in SO, or a way to "vote it up" to proven experts.
    – user241244
    Commented Feb 26, 2019 at 2:55
  • 3
    "Because the reviewers weren't sure, so they erred on the side of rejecting an edit": if the reviewers were not sure, they should skip the review and leave it to people who are sure.
    – JeremyP
    Commented Feb 26, 2019 at 14:40
-5

I will try to give a different perspective. Correctness is subjective. As far as I know, any forum or group discussion is subjective. Your post here, to me, is a part of that subjectivity. Whatever you call it, the world around us has different definitions of what correct is, and the only peaceful, workable and futuristic tool humanity has produced is Democracy where the majority will decide what's correct or what's incorrect.

Hence, the nature of SO is subjective too. No matter how helpful, correct, elaborated and well-written answer you think you have posted, the OP will decide whether it answers his/her question or not. Further, the reviewers, or other users (like you and me) may or may not vote the answer, which in-turn means whether we agree to the answer or feel the same way as OP or do nothing about it.

Let us look at it from another angle. This answer may appear wrong or giving falsified information or can be marked as irrelevant by many or a few or none. To me, SO serves as a platform where we all can ask questions and get answers or feedback. I felt to contribute my thoughts that I think may help you. You, out of all the SO, have the authority to mark it as Correct Answer while other users have a chance to register their views which may guide you or confuse you or do nothing.

If you are concerned about the correctness of an answer or comment, either post your correct version or don't take it. You have done the first which is good but if you did the latter, it would be good too. We all exercise our choices here (under SO framework) and nothing is absolute.

2
  • My subjective truth breaks down in front of the awesome and absolute power of stuff actually working.
    – user241244
    Commented Feb 27, 2019 at 16:50
  • @D_N and still it would "break down" for you and not necessarily for anybody else :) I appreciate the use of "My" in your comment.
    – Najam
    Commented Mar 3, 2019 at 5:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .