Skip to main content
5 of 6
Commonmark migration

This change was certainly... controversial. It's been running for about two months now, and I've collected quite a bit of data. I liked the stats Kendra put together from the initial set of data, so I tried to collect some similar aggregate numbers for the full two months:

Total GaveUp Truncated Replaced Expanded Doubled Upvoted Flagged 
----- ------ --------- -------- -------- ------- ------- ------- 
3987  19.5 % 16.7 %    27.6 %   45.5 %   5.8 %   14.3 %  0.9 %   
  • GaveUp == the commenter did not post a comment after encountering the blacklist
  • Truncated == the commenter merely removed the +/-1 and proceeded to post the rest of the comment
  • Replaced == the commenter replaced the +/-1 with a different prefix
  • Expanded == the commenter posted a longer comment than originally attempted
  • Doubled == the commenter posted a comment at least twice the length of that originally attempted
  • Upvoted == the comment that was posted attracted at least one upvote (note: 12.7% of all comments posted during the test period were upvoted)
  • Flagged == the comment that was posted attracted at least one flag (note: 0.9% of all comments posted during the test period were flagged)

Before attempting to draw any conclusions from this, I reflected on the analysis done by Jon Ericson and Andy regarding comment noise, specifically that length is a massive indicator of noisy or nonconstructive comments. So I broke down the stats by the length of the attempted comment:

AttemptedLen Total Cumulative% GaveUp Truncated Replaced Expanded Doubled Upvoted Flagged 
------------ ----- ----------- ------ --------- -------- -------- ------- ------- ------- 
<30          864   21.7 %      36.3 % 6.4 %     20.5 %   44.3 %   12.3 %  10.3 %  0.7 %   
<45          604   36.8 %      27.3 % 14.1 %    30.1 %   47.4 %   9.3 %   10.6 %  0.8 %   
<60          461   48.4 %      23.2 % 14.5 %    30.2 %   44.7 %   5.4 %   13.2 %  0.7 %   
<75          369   57.6 %      14.9 % 15.4 %    29.3 %   49.6 %   3.8 %   14.4 %  1.9 %   
<90          259   64.1 %      15.1 % 19.3 %    32.0 %   44.4 %   5.0 %   17.4 %  1.2 %   
<105         215   69.5 %      9.3 %  22.3 %    33.0 %   47.0 %   3.7 %   19.1 %  1.4 %   
<120         198   74.5 %      9.1 %  21.7 %    29.8 %   46.5 %   2.0 %   11.6 %  2.5 %   
<135         172   78.8 %      8.7 %  19.2 %    29.1 %   44.2 %   1.7 %   22.7 %  0.6 %   
<150         153   82.6 %      3.3 %  24.8 %    34.0 %   48.4 %   0.7 %   17.6 %  1.3 %   
<165         110   85.4 %      2.7 %  27.3 %    38.2 %   50.0 %   0.9 %   20.9 %  0.0 %   
<180         84    87.5 %      2.4 %  26.2 %    31.0 %   42.9 %   0.0 %   14.3 %  0.0 %   
<195         72    89.3 %      6.9 %  26.4 %    26.4 %   41.7 %   0.0 %   13.9 %  0.0 %   
<210         54    90.7 %      9.3 %  29.6 %    24.1 %   33.3 %   0.0 %   22.2 %  0.0 %   
<225         40    91.7 %      2.5 %  30.0 %    30.0 %   40.0 %   0.0 %   10.0 %  0.0 %   
<240         42    92.7 %      14.3 % 26.2 %    26.2 %   42.9 %   0.0 %   23.8 %  0.0 %   
<255         37    93.7 %      5.4 %  32.4 %    18.9 %   35.1 %   0.0 %   32.4 %  0.0 %   
<270         44    94.8 %      6.8 %  27.3 %    31.8 %   38.6 %   0.0 %   15.9 %  2.3 %   
<285         34    95.6 %      11.8 % 23.5 %    20.6 %   55.9 %   0.0 %   23.5 %  0.0 %   
<300         25    96.2 %      4.0 %  28.0 %    32.0 %   40.0 %   0.0 %   16.0 %  0.0 %   
<315         12    96.5 %      0.0 %  41.7 %    8.3 %    50.0 %   0.0 %   41.7 %  0.0 %   
<330         12    96.8 %      16.7 % 25.0 %    8.3 %    58.3 %   0.0 %   16.7 %  0.0 %   
<345         19    97.3 %      0.0 %  42.1 %    21.1 %   31.6 %   0.0 %   5.3 %   0.0 %   
<360         10    97.6 %      0.0 %  40.0 %    10.0 %   30.0 %   0.0 %   20.0 %  0.0 %   
<375         14    97.9 %      0.0 %  28.6 %    7.1 %    35.7 %   0.0 %   14.3 %  0.0 %   
<390         11    98.2 %      9.1 %  18.2 %    27.3 %   45.5 %   0.0 %   18.2 %  0.0 %   
<405         6     98.3 %      0.0 %  50.0 %    33.3 %   0.0 %    0.0 %   50.0 %  0.0 %   
<420         5     98.5 %      0.0 %  20.0 %    40.0 %   60.0 %   0.0 %   0.0 %   0.0 %   
<435         6     98.6 %      0.0 %  0.0 %     33.3 %   83.3 %   0.0 %   33.3 %  0.0 %   
<450         10    98.9 %      0.0 %  10.0 %    0.0 %    50.0 %   0.0 %   40.0 %  0.0 %   
<465         6     99.0 %      0.0 %  33.3 %    0.0 %    66.7 %   0.0 %   50.0 %  0.0 %   
<480         8     99.2 %      12.5 % 25.0 %    12.5 %   62.5 %   0.0 %   12.5 %  0.0 %   
<495         2     99.3 %      0.0 %  0.0 %     0.0 %    50.0 %   0.0 %   0.0 %   0.0 %   
<510         3     99.3 %      0.0 %  0.0 %     0.0 %    66.7 %   0.0 %   0.0 %   0.0 %   
<525         3     99.4 %      33.3 % 0.0 %     0.0 %    33.3 %   0.0 %   0.0 %   0.0 %   
<540         5     99.5 %      0.0 %  0.0 %     0.0 %    80.0 %   0.0 %   0.0 %   0.0 %   
<555         3     99.6 %      0.0 %  66.7 %    0.0 %    0.0 %    0.0 %   0.0 %   0.0 %   
<570         2     99.7 %      0.0 %  0.0 %     0.0 %    50.0 %   0.0 %   0.0 %   0.0 %   
<585         6     99.8 %      16.7 % 16.7 %    16.7 %   33.3 %   0.0 %   16.7 %  0.0 %   
<600         7     100.0 %     0.0 %  28.6 %    0.0 %    14.3 %   0.0 %   0.0 %   0.0 %   

Clearly, the vast majority of these comments were originally short. REALLY short. The bulk of the comments that weren't posted at all fell into this category. I'm sympathetic to the arguments some have made that +/-1 can be an effective shorthand when posting an extensive annotation, but in most cases this brevity was unnecessary.

More importantly, the block successfully motivated a significant number of authors to expand their comments, not just to replace or remove the prefix. In the face of an almost crushing volume of comments, anything that motivates authors to make them more useful is a good thing. Purely in terms of comments that didn't get posted, this block likely saved several days of work that would've been spent moderating them.

So, I'm not in any great hurry to get rid of it completely. But there's a rather obvious way to reduce the frustration it causes for conscientious commenters...

Comment Blacklist 2: length-based boogaloo

It appears that dropping this restriction for comments that exceed 120 characters in length would preserve the bulk of the positive benefits, while getting out of the way of folks who're taking the time to write reasonably informative content.

So that's what I've done. I've also re-worked the guidance presented upon encountering the block:

Easily-avoided blacklists may never be perfectly effective, but in lieu of better solutions to the comment problem, efforts to educate folks about constructive commentary are the best option we have. I apologize if you found this test (or this outcome) annoying, but please keep in mind the bigger cost of doing nothing.

As always, feedback and suggestions welcome.

Shog9 Mod
  • 159.4k
  • 177
  • 1.2k
  • 1.2k