-9

I've seen some questionable uses of the mighty Mjölnir. I'd like to start downvoting them so the site can gather statistics on the improper uses of it, and possible remove the privilege from robo-reviewing gold badges.

How can we downvote the use of the hammer?

My apologies for the discussion tag. I'm not sure if this is a "request for help" (its available but I'm not doing something correctly) or a "feature request" (check and balance for the robo-reviewers).


UPDATE (MAY 2016): As Shog's data shows, there a non-trivial amount of mistakes being made. If a reviewer were failing audits at a rate of 25%, then they would get the Ban. Also see What are review tests (audits) and how do they work? and Why am I getting the message “You've failed too many recent review audits”?

I really feel the system needs a feedback loop to ensure this tool is being used properly.

3
  • 2
    I've seen what I would consider abuses, but I'm less convinced that it's "robo-" anything, really. It does help that I have a Mjolnir of my own though to help combat any disagreements that I see, at least in the one tag...
    – Makoto
    Commented Jan 21, 2016 at 1:04
  • 17
    How can we downvote you downvoting the use of the hammer? You simply click the reopen link. We can't do anything about it. Commented Jan 21, 2016 at 1:04
  • @HansPassant - as soon as the site provides the procedure for auditing the use of the hammer, I'm guessing it will be able to audit the auditors, too. But I suspect its a useless use case. There's nothing beyond failed audits (auditing the audit), so there probably won't be anything available.
    – jww
    Commented May 3, 2016 at 23:18

1 Answer 1

58

You can't downvote the use of a close vote. That'd be some kind of meta-voting that doesn't really exist here.

You can reopen improperly closed questions though. And you should! This is trivial to track (indeed, you can find evidence of such events in the public data dumps), and also serves to correct the problem at hand.

In particularly egregious cases, don't hesitate to flag for moderator attention OR bring the matter up for discussion here on meta.

Matt was curious about this a while back, so together we dug into the data. Turns out that over the past 365 days,

  • 55,982 questions were closed using the "hammer". Of these, 4,090 were later reopened - an "error rate" of about 7% (note that "error" here ignores questions that were legitimately closed, then edited, then reopened; modifying the query to account for these is left as an exercise for the reader).
  • 10,193 questions were closed by the system in response to affirmation from the asker themselves. Of these, 134 were later reopened, for an error rate of about 1%.
  • 8565 questions were closed as duplicates in the normal fashion, by voters or moderators. Of those, 259 were later reopened, for an error rate of about 3%.

One caveat applies here: roughly 20% of duplicate questions were deleted at some point during the past year, and are not counted in the above stats. You can safely assume they were not reopened.

Your concern was particularly focused on "robo-reviewers", or people who might use their privilege frequently without careful thought. We looked into this as well, focusing on the top 100 users of the hammer:

User Link               HammerClosed Reopened PctError 
----------------------- ------------ -------- -------- 
Community               15906        735      5        
BalusC                  2014         67       3        
Sotirios Delimanolis    1202         95       8        
John Conde              1022         37       4        
Fred -ii-               932          56       6        
Bergi                   850          51       6        
Jarrod Roberson         845          108      13       
Rizier123               756          35       5        
Quentin                 712          30       4        
πάντα ῥεῖ               583          117      20       
Felix Kling             555          29       5        
cimmanon                538          31       6        
Barmar                  516          53       10       
Martin R                477          16       3        
Tunaki                  420          55       13       
Paulie_D                399          36       9        
jonrsharpe              388          30       8        
Hovercraft Full Of Eels 378          34       9        
Alexei Levenkov         360          48       13       
rmaddy                  337          10       3        
mario                   319          7        2        
Wiktor Stribiżew        309          34       11       
CodeCaster              308          39       13       
deceze                  307          16       5        
Martijn Pieters         290          18       6        
Hrundi V. Bakshi        288          14       5        
David Arenburg          287          20       7        
rayryeng                260          25       10       
EJP                     250          6        2        
Pshemo                  239          18       8        
CL.                     239          9        4        
TigerhawkT3             234          22       9        
Rory McCrossan          229          21       9        
Marc B                  224          9        4        
Oriol                   223          11       5        
Arun P Johny            222          21       9        
tripleee                213          5        2        
MadProgrammer           211          16       8        
Hans Passant            207          16       8        
Your Common Sense       207          20       10       
Bhargav Rao             202          19       9        
juanchopanza            186          14       8        
Bo Persson              186          24       13       
Tushar                  184          26       14       
CommonsWare             183          6        3        
Dagon                   178          13       7        
hakre                   175          5        3        
Padraic Cunningham      174          9        5        
Josh Caswell            173          7        4        
T.J. Crowder            173          22       13       
JohnnyHK                173          8        5        
Jon Skeet               169          10       6        
akrun                   154          32       21       
Blackbelt               150          9        6        
James_D                 149          7        5        
Cerbrus                 149          20       13       
torazaburo              148          11       7        
David Heffernan         148          23       16       
Daniel A. White         146          19       13       
PreferenceBean          143          12       8        
Praveen Kumar           141          32       23       
Peter Duniho            141          10       7        
J.F. Sebastian          136          8        6        
davidism                136          15       11       
Soner Gönül             136          9        7        
CoryKramer              132          23       17       
mplungjan               131          11       8        
matt                    128          9        7        
talonmies               127          3        2        
alk                     126          8        6        
mu is too short         125          5        4        
Stephen Muecke          125          2        2        
durron597               125          5        4        
Leo Dabus               121          30       25       
Yuval Itzchakov         117          15       13       
Jonathan Leffler        116          10       9        
epascarello             115          3        3        
M D                     114          8        7        
assylias                114          9        8        
j08691                  113          10       9        
R Sahu                  113          13       12       
Lars Kotthoff           112          4        4        
Mark Rotteveel          112          6        5        
Charles Duffy           110          7        6        
NathanOliver            110          14       13       
Makoto                  108          17       16       
nhahtdh                 108          10       9        
thefourtheye            107          14       13       
fedorqui                103          10       10       
jfriend00               103          8        8        
web-tiki                97           8        8        
Patrick Hofman          95           23       24       
Josh Crozier            95           6        6        
Gert Arnold             94           6        6        
jtbandes                93           3        3        
squint                  93           8        9        
Bryan Oakley            93           2        2        
adeneo                  93           7        8        
charlietfl              91           4        4        
Martin Smith            90           9        10       

I should note here that the first entry - "Community" - represents questions that were closed with the participation of a hammer-wielding voter and other close voters. Beyond that, the same caveats for error rate apply as for the previous statistics. You'll note that the most avid user of the hammer (by a wide margin!) has an error rate comparable to that of normal voters.

It's useful to also consider the number of questions reopened using the "hammer" - and how many of the questions reopened were closed by the person reopening them. Closing a question in need of clarification and then reopening it upon receiving that clarification is one of the most satisfying uses of this tool, in my opinion:

User Link               HammerReopened ReopenedOwnClose 
----------------------- -------------- ---------------- 
Community               785            174              
πάντα ῥεῖ               69             50               
T.J. Crowder            52             10               
Boann                   48             0                
BalusC                  46             45               
Fred -ii-               43             33               
Bergi                   43             17               
Makoto                  42             7                
akrun                   41             24               
Barmar                  40             31               
Alexei Levenkov         37             29               
Wiktor Stribiżew        37             18               
Sotirios Delimanolis    36             30               
Praveen Kumar           29             20               
CodeCaster              28             26               
Pshemo                  27             14               
Felix Kling             27             22               
Josh Crozier            26             3                
juanchopanza            25             7                
Rizier123               24             17               
rayryeng                23             15               
Paulie_D                23             22               
John Conde              22             19               
Hovercraft Full Of Eels 22             19               
Leo Dabus               21             16               
Jarrod Roberson         20             9                
Tushar                  20             14               
jonrsharpe              20             14               
dasblinkenlight         19             2                
ρяσѕρєя K               18             5                
Rory McCrossan          18             16               
Martijn Pieters         18             14               
A. Wolff                17             12               
David Heffernan         17             9                
Nathan Hughes           17             12               
Padraic Cunningham      17             3                
Gordon Linoff           17             0                
Patrick Hofman          16             12               
David Arenburg          16             13               
Bhargav Rao             16             11               
Quentin                 16             12               
PreferenceBean          15             4                
Martin R                15             11               
TheLostMind             15             6                
Chris Stratton          15             2                
chrylis                 14             2                
Arun P Johny            14             12               
deceze                  14             6                
Maroun Maroun           14             8                
Tunaki                  13             13               
camickr                 13             3                
Xan                     13             9                
thefourtheye            13             6                
Madara Uchiha           13             7                
MadProgrammer           13             10               
J.F. Sebastian          12             5                
Lundin                  12             4                
M.M                     12             3                
Jon Skeet               12             5                
cimmanon                11             10               
Steve                   11             7                
zero323                 11             11               
Richard Scriven         11             9                
vaultah                 11             10               
rmaddy                  11             5                
Benjamin Gruenbaum      11             1                
D Stanley               11             7                
jfriend00               11             4                
assylias                11             5                
zenith                  10             8                
Kasramvd                10             2                
TigerhawkT3             10             9                
Cerbrus                 10             4                
EJP                     10             1                
Guffa                   10             0                
T.C.                    10             3                
Drew Dormann            10             3                
user2357112             10             3                
nhahtdh                 10             3                
Piotr Skotnicki         10             0                
Tim Schmelter           10             5                
amit                    9              3                
iharob                  9              3                
Bathsheba               9              1                
Oriol                   9              4                
Holger                  9              1                
Your Common Sense       9              7                
Bo Persson              9              4                
R Sahu                  9              5                
Borodin                 9              0                
Ben Voigt               9              1                
Joachim Pileborg        9              5                
Stephen C               9              6                
sᴜʀᴇsʜ ᴀᴛᴛᴀ             8              1                
fedorqui                8              5                
Satpal                  8              5                
Pointy                  8              4                
Scott Chamberlain       8              3                
Vohuman                 8              3                
Andy Thomas             8              0                
23
  • Thanks Shog. Maybe I'm splitting hairs... A close vote is one thing, the hammer is a completely different matter. And to be clear, I'm not suggesting flagging close votes.
    – jww
    Commented Jan 21, 2016 at 1:28
  • 11
    Fanciful language aside, the "hammer" is just a vote that counts a bit more than a normal vote. In every other fashion, they behave the same, are handled by the same logic, and - when misused - are corrected in the same manner. If you're interested in stats on use (including error-rates), I can post them.
    – Shog9
    Commented Jan 21, 2016 at 1:30
  • @Shog9: That'd be useful to know, actually...
    – Makoto
    Commented Jan 21, 2016 at 1:36
  • 1
    Added, along with queries for you to play with @Makoto
    – Shog9
    Commented Jan 21, 2016 at 2:13
  • What is the source of the difference between 10,193 Community-closures cited in the text, and 15,420 ascribed to Community in the table?
    – user3717023
    Commented Jan 21, 2016 at 2:25
  • See the last note, @Norma: in the first set of stats, Community user is closing questions on behalf of the asker; in the table, Community is a stand-in for "multiple people voted so crediting the closure to one person is unfair and also I'm too lazy to try and summarize that in the table itself".
    – Shog9
    Commented Jan 21, 2016 at 2:27
  • I'm on the List. dun dun duh I don't know how to read the percent error statistic. I, and most people on the list, have higher error rates than the other dupe methods. Is this actually significant? Are we doing something wrong? I don't feel like the hammer has been misused though.
    – davidism
    Commented Jan 21, 2016 at 3:00
  • See my notes about how "error" is calculated, @davidism. If you want to know what it means, you'll have to dig into the actual cases where something you closed was later reopened. I'm entirely too lazy to put real work into this tonight, but a few cheap ways to make this more accurate might include: excluding cases where you reopened yourself, where the post was edited between closing and reopening, and where it was eventually re-closed. Remember, this is all public data; play with it to your heart's content.
    – Shog9
    Commented Jan 21, 2016 at 3:02
  • 10
    All that said, my gut feeling is that there isn't enough to worry about here. Globally, the "error" rate is still quite low for hammer-closes, while the volume of questions closed in this manner completely dwarfs all other methods of dup-closure. This is by far the most effective thing we've ever done at improving the duplicate situation, and if the cost is a slightly higher rate of error, so be it - the same change that allows faster, easier closure also allows for faster, easier reopening.
    – Shog9
    Commented Jan 21, 2016 at 3:06
  • 9
    Also a note on the error margin - Some 'errors' were duplicates, but could have been pointed at a better canonical. So in some cases the closing isn't what's disputed, it was the target.
    – user50049
    Commented Jan 21, 2016 at 5:33
  • It's public data, so you can add this yourself if you wish @BalusC (I'll try & get to it eventually, but it'll be a bit). Also see my comment to davidism - I'm, ahem, erring on the side of error here, but even with that I have to be impressed by your work.
    – Shog9
    Commented Jan 22, 2016 at 20:18
  • "Closing a question in need of clarification..." as a duplicate, rather than "unclear"?
    – user3717023
    Commented Feb 21, 2016 at 23:20
  • 2
    @Sally First it looks like a dupe of A, and gets closed thusly, but then the OP clarifies what they really meant. Commented Feb 22, 2016 at 0:09
  • 1
    Just a note here. Your last sentence "Closing a question in need of clarification and then reopening it upon receiving that clarification is one of the most satisfying uses of this tool, in my opinion:" is just an assumption and, well, I know at least on guy on that last list that has a high "HammerReopened" rate because he uses his hammer in order to revenge other people for hammering his answers with legitimate dupes by closing their answers by some random/unrelated dupes - and hence these questions are being reopened. @jww has a point here- there are people who abuse this privilege. Commented May 3, 2016 at 19:03
  • 1
    Though TBH, I like the Mjölnir very much, but indeed it is being been used, same as downvotes, as method of punishment/revenge, though unlike the serial downvoting, it isn't being monitored by the system. Commented May 3, 2016 at 19:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .