109

I recently stumbled upon the Data Explorer and figured I'd give it a try to practise my SQL. I wrote a query to fetch all users that had received serial voting (here defined as having more than 10 posts created on previous days, which were all upvoted in a short time interval) on the same day that they received the Mortarboard badge. The result was a list of 83 users on Stack Overflow. I figured most of these users would have received a serial upvoting reversed correction. However, to my surprise, it seems that most of these users has gotten away with serial upvoting.

Here are a few example to highlight the issue (screenshots at the end of the post):

User 3261432:
Received on Feb 5 '14, within a 2 minute interval, 11 upvotes on different posts with ages 1-5 days. Happened again an hour later with 13 posts. Cumulated to >200 points. The next day the user received only a -25 points reversal.

User 1407020:
Received on Jan 13 '13, within a 5 minute interval, 12 upvotes on different posts with ages 1-6 days. Cumulated to 130 points. Never received a reversal.

User 3049546:
Received Jan 3 '14, within 10 minutes, 11 upvotes on different posts. Then, later the same day received 14 more within half an hour (in two bursts). Cumulated to >200 points. Never received a reversal.

A few of users have rightfully gotten a full reversal (such as user 2940056), but most have either been overlooked or have only received a minor adjustment (such as user 2931123, who got 37 points out of ~400 reversed).

With all this information, my question question is therefore: Have these users purposely been overlooked, or is there something wrong with the script that catches fraud voting? Because if something as simple as my query, which does not have access to moderator info (such as who votes on what), can find this many overlooked serial voters, there must, in my opinion, be something wrong.


Screenshots of current reputation for users 3261432, 1407020, and 3049546, respectively:

enter image description here


16
  • 140
    Looks like you've just created a pretty awesome witch-hunting script.
    – Mysticial
    Commented Aug 5, 2014 at 22:10
  • 2
    Needs a little work. It should out votes where the PostId is not unique. Looks like all the high rep users simply getting upvotes on old questions (upvotes in 2014 for a 2009 post; but some are old votes as well). Which is how it is supposed to work. Unless an 88K user is bothering with multiple sock puppets those seem legit. Is there no time associated with the VoteDate ("short time interval" in OP doesnt show)? That might be more like what the real script does. Some of the low rep ones do look flaky. Commented Aug 5, 2014 at 22:24
  • 15
    @Mysticial Witches? Where? prepares pitchfork
    – Undo Mod
    Commented Aug 5, 2014 at 22:26
  • 32
    @Undo can we toss them in water to see if they float?
    – user456814
    Commented Aug 5, 2014 at 22:35
  • @Cupcake At first I thought you were referring to the Salem Witch Trials, then I saw 'monty python'. :D
    – hichris123
    Commented Aug 5, 2014 at 22:45
  • 4
    @Plutonix As the Mortarboard badge only can be achieved once, 88K users are pretty much in the clear. I agree it needs more work, but it can quickly become very messy without moderator access to IP addresses etc.
    – dwitvliet
    Commented Aug 5, 2014 at 22:47
  • 10
    That's some awesome rough querying there. Before I was a mod, I enjoyed the "forensic analysis" opportunities in looking for this stuff. Never thought of using the data explorer.
    – Andrew Barber Mod
    Commented Aug 5, 2014 at 23:16
  • 3
    @AndrewBarber Time to do some laundry I guess... and pair the socks... awwww yeah! Commented Aug 6, 2014 at 0:35
  • @AndrewBarber I have a query that lists all the suspended users in descending order of rep. It's not 100% accurate, but it's close enough. The difference is that those don't need moderator attention because they already got it. :)
    – Mysticial
    Commented Aug 6, 2014 at 0:41
  • 7
    Yeah, well, @Mysticial, I have a query that shows all the orders for aniseed syrup by customers in Dallas!
    – Andrew Barber Mod
    Commented Aug 6, 2014 at 0:47
  • 2
    Socks will be scared of bananas now :P Commented Aug 6, 2014 at 4:18
  • 12
    "Unless an 88K user is bothering with multiple sock puppets those seem legit." - How did you think they became an 88K user? MWUAHAHAH ;-)
    – GolezTrol
    Commented Aug 6, 2014 at 15:02
  • 7
    I just serial upvoted your question.
    – ouflak
    Commented Aug 6, 2014 at 15:07
  • 2
    @Cupcake We have to make sure they weigh the same as a duck.
    – John Odom
    Commented Aug 6, 2014 at 15:11
  • 4
    Duck! Now I am offended :| Commented Aug 6, 2014 at 15:49

2 Answers 2

85

First, before I say anything else, please don't start a witch hunt based on this query. We know people all too often like to blindly flag things based on a query, so please don't do that or otherwise go after these users.

As to why apparent serial voting hasn't been reversed by the system, the serial voting script has been designed to be conservative and to minimize collateral damage from innocent voting activity. This occasionally lets targeted voting through. Also, commonly people trying to defraud the voting system try to serially vote between accounts, get it reversed, try again with a smaller number or more staggered across time, and repeat until the votes stick.

For several cases like this, we will catch the voting patterns later and remove sock puppet accounts or get votes manually invalidated. That will show up much later in the reputation history and might be hard to associate to a particular period of serial voting. Looking through your list, I see a couple of those. I also see a bunch of completely innocent folks who just happen to be highly active in certain tags.

Let me comment on how we generally handle suspected voting fraud. First, we tend to only step in for the most obvious or most troublesome cases. Every single case of this has to be inspected by a moderator, and we have to be absolutely certain that there's a problem before we act on it. Unless someone's really lazy and is doing things like giving their sock puppet the exact same name as their main account (which happens quite frequently), it takes effort to investigate a single account.

Moderators cannot invalidate votes, and the most we can do ourselves is to delete clear sock puppet accounts. For the rest, we have to manually notify SE employees and provide a clear case to invalidate votes between accounts. They then have to look into it even deeper to see if we were right. That means that apparent coordinated voting between accounts of people who work together, but where the accounts are each legitimately operated by a different person, can take a while to deal with. Glancing at your list, many of the ones I see there fall into this category.

Unfortunately, this kind of coordinated voting is prevalent in users from certain cultures, for reasons that Tim explores in this article. Also, it can be hard to tell where to draw the line between the natural impulse to vote for people you know and when this becomes excessive. This means that sometimes we knowingly allow targeted voting to pass, because it just hasn't crossed the threshold where we feel we need to get SE employees to intervene.

Making targeted vote invalidation available to moderators so that we could deal with this more easily might seem like a way to combat this, but handing a group of non-employee users the ability to change votes leaves open the potential for us to abuse this or otherwise reduce trust in the voting system.

So that's a fairly long-winded explanation of why moderators haven't dealt with every single person in this query. Some are innocent, some have been handled already, a few we missed, and some aren't yet at the stage where we feel the need to intervene. Again, please don't turn this into a witch hunt based on a simple query.

3
  • 10
    Thanks for the Tim Post article. That was fascinating. Commented Aug 7, 2014 at 13:24
  • I really wanna read it that article, but link rot killed it. Where did it go?
    – Pimgd
    Commented Mar 27, 2015 at 10:48
  • 7
    @Pimgd The article is archived here. Commented Aug 11, 2015 at 3:35
67

Keep in mind that some patterns like these can actually be legit. I've seen votes reversed that should not have been because the reversal script was too touchy. It's a hard balance to catch offenders while not creating false positives. In order to do it well, there will necessarily be some cases that do not get auto-reversed.

The other day I was looking for the answer to a specific programming issue and ran across a dozen or so SO questions that seemed related. I opened them all in tabs, scanned them quickly and closed the tabs that looked less related or had nothing but down-voted answers, then skimmed through all the ones that looked promising to see generally how they were handling the scenario. Once I got a general idea of what the solution was and how to understand the problem, I went back through again and up-voted all answers that were clear and had helped guide me. By the end I realized that the best answers on almost all those questions were from one user who seems to be a bit of a ninja in that field. I probably gave him 15 up-votes in less that 5 minutes.

From what you describe, your query would have that user (who did nothing wrong) strapped to a chair in the middle of a pond.

Of course you might find some legitimate issues this way too. They are out there for the finding.

  • Since the scanner is currently tuned to catch egregious instances without too many false positives (which are way more trouble than missing a few) there will always be some abusers that get away with something.

  • Besides the auto-reversal script which you can see the results of, there are also manual interventions. Mods have access to some tools that spot and warn them of patterns that the automatic tools haven't taken action on. These are meant to be investigated further by hand.

    You can't see the extra details that are used to investigate these so you're far more likely to end up with false positives in your analysis than the actual system is. Leave some room in your understanding for the possibility that some of the cases you spot are actually legitimate for one reason or another.

8
  • 1
    I agree that this method undoubtedly will give a bunch of false positives - that's what I expected from something this simple and crude. What surprises me is how many obvious true positives it actually does give, when, as you say yourself, mod tools are far superior. Shouldn't these few obvious examples already have been caught?
    – dwitvliet
    Commented Aug 6, 2014 at 13:36
  • 1
    I have done the same with tabs in the past. Commented Aug 6, 2014 at 14:57
  • 15
    @Banana I think you missed part of the point of this post. You say «What surprises me is how many obvious true positives it actually does give», but you don't actually have a way of confirming if something is a true positive! You can't see extenuating conditions such as whether a post was listed in the supercolider or offsite or whether votes are actually from the same account(s) or ip(s). You simply have no way of confirming a positive hit and there are lots of ways you could have false positives.
    – Caleb
    Commented Aug 6, 2014 at 15:16
  • 2
    Nobody gets strapped to a chair when serial upvoting is detected. Most definitely not the person receiving the votes, since they have nothing to do with it (unless it's a real case of fraud). All that happens is that the upvotes and corresponding rep go away. Commented Aug 6, 2014 at 15:27
  • 1
    @Caleb I can't. Mods can. As most (3 of 4) of the users I mentioned in my post already have received a reputation reversal because of mod attention (2 of them by user deletion), that IMO confirms that at least these users were true "obvious" positives.
    – dwitvliet
    Commented Aug 6, 2014 at 15:28
  • 4
    @Banana: Caleb is well aware of the issues involved; he's a mod elsewhere on the SE network. Commented Aug 6, 2014 at 19:03
  • @Caleb - "you don't actually have a way of confirming if something is a true positive! You can't see extenuating conditions such as whether a post was listed in the supercolider " - nonsense. None of the cases that Banana listed involve a single post receiving lots of upvotes; they are all cases of otherwise low-rep users suddenly getting 10+ upvotes on different posts within 2 minutes. That is more than enough evidence to be sure that serial user-targeted voting is going on. If the votes are from different accounts and IPs, that just means it was unusually sophisticated malfeasance.
    – Mark Amery
    Commented Jun 12, 2018 at 11:00
  • 1
    @Mark Did you even read my post? I outline one of several scenarios I've encountered over the years where seemingly obvious malfeasance is actually legitimate. The trouble here is you and others sometimes assume there is no other option but foul play when reality is somewhat more complex and you don't have the tools to identify false positives.
    – Caleb
    Commented Jun 14, 2018 at 10:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .