81

Over the holidays I have reviewed over 300 tagged questions with my team to bring them up to date as Kotlin has left alphas and betas and moves towards release of 1.0. We want content to be correct for the current state of Kotlin.

During this time, probably 95% (or more) of all content on the tag was me, as one author. New answers, edits, comments, and self-answered questions for idiomatic items. All with more current, accurate, detailed, and thorough content.

So, people helping in this effort and reviewing content are watching the tag, and also the chat rooms where edits on these tags are noted. They vote when they see the posts come in as they agree. And they agree often, maybe because I am one of the longest users of Kotlin and I am writing the best content.

Then, one of the Kotlin contributors received an email, which he forwarded to me:

Hello,

I'm writing in reference to your Stack Overflow account:

https://stackoverflow.com/users/1215902/alexander-eliseyev

We recently noticed a substantial number of votes on your account to or from specific users. While we encourage everyone to upvote great posts, the motivation for doing so needs to be anchored in the merits of the post, not the person who wrote it. This is just a reminder to please refrain from targeting specific users when voting.

Stack Exchange works by ensuring that the best information rises to the top. Voting specifically for content that benefits your friends, family or colleagues unfairly skews that system, and continuing to do so can result in a lengthy suspension for all involved.

Please note that the offending votes have been invalidated. The system has processes in place to detect various types of voting between groups of users, so if you know anyone who may be voting for your stuff in kind, please ask them to refrain before the system detects such activity and takes similar actions on their account. If by chance you created separate accounts for the purposes of voting for another, please send me a link to those profiles so I can merge them without further incident.

This is often just a simple misunderstanding, so no harm done. But continued activity of this sort can result in a prolonged suspension for all involved, so please let me know if you have any questions regarding this policy.

Regards, meagar Stack Overflow moderator

Sorry, but moderator meagar didn't do their homework. (they and others did do their homework, just didn't have all the info) If in 2 weeks I went from 0 reputation on the tag to 2nd all time for the tag, and all content on the tag was from me during that time, and people watch the tag because we ALL AGREED TO HELP MAKE THIS TAG BETTER OVER THIS TIME FRAME, then we should not be punished for improving content for the tag and Stack Overflow.

People voted mostly for me, because there was only me to vote for.

The votes should be reinstated (the receiver of the email has asked for that as well), and nothing should be noted on our accounts as a negative since this is a group effort to review and improve content and not some voting scheme. The quality of the posts and answers should make that clear. People are helping the review by watching changes and voting to help push down old content that is bad/outdated and bring up new good content.

Examples like this:

or from a few minutes ago:

And I'm sure moderators can see the hundreds of changes I've made in a short time.

How do I get this resolved?

Kotlin Tag Stats

Update for January 28 after voting reversal:

Looking at the accepted answers and distribution of users for the Kotlin tag which is what I mostly only work under... (note votes are all votes, up and down)

In January the distribution of posts with accepted answers and the owner of the accepted answer is:

Answerer Number of
Posts Accepted
% of Posts Avg Answer
per Post
Avg Post
View Count
Votes
3679676 (me) 14 25.45% 1 236 150
155213 (miensol) 8 14.55% 1 65 34
147024 (yole) 7 12.73% 1 70 38
2196460 (hotkey) 4 7.27% 1 87 24
615306 (kirill) 2 3.64% 3 83 9
1538877 (IRus) 2 3.64% 1 53 8
18 others 18 (1 per) 1.82% each 1.44 38..414
(62 med)
5.3 avg
(5 med)

And for December:

Answerer Number of
Posts Accepted
% of Posts Avg Answer
per Post
Avg Post
View Count
Votes
3679676 (me) 13 20.31% 2 300 87
3255152 8 12.50% 1 101 25
2196460 3 4.69% 1 85 21
3144601 3 4.69% 1 109 10
3699808 3 4.69% 1 95 16
288456 2 3.13% 1 41 5
615306 2 3.13% 1 54 17
885028 2 3.13% 2 368 10
28 others 28 votes, 1 per 1.56% each 1.36 30..462
(79.5 med)

So I had 27 accepted answers in 2 months which was 45% of accepted answers for the Kotlin tag in that time period and also 45% of all post views with accepted answers were for these posts. Most of these posts averaged 1 or 2 answers, meaning it was likely that I was the only answer, the first answer, or competing with only 1 answer. And as accepted answer and with more votes, my answer likely is appearing first. Therefore it is likely people that monitor the tag would always be voting for me 45% of the time when viewing posts that had accepted answers.

So how would you detect serial voting in that case?

Looking at all answers, in January I was 33% of all answers and 42% of all votes, for questions that had 54% of the views. So 1 out of 3 answers would be mine, and people were voting slightly higher than that ratio. In December 24% of all answers and 29% of all votes, for questions that had 50% of the views. And again for both months there were average of 2 answers per question in which I answered, not a lot of alternatives for voters. So 1 out of 4 answers would be mine, and people were voting about that ratio.

Again for all answers, I see that I had in January, 14 accepted votes, 232 upMod, 8 downMod, 2 undelete, 2 moderatorReview. December 13 accepted, 104 upMod, 4 downMod, 2 ModeratorReview. So I guess that means 4 answers were flagged for moderator review at some point.

Doesn't seem like the data means much of anything other than what was said before. Maybe moderators see other patterns, but given the attention to my posts and the compressed time frame, it seems like a silly reversal of 10% of my total reputation gained from writing the best answers, correcting old errors, and doing the grunge work of reviewing 500+ posts.

16
  • 4
    Stack Overflow has very few tools to help when content is outdated (technology changes, some answers are now wrong) other than doing what we are told: "write a new answer, and it can outvote the old answers" except that when people watch and try to do this en-masse it is seen as negative activity. If the content were looked at by the moderator, I think it would be seen as "community review and corrections" Jan 6, 2016 at 5:52
  • I think if you get a message by a mod you can reply to these messages, AFAIK. Also it says: This is often just a simple misunderstanding, so no harm done in the message. So if really nothing fishy is going on, I'm pretty sure that everything can be undone.
    – Rizier123
    Jan 6, 2016 at 5:54
  • 5
    It also says "Please note that the offending votes have been invalidated." ... and the moderator was replied to, but not all moderation is consistent or perfect here. Since this sounded like a serious message, having a review or audit of the moderator seems fair. Jan 6, 2016 at 5:56
  • It also isn't clear how many people will receive this type of message and not everyone may reply to clean it up. So having the situation reviewed as a whole would be a good outcome. Jan 6, 2016 at 5:57
  • 32
    It also says But continued activity of this sort can result in a prolonged suspension for all involved. We want to make sure it doesn't happen, as we're going to continue activity of this sort. Jan 6, 2016 at 5:58
  • As people come back from vacation, a whole large group of Kotlin people could start dropping in and doing the same voting for all the new content from me. Same problem. Jan 6, 2016 at 5:59
  • 16
    Note that moderator messages are visible - and copied - to all moderators and staff. So even if you wanted a second opinion, you could simply request it in your reply to the message and we'd see it.
    – BoltClock
    Jan 6, 2016 at 6:08
  • 15
    Also, this is just one incident. We really don't need to conduct an audit of a moderator's entire history just for one mistake he might have made.
    – BoltClock
    Jan 6, 2016 at 6:12
  • 2
    Not asking for their entire history, just this issue and people that may have all been flagged that were working on improving this tag. Jan 6, 2016 at 6:13
  • 3
    Old, old blog post on this sort of thing: blog.stackoverflow.com/2009/07/stack-overflow-flash-mobs
    – Shog9
    Jan 6, 2016 at 6:14
  • ja, instead of top 100 content, more like "we don't want the Kotlin tag to be stinking pile of poo based on weird things from 2+ years of changes during alphas and betas that confuse anyone dropping into SO and searching". but yes, a flash mob using our holiday time to get ready for 1.0 release. Jan 6, 2016 at 6:16
  • I have around 200 more posts to review by end of weekend, so here goes more! (on page 16 going forward in time towards most recent, but have seen most of the newer things) Jan 6, 2016 at 6:19
  • Wrongly invalidating votes is no big deal, but threatening emails? Not good. Jan 7, 2016 at 7:09
  • Eventually I was hit with a 330+ reversal due to unknown along with a copy of the same message as above. Jan 28, 2016 at 11:44
  • And @AndrewGrimm I definitely don't appreciate receiving the same threatening email TO ME the other day when votes to MY posts were reversed that I had nothing to do with the voters. If it is about OTHER people voting for me, then send them the emails, why would I get threatened because I'm attracting voting attention with good posts?!? Jan 28, 2016 at 11:46

2 Answers 2

101

This is an old, old problem. In the very early days of Stack Overflow, a few projects decided to make answering questions a regular event in their communities: they'd get together one day out of the month and blast through as many questions as they could, asking, editing, answering and voting. When done well, this could be a boon to both the site and the community gathering to participate here; when done poorly, everyone had a bad time.

Over the years, we've compiled a list of guidelines to help folks avoid trouble. You can find it in the help center: https://stackoverflow.com/help/product-support

To that, I'd add one more bit of advice specific to these sorts of organized events: mention it here first. In the years since that R mob, plenty of other groups have done semi-formal events like it; when they posted an announcement here ahead of time (or even just emailed us) we could offer them specific guidance and also give folks in the community a heads-up that they were coming. Without that advance notice, we're left to figure it out on our own... And when that happens, it's easy to get the wrong impression.

The moderators here are trained to look for suspicious activity; in fact, the system will notify them of many unusual voting patterns. If there's a reasonable explanation, they'll ignore them. But they do need to be able to find that explanation.

6
  • 1
    In our case we are correcting posts that are misleading based on the current state of the technology that is now stable and ready for 1.0 release. We are doing corrections first, adding answers where wrong, creating a few idiomatic posts where we need to constantly point to the same content (keeping it within SO instead of an outside link), adding comments to give the author time to make changes, suggested edits for smaller things, ... mostly it is me doing the review of the 550+ Kotlin tagged items, others watching and weighing in as they see my changes. Not quite a mob. Jan 6, 2016 at 6:25
  • I really didn't know how many edits I would find when starting the review, but it has been quite a lot. So not sure I would have though to post it in advance. I'm assuming there is the email somewhere in the product-support link... Jan 6, 2016 at 6:27
  • 4
    So be warned, I have 200+ questions left to review, and while doing so I will see any new question posted and probably be in the first few to answer. So I will attract more votes. Others are on vacation skiing, I'm doing this. So I don't have a lot of competition for attention on the tag. So voting will be weird while I generate a higher load than normal of content changes. Jan 6, 2016 at 6:29
  • 5
    I can't help but think that the introduction of Teams is just going to exacerbate this problem in the future when that feature becomes generally available.
    – user177800
    Jan 7, 2016 at 0:23
  • 3
    I had the same thought, @Jarrod. Working on ways to make this less disruptive.
    – Shog9
    Jan 7, 2016 at 0:24
  • And worse with products like ours that are going into large consulting companies which allow members of the company follow development actions of leaders in the company, for example dev leads or technology specialists (coding, reading, library uses, tweets, blogging, found solutions, code examples and SO Activity, Server Fault activity, etc). Some clients are 3.000 to 5.000 developers, other 13.000, one in india is over 5-6 times that. Jan 7, 2016 at 16:17
83

I'd like to defend meagar here by saying that he wasn't the only one looking into this, nor was he the first one to act here.

We received flags from community members yesterday that stated that you were operating falsified accounts to inflate the voting on your posts. I handled those flags and checked to see if they had any merit. In the process, I noticed that there was a coordinated pattern of voting from a small group of users towards your posts.

However, I also saw that you were leaving quality answers, that they all came within a specific tag, and that the people voting for you looked like discrete individuals with good posting histories. It's common for subject matter experts to get votes targeted towards them, particularly in smaller tags, so we know to look out for false positives.

What still made myself and the other moderators a little suspicious was the degree of voting coordination (several users that cast over 50% of all votes they had ever given to you alone), the short time period in which these all came, the way they were timed around specific posts, and the fact that at least one of these people seemed to share a physical location with you. We tried searching to see if there was some kind of event going on, but couldn't see one on Meta, in chat, or in any of the other locations we could think of.

Multiple moderators debated what to do about this yesterday in response to the accusations from community members about falsified accounts. meagar split the difference by providing a warning without doing anything else.

I should say that despite the warning, the votes haven't actually been invalidated. The phrase

Please note that the offending votes have been invalidated.

is part of the base template for the standard targeted voting message, and it's easy to forget to edit that out. If we're not quite sure about suspicious voting behavior, we'll often send this message to let people know we saw something odd and give them a chance to explain. Again, it's easy to forget to edit this to match the exact circumstances we're dealing with.

You've clearly explained what's going on here, which satisfies me about this. The voting makes sense in light of this.

17
  • 10
    Thanks Brad, makes sense from that viewpoint as well. Obviously I cannot see who makes which votes to know how coordinated it is or not or to know if I've created a snowball effect. Coworkers obviously watch what each other does, and people in Kotlin Chat who talk a lot do the same (I think one guy follows me to wait for me to make an error so he can jump on it, but also votes honestly while waiting for that moment). Sharing physical location would likely be someone within my company in Uruguay so that makes sense. Thanks for the details! (sorry meagar for my harsher tone) Jan 6, 2016 at 15:36
  • I've made an edit to the question about him "doing his homework" Jan 6, 2016 at 15:38
  • 3
    (I've turned off our slackbot that sends notices of our company member questions/answers/comments so people won't track them so closely, they can watch in whatever other way they want. this will lower the "coordination" with the feed turned off) Jan 6, 2016 at 15:43
  • 12
    @JaysonMinard - Thanks for your understanding. It's clear you meant well, but we've had enough people try to game their company's ranking, cheat the voting system to keep members out of question bans, etc. that it can sometimes be difficult to identify innocent efforts like this. As Shog9 points out, some kind of public notice or event listing is usually enough to get us to connect the dots. We know what to look for now within this tag.
    – Brad Larson Mod
    Jan 6, 2016 at 15:57
  • I think it might be appropriate to remove the last line (the one with the strike-through) in the OP @JaysonMinard. That line may have justified by the lack of understanding, but now that we all know what happened and why, I think it can go.
    – CaptJak
    Jan 7, 2016 at 16:13
  • sure, its gone. Jan 7, 2016 at 16:15
  • 3
    @JaysonMinard I'd just like to point out that you, as an organizer of events like these, should spend a bit of time to ensure that the participants understand what types of voting patterns are appropriate. If a few bad eggs in your community start voting abusively, it can reflect poorly on the rest,and possibly make mods like Brad inherently more suspicious, and less trusting/forgiving in situations like these. Make sure that the members of your community understand that if they just blindly go through all of your posts and upvote them they're harming your community, not helping it.
    – Servy
    Jan 7, 2016 at 16:20
  • They know that already. But most of my new posts are a large improvement over the existing content (the whole point of the exercise is this), therefore unless I do something stupid, they are likely to upvote the replacement answer. Jan 7, 2016 at 16:28
  • If you see a bad post of mine up-voted, first let me know it is bad because I'm not trying only to improve the quality and fix errors. Some of the complaints are probably from people valuing their reputation over quality, and therefore want their old incorrect post to stay with its vote count. In many cases I commented on their post with the suggested fix, offered that they should steal from my answer, and I would even drop my answer if they corrected theirs. Some did, others kept theirs for "historical reasons", others abandoned theirs. Jan 7, 2016 at 16:30
  • One last case, if I was trying to steal reputation with fake voting, I would need to do it slower, this many edits in a short time causes me to hit the daily cap and lose much of the vote +reputation and instead just positions the answer to let the reader know it is more correct. A reputation steal-a-thon would need to be done much more slowly. Jan 7, 2016 at 16:31
  • 1
    Just to be clear on this, it's perfectly fine (and welcome) for folks involved in a project to go through answers on the topic here, review them, and upvote quality posts. What tends to raise a red flag is when these folks vote unusually quickly and never vote on anything else - that's a classic abuse pattern. You mentioned having a feed for answers in your chat, @Jayson - that's a great example of something that's ripe for abuse, unless everyone in the room understands that they're expected to vet not just support the posts being sent through (we've dealt with this in our own chat too).
    – Shog9
    Jan 7, 2016 at 19:29
  • Yep, understood. Many things add danger. Jan 7, 2016 at 20:07
  • @BradLarson so after a pause there was a vote reversal of 330+ points, 10% loss during a time where most votes appear to be going to big articles that have attracted the most votes for any Kotlin tagged item ever. The response to moderators immediately after receiving their notification has gone unanswered. A small community that follows their tag closely can't vote for answers they like if they happen to come from the same person? it seems like they have to intentionally withhold voting so it isn't reversed, or do you have to vote for content you don't like to mix it up? Jan 28, 2016 at 11:43
  • Or is there something in the pattern I don't see (I can't see it at all, I only see reputation gain across all my posts, not from who). Jan 28, 2016 at 11:43
  • Tried to do some analysis (added that to bottom of question), but fail to see what stands out other than what is expected to stand out. Jan 28, 2016 at 14:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .