-6

I won't argue on whether we should or should not discuss politics on SO, but I'm wondering what the effects of Time to take a stand were.

I'm looking for any metrics that indicate the community might have been divided like:

  • Flags: not constructive, offensive/rude, custom; both declined and helpful.
  • Serial downvotes
  • Participation of users in main SO (e.g. flagging, posting, editing, voting, visiting SO)
  • Whatever you consider relevant.

Of course some of the above might not provide any insight; they are simply examples of things that might have to be checked.

Please share mods! Or perhaps wait until the whole situation has settled down.

21
  • (e.g. permanently locked with comments disabled) They've explicitly said that they won't do this.
    – Servy
    Commented Feb 3, 2017 at 16:16
  • The entire second half of the question is dedicated to a statement that they're leaving it.
    – Servy
    Commented Feb 3, 2017 at 16:17
  • 1
    @Servy Yeah but it's on hold and my guess is that eventually it will be locked so.. Doesn't really matter though, as far as this Q is concerned.
    – user
    Commented Feb 3, 2017 at 16:18
  • 1
    Considering that they have also been reopening it I would not count on that.
    – Joe W
    Commented Feb 3, 2017 at 16:20
  • It's been closed dozens of times (I'm too lazy to count, but at a glance it looks like it's over 50 at this point). SE specifically came out and said that they will continue to keep it open and unlocked. They might go back on that statement (although I doubt it) but they were quite explicit about it.
    – Servy
    Commented Feb 3, 2017 at 16:21
  • 7
    community might have been divided you doubt that?
    – rene
    Commented Feb 3, 2017 at 16:21
  • @rene No not at all. Just wanted to keep it neutral. For example, seeing all those downvotes i wonder if the title is not neutral enough. Feels like they disagree with discussing politics here. Or perhaps his is not a useful Q (like who would care about whether it divided us or no, right?:P).
    – user
    Commented Feb 3, 2017 at 16:22
  • Let's assume the outcome is 42/42. What does that tell us? Also: what is serial down votes, I mean how is it relevant here?
    – rene
    Commented Feb 3, 2017 at 16:24
  • I can imagine on rare occasions arguments getting heated to the point one party decides to "show that insolent person that dares being a disagreeing racist!!1".. just a guess though.
    – user
    Commented Feb 3, 2017 at 16:26
  • 17
    Probably a bit too early for a post-mortem here.
    – Shog9
    Commented Feb 3, 2017 at 16:29
  • 1
    Sure, serial downovtes did happen but it is still unclear how having all these numbers helps in going forward, assuming SE Devs/CM wants to dig those up. What are going to do: laugh about them? Look how silly we have been doing over the weekend or start another bashing looking at these numbers it is even worse then I expected: FIRE ALL CM's. I mean, what is the point of this question?
    – rene
    Commented Feb 3, 2017 at 16:31
  • 1
    @rene "what is the point of this question?" - Damage assessment; ensure we avoid similar things or in case it didn't have such a negative effect.. just ignore it. Not sure why you exaggerate that much though... feels strange.
    – user
    Commented Feb 3, 2017 at 16:35
  • 2
    @Shog9 You mean it will happen at some point in the future, or that you do not consider it worthy of checking?
    – user
    Commented Feb 3, 2017 at 16:36
  • 2
    I mean, generally you'd want to do a thorough examination of a situation once it's conclusively over, but while the details are still fresh in your mind.
    – Shog9
    Commented Feb 3, 2017 at 17:19
  • 5
    Not that that narrows down the time table any. I don't think we're in much danger of forgetting about this any time soon.
    – Cody Gray Mod
    Commented Feb 3, 2017 at 17:50

1 Answer 1

19

So... I gotta admit up-front, this isn't the sort of analysis that's gonna make me feel good at the end of the day: we don't often bring up politics on meta, and when we do it's definitely not for the purpose of generating harmony and togetherness among users of the site. If we're bringing up some topical governance issue here, it's because we believe - right or wrong - that it stands to harm the ability of this site and its users to function effectively, and thus it is worth a bit of disharmony to spread the word.

And so, we're probably gonna keep doing this on rare occasions regardless of the outcome. But... That's no reason not to look at the outcome! It's sorta meaningless to say, "we'll do this regardless of cost" if you don't know the cost: y'aint really sticking your neck out if you secretly believe there's nothing to lose. So, without further ado, here's what I found...

Who I looked at

The first critical question here is: which people am I looking at? There are millions of people visiting Stack Overflow; most of 'em never saw the "taking a stand" post, so measuring overall trends isn't likely to produce meaningful results. The view counter shows not quite 64K views; the actual number of people who clicked through to the post is probably a bit more than that. But again, the section of those people who have any meaningful activity on Stack Overflow is likely to be small; the section who actually read the post or the ensuing discussion and were meaningfully affected by it are almost certainly drowned out by an immense number of people who clicked and bounced and no longer even recall that this ever happened.

What we're really interested in here are the small subsection of users who were both strongly affected by the post and engaged in Stack Overflow proper. To that end, I decided to look at a smaller and much more well-defined group: those users who interacted in some way with the 'Stand post:

  • Folks who voted on the post
  • Folks who posted answers to the post
  • Folks who voted on answers to the post
  • Folks who commented on the post
  • Folks who commented on answers to the post
  • Folks who voted on comments to the post
  • Folks who voted on comments to answers to the post

This gave me a list of 3439 users, which seems like a reasonable sample. Of those users, 3401 had joined Stack Overflow at least 30 days prior to the 'Stand post's creation, giving me an opportunity to compare their activity before and after.

Those who left

First, it's important to note that 10 of the users who participated in ways described above have since deleted their accounts. The rest of the data here excludes them for my convenience; while it's not appropriate to assume that all of them deleted their accounts because of the post, it's probably safe to assume that they were not happy being members of the site anymore at some point after their initial participation in the thread.

One user disassociated their answer, but did not remove their account. Their activity is counted below.

Signs of controversy: interactions between participants on main

Goal here is to look for an up-tick in rude/abusive flagging between participants in the discussion on the main site. Also, any series of downvotes that might indicate an effort to "punish" folks who the voter disagreed with (again, on the main site). I'll be looking at the activity 30 days prior to and 30 days after the date the Take A Stand question was posted.

Offended Flagging

This counts all rude/abusive flags raised by a participant in the discussion on a post or comment authored by another participant. In the 30 days prior to the post, 14 such flags were raised; in the 30 days that followed 8 were raised.

Targeted Downvoting

This counts serial downvoting by a participant in the discussion on posts authored by another participant. In the 30 days prior to the post, 4 such downvote series were detected; in the 30 days that followed 10 were detected.

What they did: before and after

Goal here is to gauge participation: was there a noticeable drop-off in activity by folks who were involved in the discussion? I'll again be contrasting the 30 days prior to with the 30 days following the post, and as with the checks in the previous section this is liable to be heavily affected by the overall trends of participation on the site: there's generally an increase in activity across the board as we move away from the holidays and toward spring / end-of-semester activity...

Therefore, any decrease by members of this particular group is interesting, but even an increase that's less than the overall increase is worth considering.
To that end, I'm representing the rest of these findings as a percentage increase from the month prior to the month following the discussion - and including the site-wide increase for comparison. Since you have to have at least 15 rep to vote on meta (and thus be considered a "participant" for the purpose of this answer), I'm also including the percentage increase for all users with >= 15 rep in these results.

visited the site

This counts the total number of days visited by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 5.877919006798            7.880587852816             5.996004455345  

posting

This counts the total number of questions and answers created by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 -1.105074700581           5.919611320585             5.375062335412  

voting

This counts the total number of up- and down-votes created by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 0.785248545186            5.891314530941             5.916338381861  

editing

This counts the total number of edits (to anyone's posts) made by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 -5.301593989187           3.776750206507             6.548879708592  

flagging

This counts the total number of flags (and close votes, and reopen votes) made by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 8.558332496905            11.11673803198             11.413804242489 

review

This counts the total number of reviews made by users in each group for the period.

 % Increase: participants  % Increase: 15-rep+ users  % Increase: all 
 ------------------------  -------------------------  --------------- 
 16.78161236315            18.973242718737            18.773005834371 

Conclusions

I suspected going in that any effect we might observe here would be pretty subtle; most of the folks who participated are still around and have long ago moved on to other concerns. But there does appear to have been an effect: participation was lower than might otherwise be expected for pretty much every activity following the discussion, and in two categories actually decreased significantly from the month prior.

There's no denying that such an event can be fairly disruptive for those involved; the tone and volume of comments attests to that if nothing else. Fortunately, the controversy did not appear to inspire anything approaching outright war on the site: cross-flagging and revenge-voting remained rare enough to be discounted following the discussion. While we'll always strive to improve the way we conduct such discussions, it's still good to see that boundaries and some amount of mutual respect isn't a strange notion to folks in this community.

Finally, thank you for your patience. I've had this post half-written for a good many months now, but struggled to find the time to finish it. Hopefully you're still interested...

11
  • "there's generally an increase in activity across the board as we move away from the holidays and twoard spring / end-of-semester activity... Therefore, any decrease by members of this particular group is interesting" please note also that there are people that are actually counter-cyclical, so maybe a closer look may be worth.
    – Braiam
    Commented Nov 20, 2017 at 22:30
  • 1
    Yes, it's still entirely possible that the group of people examined here simply skews toward folks who were already inclined to reduce participation during this period; a rigorous examination would attempt to match a baseline by other attributes of those involved (past participation, specific rep levels, etc.); even a lazy approach (picking the median rep level of the sample group instead of the minimum of 15) eliminates many of the differences. Given the time I had to spend on this, I felt that erring on the side of showing false-positives rather than potentially erasing evidence was ok.
    – Shog9
    Commented Nov 20, 2017 at 22:36
  • 1
    @Shog9 Are you able to tell if participation rebounded after those 30 days? Or even better, how long did the reduced participation last for the participants?
    – Rob Mod
    Commented Nov 20, 2017 at 23:40
  • 1
    I haven't looked beyond about 6 months, but there's no rebound in that time; if anything, activity falls off more. @Rob
    – Shog9
    Commented Nov 20, 2017 at 23:52
  • for the sake of completeness I would extend the list in "Who I looked at" with mentions of those who flagged and edited (these folks almost certainly also voted but listing them explicitly wouldn't hurt)
    – gnat
    Commented Nov 21, 2017 at 8:27
  • Adds about 30 users, increases participation rates by a tenth of a percent or so in all categories @gnat
    – Shog9
    Commented Nov 21, 2017 at 16:30
  • that's about as little impact than I expected - vast majority of flaggers / editors should be already covered by other categories
    – gnat
    Commented Nov 21, 2017 at 16:35
  • With lots of background noise I wouldn't be surprised if we can't draw a clear conclusion. Anyway. Those deleting their accounts should be counted as well, since they clearly do not contribute anymore (if you think the reason they quit is unclear, then there are about 3 users if i recall, that stated clearly why they quit, and should be included). Also note that the small reduction in numbers, could be much higher when it comes to quality. I'd be more concerned about 1 extremely experienced user quitting than 3 newbies contributing with edits etc.
    – user
    Commented Nov 24, 2017 at 9:32
  • For example I remember there was an issue with suggested edits pilling up (17 June 2017), which you attributed to the layout. Then there were mod elections (perhaps to address edit-pilling?). But maybe it's not just the layout. Also, 30-days is not enough. I remember waiting for a few months before stopping contributions completely; was expecting a "we shouldn't have used the power the community gave us for political goals". But you don't feel that way.
    – user
    Commented Nov 24, 2017 at 10:12
  • You believe that "it stands to harm the ability of this site and its users to function effectively" and that you did the right thing. I don't think it harmed the site. I think it harmed one of your employees (that might have very well been just an irrational fear on his part, but i don't know all the details). Regardless, you might want to check political demographics on SO. Feel free to tag me if you do. The reason you didn't get a 10 times greater backlash with Stand was probably a left-leaning domination.
    – user
    Commented Nov 24, 2017 at 10:19
  • Some users might edit their own post dozens of times within a couple of days. This would be background-noise in your analysis. You should separate own-posts edits from edits. I'm pretty sure we would see a much more profound effect of Stand.
    – user
    Commented Dec 3, 2017 at 11:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .