62

tl;dr: How many flags do we actually get a day on Stack Overflow?


The current, 2019 elections say (emphasis mine):

Due to the size of Stack Overflow (averaging nearly 2,000 flags per day!), moderation can be a significant responsibility - if you can't dedicate at least 30 minutes each day to help shoulder the load, please defer to those who can.

The linked question is dated August 2011 and says (emphasis mine):

in the last 30 days as of the time I am writing this, there have been 26,710 flags. That is 890 flags per day on average.


I initially wanted to look back through the past elections to see at what rate this number was increasing, but was thwarted by the election page text having been retrospectively made to all be the same - from the first 2011 elections to the current, 2019 elections, even though question linked to in the text

...and cannot have been suspended during the past year.

is to "Let's disallow nominations from people who've been suspended in the past year" (a proposal) dated February 2016 and so can't have been in effect (publicly, at least) in the earlier elections.


Luckily, The Internet Archive exists so I was able to do some investigation, and found the following:

  • The first mirror of the second 2011 moderator elections (captured December 2011 - link) has no reference to the number of flags per day.

  • The first mirror of the 2012 moderator elections (captured June 2012 - link) says

    Due to the size of Stack Overflow (with an average of over 1,000 flags per day!)

  • The first mirror of the 2013 moderator elections (captured March 2013 - link) says

    Due to the size of Stack Overflow (averaging nearly 2,000 flags per day!)

Since then the mirrors of the successive elections have said the same thing (about the number of flags per day, at least).

Looking at meta questions like How many new users does Stack Overflow get per day?, I found queries on the Stack Exchange Data Explorer like: New accounts per day (moving average) which shows that the number of new users since 2013 hasn't exactly stayed about the same (I set the moving average to be over 180 days to smooth out anomalies):

New accounts per day, 180-day moving average

I fully get that lots of users register, ask one question, then their account is neglected afterwards, but I would have thought that the number of active users, and therefore the number of flags per day, has also increased since 2013?

Is there some fault in my logic, and/or is the "nearly 2,000 flags per day" statistic wrong?

25
  • 2
    I expect a lot of the flags to be close flags, which are not flags requiring moderator action. If that statistic is to be updated, could it be specified if it's nearly 2000 flags requiring moderator intervention or nearly 2000, but 70% of that is close flags by <2K users so actually more like 600 flags per day
    – Erik A
    Commented Mar 5, 2019 at 8:43
  • 26
    Insider info: we've got 16,000 flags in the past week ... so that is around 2.3k~ ... it goes up to 3k on Wednesdays and slips to 700~800 on the weekends.. It is still accurate, and would perhaps be shifted to 3k, once we cross getting 2.5k~ per day on an average.
    – Bhargav Rao Mod
    Commented Mar 5, 2019 at 8:49
  • 1
    Yes, I hadn't considered close flags. If the close flags are included in the statistics, then it'd be odd to include them there. When I read it, I had assumed that to be actual moderator flags which 'normal' users will never see. Clarification would be nice.
    – Wai Ha Lee
    Commented Mar 5, 2019 at 8:49
  • 1
    NAA, VLQ are moderator flags too! Spam, R/A aren't. Moderator flags are the custom flags, NAA/VLQ, comment flags and the numerous automatic flags.
    – Bhargav Rao Mod
    Commented Mar 5, 2019 at 8:53
  • 2
    @WaiHaLee just to note - the previous versions of the election pages are available - see: stackoverflow.com/posts/8041931/revisions
    – Jon Clements Mod
    Commented Mar 5, 2019 at 8:59
  • 2
    @WaiHaLee if it wasn't for the fact I can see a link for the history on it and that I knew from a while back it's in SEDE - I wouldn't have guessed that either :)
    – Jon Clements Mod
    Commented Mar 5, 2019 at 9:11
  • 3
    @WaiHaLee: we are 10 hours into the UTC day and moderators have directly handled more than 400 flags. We'll easily have handled over 1000 by the end of it. Not all flags we handle show up in that count; if I see a spam flag in the queue and agree by flagging it a spam too, it is Community that then deletes the post and gains a +1 on the flags handled counter, not me.
    – Martijn Pieters Mod
    Commented Mar 5, 2019 at 10:05
  • 2
    @WaiHaLee: also, comment flags on a post count as '1' flag handled. Someone flagging 25 comments to delete is counted the same as a single 'please delete the comments' custom moderator flag in that respect. All in all, there are a lot of different ways to slice and dice the flag stats. The 2k count is a decent middle ground.
    – Martijn Pieters Mod
    Commented Mar 5, 2019 at 10:08
  • 15
    different perspective: in the past 30 days, the top 5 most prolific flag-handling moderators handled 11.3k, 6k, 3.6k, 3.1k and 2k flags, respectively. But not all flags are equal, cleaning up an account that has plagiarised about 60% of their 5000 answers takes a lot more time than cleaning up 'thank you, it works!' comments.
    – Martijn Pieters Mod
    Commented Mar 5, 2019 at 10:13
  • 1
    And finally: chat flags are nowhere to be found in any of these statistics. Some moderators do a lot of chat moderating, but we have no stats for that.
    – Martijn Pieters Mod
    Commented Mar 5, 2019 at 10:17
  • 2
    @MartijnPieters I'd be very interested to see breakdowns based on "work hours" per day instead of flags per day. As you said, some flags require a couple seconds of deliberation while some can involve half an hour or more of sleuthing. Are any such stats recorded?
    – Magisch
    Commented Mar 5, 2019 at 10:53
  • 3
    @MartijnPieters Was there really someone plagiarizing 60% of their 5000 answers?
    – justhalf
    Commented Mar 5, 2019 at 12:10
  • 5
    @justhalf those numbers were made up for illustration, but I’ve certainly cleaned out plagiarism counting in the hundreds of posts for a single account. I don’t keep stats however.
    – Martijn Pieters Mod
    Commented Mar 5, 2019 at 13:28
  • 4
    @WaiHaLee, it is still nearly 2000 if you round down. ;p ... but yeah, it does makes sense to put it at "more than 2000".
    – Bhargav Rao Mod
    Commented Mar 5, 2019 at 17:59
  • 1
    @smileycreations15 - that's because they're all the same post.
    – Wai Ha Lee
    Commented Mar 7, 2019 at 12:50

1 Answer 1

19

Let's have a look at 2018: a year in moderation...

                 Action                  Moderators Community¹
---------------------------------------- ---------- ----------
Questions flagged                             1,453    592,454
Question flags handled                       38,092    555,815
Comments flagged                              1,634    345,828
Comment flags handled                       211,510    135,952
Answers flagged                               3,504    462,254
Answer flags handled                        219,741    246,017

If we add up all the flags raised, we get 1,408,127 flags - or an average of about 3,857 flags per day. Clearly that's somewhat more than 2,000...

However, that includes flags raised by moderators, as well as flags that were handled by the system or by members of the community - and there are a lot of those! So let's add up the flags handled by moderators instead:

38,092+211,510+219,741=469,343

...or about 1285 per day. And that's significantly lower than 2,000!

But, it still doesn't give us the full picture. Some flags age away if not handled; others must be handled - if not handled by the community via /review, then by moderators. And the volume of flags varies considerably over time - both by day of the week, and by time of year.

So I looked up the number of flags excluding close flags raised per day over the course of 2018, and calculated the first quartile (969), median (1,646), and 3rd quartile (1808).

With all this in mind, I think it's probably best to say that the moderator team should be prepared to handle nearly 2,000 flags per day... Even though the average number of flags raised is much higher, and the average number of flags that must be handled by the moderators is much lower.

To satisfy your curiosity for how this has changed over time, here's a funky-looking box-plot showing the distribution for each year since 2010:

Footnote

¹ "Community" here refers both to the membership of Stack Overflow without diamonds next to their names, and to the automated systems otherwise known as user #-1.

2
  • 1
    Can you please explain why older election pages were retroactively edited?
    – gparyani
    Commented Mar 10, 2019 at 5:12
  • 4
    @gparyani - they are all a single post so edits apply to all elections - see comments above: 1, 2.
    – Wai Ha Lee
    Commented Mar 10, 2019 at 10:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .