85

As we say goodbye to the old year and welcome the new one, we have a tradition of sharing moderation stats for the preceding calendar year.

As most of you here are aware, sites on the Stack Exchange network are moderated somewhat differently to other sites on the web:

We designed the Stack Exchange network engine to be mostly self-regulating, in that we amortize the overall moderation cost of the system across thousands of teeny-tiny slices of effort contributed by regular, everyday users.
-- A Theory of Moderation

That doesn't eliminate the need for having moderators altogether, but it does mean that the bulk of moderation work is carried out by regular folks. Every bit of time and effort y'all contribute to the site gives you access to more privileges you can use to help in this effort, all of which produce a cumulative effect that makes a big difference.

So as we say goodbye to 2021, let us look back at what we accomplished as a community... by looking at some exciting stats. Below is a breakdown of moderation actions performed on Stack Overflow over the past 12 months:

Action Moderators Community¹
Users suspended² 1,130 2,283
Users destroyed³ 17,851 0
Users deleted 4,826 0
Users contacted 4,793 0
User suspensions lifted early 29 0
User review-bans lifted early 86 0
User banned from review 280 1,658
Tasks reviewed⁴: Triage queue 9 434,241
Tasks reviewed⁴: Suggested Edit queue 2,008 529,477
Tasks reviewed⁴: Reopen Vote queue 46 122,593
Tasks reviewed⁴: Low Quality Posts queue 120 284,004
Tasks reviewed⁴: Late Answer queue 14 244,962
Tasks reviewed⁴: Helper queue 0 1,974
Tasks reviewed⁴: First questions queue 120 193,593
Tasks reviewed⁴: First Post queue 12 447,614
Tasks reviewed⁴: First answers queue 16 142,421
Tasks reviewed⁴: Close Votes queue 125 213,104
Tags merged 180 0
Tag synonyms proposed 102 160
Tag synonyms created 179 38
Tag highlight language set 43 0
Revisions redacted 833 0
Questions unprotected 3 72
Questions reopened 715 9,328
Questions protected 170 4,492
Questions migrated 402 1,282
Questions merged 98 0
Questions flagged⁵ 2,973 359,525
Questions closed 22,640 432,352
Question flags handled⁵ 37,327 324,808
Posts unlocked 144 645
Posts undeleted 4,597 67,159
Posts locked 544 13,139
Posts deleted⁶ 114,668 1,417,149
Posts bumped 0 32,928
Escalations to the Community Manager team 816 0
Comments undeleted 658 0
Comments flagged 385 451,292
Comments deleted⁷ 403,458 867,568
Comment flags handled 266,989 184,705
Bounties canceled 147 0
Answers flagged 4,388 336,369
Answer flags handled 253,134 87,594
All comments on a post moved to chat 345 0

Footnotes

¹ "Community" here refers both to the membership of Stack Overflow without diamonds next to their names, and to the automated systems otherwise known as user #-1.

² The system will suspend users under three circumstances: when a user is recreated after being previously suspended, when a user is recreated after being destroyed for spam or abuse, and when a network-wide suspension is in effect on an account.

³ A "destroyed" user is deleted along with all that they had posted: questions, answers, comments. Generally used as an expedient way of getting rid of spam.

⁴ This counts every review that was submitted (not skipped) - so the 2 suggested edits reviews needed to approve an edit would count as 2, the goal being to indicate the frequency of moderation actions. This also applies to flags, etc.

⁵ Includes close flags (but not close or reopen votes).

⁶ This ignores numerous deletions that happen automatically in response to some other action.

⁷ This includes comments deleted by their own authors (which also account for some number of handled comment flags).

Further reading:

Wishing everyone a happy 2022! ^_^

35
  • 24
    Updated comparison spreadsheet: Stack Overflow Yearly Stats
    – Samuel Liew Mod
    Commented Jan 12, 2022 at 14:49
  • 27
    @SamuelLiew Wow, that 283.6% increase on users destroyed really shows the scale of the spam plague. Commented Jan 12, 2022 at 14:58
  • 22
    Not sure how I feel about the decline in almost every action, compared to last year
    – Lino
    Commented Jan 12, 2022 at 14:58
  • 2
    @Lino Some of the decrease in triage and such could be due to the first post and answer queues being created. But even then, its still a decrease.
    – code11
    Commented Jan 12, 2022 at 15:24
  • 1
    Can you expand upon "Every bit of time and effort y'all contribute to the site gives you access to more privileges ...". I thought privileges depend on reputation and that, in turn, depends on writing questions and answers. (A few reputation points come from ones first few edits.)
    – AdrianHHH
    Commented Jan 12, 2022 at 18:07
  • 2
    @AdrianHHH Time and effort is put into writing answers, questions, edits, etc. which in turn give reputation which in turn gives privileges, seems pretty clear. Commented Jan 12, 2022 at 18:26
  • 1
    @Nick The topic is moderation, the text I quoted implies that time spent doing moderation contributes to getting more privileges.
    – AdrianHHH
    Commented Jan 12, 2022 at 18:33
  • 1
    @AdrianHHH The topic is indeed moderation, but I read that point as a thank you, as in, "you've put in all this effort, you've unlocked the privileges, and you've used them, good job". Commented Jan 12, 2022 at 18:36
  • 1
    I wonder how much did the reopening of mobotics contribute to the increase in number of destroyed/deleted users... Commented Jan 12, 2022 at 21:20
  • 2
    we already have a process for auditing moderator actions.
    – Kevin B
    Commented Jan 12, 2022 at 22:25
  • 1
    I love statistics and data analytics! However, I'm a little disappointed at how few of my personal flags were handled compared to total site flags that were handled: while 702,863 of the 703,255 question and answer flags were handled (99.9%), only 126 of my 159 flags were handled (79.2%). 23 of the remaining unhandled flags were "aged away" (14.5%). I'm counting "helpful", "declined" and "disputed" among my flags as "handled", too. What's with that difference in numbers? I'd at least expect my flags to approach 90% handled. Commented Jan 12, 2022 at 22:25
  • 6
    well, it depends on the kinds of flags, @SylvesterKruin. close flags for example likely age away far more often than any other kind of flag.
    – Kevin B
    Commented Jan 12, 2022 at 22:28
  • 1
    @SylvesterKruin See my "answer" below (this is "meta", of course, so it's just "discussion" rather than an "answer"), but my theory is that Flags are being considered "Handled" for the sake of this report when they are placed in the Review queue. But as for how many reviews get through the review queue, as you've noticed, I think it's much lower. Commented Jan 12, 2022 at 23:50
  • 3
    It's the past calendar year, @holydragon, so 1/1/2021 to 31/12/2021.
    – JNat StaffMod
    Commented Jan 13, 2022 at 11:43
  • 3
    @AlexGuteniev We primarily clear fraudulent bounties (extremely infrequent), and periodically bounties on off-topic posts that weren't closed prior to the bounty being placed. The latter does happen relatively regularly; 147 cancelled bounties means one every few days, and I'll go ahead and argue the majority of them are on off-topic posts. (letting them expire instead does happen, but it really depends on the mod, remaining duration, activity, etc.) Commented Jan 14, 2022 at 22:59

3 Answers 3

21

A "grand total" might be a nice addition, but it's easy to copy/paste into Excel and arrive at:

1,147,403 total Moderator actions in 2021. Wow! And a big "Thank You" to the Mod team for your hard work!

4
  • 3
    But various actions aren't really comparable. Handling a suggested edit could be about a minute of work (or even less), but something like a suspension indicates a lot more work potentially (imagine having to investigate plagiarism across dozens of posts).
    – Laurel
    Commented Jan 12, 2022 at 22:31
  • 5
    @Laurel Fair enough - Like I said "might be nice" ;-). But regardless, my point is that's a lot of action taken by the Mod team, big or small, those actions add up and are appreciated. Commented Jan 12, 2022 at 22:34
  • And we have 23,202 users who got in trouble last year (that includes users suspended, destroyed, deleted, and banned). That's about 63 users getting in trouble per day on average, or about 3 per hour :-|. On the bright side, that's 11,933 users less than 2020, which had ~96 users that got in trouble per day, or ~4 users an hour! Or is that a good thing? I guess it depends on how you look at it.... Commented Jan 14, 2022 at 22:18
  • @SylvesterKruin A matter of perspective, 'tis true. And the spam problem (which often precedes deleted/suspected/destroyed accounts) seemed to get much worse in the last few weeks of the year. Again, thankful for the moderation actions which cleaned these up, but yes, wish they weren't necessary in the first place. Commented Jan 14, 2022 at 23:08
17

There were 2,973 questions, 385 comments and 4,388 answers flagged by moderators. I always considered moderators as flag handlers, not generators.
In what situations would moderator raise a flag? And who handles them?

5
  • 29
    We raise binding spam or R/A flags on posts, which immediately nukes the post. We can also flag comments as unfriendly so they become a part of the "paper trail". As those flags are binding, no one handles them. Those are the bulk of it. Additionally, you'd occasionally come across something you don't have the time to deal with at the moment, so you custom flag it to handle it later or have another mod pick it up.
    – Baum mit Augen Mod
    Commented Jan 13, 2022 at 8:59
  • 3
    In short, 'raised' flags by mods are also 'handled' flags by mods. Mods browse the site, too, not just the moderation tools pages :-) They can (and should) flag stuff the come across naturally (or unnaturally, which still needs flags/handling).
    – TylerH
    Commented Jan 13, 2022 at 14:38
  • @TylerH I think that they need to raise a flag to be able to elevate issues to CMs too.
    – Braiam
    Commented Jan 20, 2022 at 17:24
  • @Braiam That's certainly one vector. They also have chat, the Moderator Team, and probably some other methods, too.
    – TylerH
    Commented Jan 20, 2022 at 18:05
  • @Braiam No, that works on a different system since '16.
    – Mast
    Commented Jan 28, 2022 at 20:43
3

Can someone provide more detail on the "Question Flags Handled" and "Answer Flags Handled" stats. They show right at 100% of the total flags were handled.


Edit/Update: Zanna, a mod on Ask Ubuntu, provided the information here that explains this stat:

A flag to close a post is considered to be handled if it is marked helpful. This happens if the post receives at least one close vote, regardless of the outcome of the close review.

So yes, it's entirely feasible that such a large percentage of Close flags result in at least one vote in the queue (either to Close or "Leave Open"), and as a result were "Helpful" or "Declined", and as a result of that, "Handled".

At the same time, it's entirely normal (and yet unfortunate) for a large number of those to never receive enough votes to make it all the way through the queue before the votes expire. Not an ideal situation, to be sure, but it at least explains the "Handled" stat.


We know that a number of these (at least questions) "aged out" (invalidated) of the review queue they were placed in as a result of the flag, right? Does "Invalidated" count as "handled"? Should it?

I know that before I hit 3k and could cast Close votes, I flagged a number of questions that "aged out" of review. I'd assume this is the case for a lot of users.

I adapted this SEDE query from one that was referenced in a chat on Ask Ubuntu with one of the mods. It checks for Close Vote reviews that were invalidated after 4 days or 14 days, with the assumption that these were due to expired votes per this answer. It hit the max of 50,000 invalidated reviews in 2021. This would seem to indicate to me that a large number of Close flags that get moved to the Close Vote Review queue are expiring, rather than being "handled."

Not that the Mods (and community) aren't doing a great job of curation and moderation, but is it fair to say that we really are at nearly 100% of question flags being handled? It seems to me (based on 50k expired CV reviews) that there's still a lot of room for improvement that isn't indicated by a "near 100%" handled stat.

6
  • 1
    Flags can be carried over from year to year. Some of the flags raised in 2020 would have been handled in 2021. Like wise some raised in 2021 will be handled this year. Though the numbers aren't at 100%. There were 362135 questions flags handled and 362498 raised (that's 99.89%) and for answers 340757 and 339728 respectively (99.7%). It does, however, show that the mods are (more or less) keeping up with the volume.
    – Thom A
    Commented Jan 12, 2022 at 15:24
  • @Larnu Yes, I know that the numbers aren't exact due to some flags and their handling falling outside the calendar year. That's negligible. My point is that "Mods" only handled about 10% of the questions (but about 75% of answer flags). The other 90% of the question flags were handled by the Community, either through review queues or automatic handling. However, we know that questions "age out" of community review after 2 weeks and are marked "Invalidated". It seems that those that age out are being included as "Handled". Commented Jan 12, 2022 at 16:31
  • 4
    @NotTheDr01ds "In need of moderator intervention" flags don't age-out. My understanding is that NAA and VLQ flags also don't age-out. The NAA and VLQ lists, usually, don't get long enough to verify that, but the VLQ list is fairly long at the moment, which gives the opportunity to see they don't age-out after at least several/many days. Spam and R/A flags can age out after 4 days, but are never active for anywhere remotely close to that (i.e. usually minutes). Close-flags, like close-votes, do age out but those flags are not presented to moderators. They go to the CV review queue.
    – Makyen Mod
    Commented Jan 12, 2022 at 17:05
  • Thanks @Makyen - That's good info on the process flow. And yes, it's primarily the Close flags that I think are getting invalidated, but according to the stats above, they would appear to be "handled" by the Community review at nearly a 100% rate. It's mainly that particular stat that I'm questioning. I think there's a SEDE query that might get me pretty close to the actual number that age out. Zanna (mod on Ask Ubuntu) brought it up a while back in chat. Let me see if I can adapt it. Commented Jan 12, 2022 at 19:01
  • @Makyen Added the SEDE query. Unless the assumptions in my query are faulty (entirely possible, but again, based on one of the AU mods) it appears that more than 50,000 questions aged out of Close Vote review in 2021. Commented Jan 12, 2022 at 19:26
  • And just a small correction - I found the original query, and it didn't come from Zanna, but from another person who was in the chat with Zanna. Commented Jan 12, 2022 at 19:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .