Skip to main content
7 of 8
Commonmark migration

Stack Overflow is not yet a vast wasteland: a history of moderator tooling

Warning: this is long, rambling and extremely boring. I'm writing it because I tend to get a lot of questions regarding the rationale for changes to the moderator tooling on SO, and I'm hoping to have something to point to next time. If you already know all there is to know about this - or just don't care - then go mine some Unicoins instead.

Moderating a site with as many users and as much daily traffic as Stack Overflow is... Challenging. Just ask any of the hardy souls who've volunteered to do it. Conventional wisdom holds that when a community grows beyond a certain size, self-moderation becomes effectively impossible: either the lion's share of moderation must be performed by a group of dedicated moderators, the community itself fragments into smaller, more cohesive groups, or the group decides to close the doors and stop accepting new members entirely outside of some controlled approval process. Clay Shirky wrote about this years ago:

And, finally, you have to find a way to spare the group from scale. Scale alone kills conversations, because conversations require dense two-way conversations. In conversational contexts, Metcalfe's law is a drag. The fact that the amount of two-way connections you have to support goes up with the square of the users means that the density of conversation falls off very fast as the system scales even a little bit. You have to have some way to let users hang onto the less is more pattern, in order to keep associated with one another.

By rights, Stack Overflow should have died already, turned into an irredeemable cesspool by a combination of outsider influx and insider burnout. You can argue (and many do) that we're headed that way - but we've been headed that way since day one. The best we can hope for is a stable orbit, forever falling but never crashing. I believe there are two major reasons why Stack Overflow has managed to scale far beyond the expected limits of a group:

  1. Conversations not required. When a question is asked on a traditional forum, answering it often demands some amount of participation from at least a portion of the community. Details are fleshed out, the problem is clarified, solutions are proposed and debated, others with similar problems chime in with their experiences, tangential points are made, and eventually - anywhere from hours to months later - the conversation dies out. It's a very social, very natural way to interact. And it suffers mightily from the problem that Shirky talked about: all that back-and-forth and associated latency kills any hope of scale. On Stack Overflow, we close or delete questions that can't be answered straight away - it's not very sociable, but it scales wonderfully by effectively enabling a vast, human-powered computational grid.

  2. Tools that allow decoupling moderation from communication without separating moderators and users. While Stack Overflow does have a powerful "moderator class" elected by the community, a fairly large portion of the actual moderation is performed by individual members of the site, those who've participated enough to demonstrate sufficient familiarity with the community. While this has been a fundamental part of the system for a very long time, I didn't fully appreciate how it relates to scale until I started working with very small Stack Exchange sites: the proportional cost of moderation is much higher, even though the total volume of work is lower. Many hands make (relatively) light work... As long as the system puts tools in those hands.

Number 1 has been discussed endlessly here and elsewhere; if you're a Meta regular, you're more than familiar with how it works. Number 2 is still something that seems to prompt a lot of questions and confusion though, so that's what I'm here to talk about today.

By the community, of the community

Stack Overflow - and by extension, Stack Exchange - was founded with a very simple principle:

Stack Overflow is you.

This is the scary part, the great leap of faith that Stack Overflow is predicated on: trusting your fellow programmers. The programmers who choose to participate in Stack Overflow are the “secret sauce” that makes it work. You are the reason I continue to believe in developer community as the greatest source of learning and growth. You are the reason I continue to get so many positive emails and testimonials about Stack Overflow. I can’t take credit for that. But you can.

The essential nature of the software here is that of a force multiplier, enabling the group of skilled individuals to work together to tackle problems larger than they would be able to without it. It strives to afford and encourage uses which align with our goals (creating a knowledge base of questions and answers) while discouraging those that do not (extended discussion, socialization). But ultimately, it is up to those using it - YOU - to decide what it is used for.

This applies to moderation as well to the actual creation of content. Unlike many similar systems, Stack Exchange moderation does not depend entirely or even primarily on a small group of official moderators. Rather, participation within the system itself grants you the ability to moderate it:

We designed the Stack Exchange network engine to be mostly self-regulating, in that we amortize the overall moderation cost of the system across thousands of teeny-tiny slices of effort contributed by regular, everyday users.

That was a brilliant concept, but after a couple years it was clear that the implementation was flawed: debris were accumulating. Not-answers, blatantly off-topic questions, unintelligible posts of every kind... Small bits of decay, insignificant alone but harder and harder to ignore as they piled up. Virtual mounds of rotting, festering excrement, clogging search results and discouraging participants.

Streets of NYC before sanitation reform
(Source: Harper's Weekly via Collector's Weekly)

User-moderators turned supplicants: the creation of a flagging culture

Something interesting happened back at the start of 2011:

Another change we’ve instituted, based on the popular Newgrounds flash game portal, is the concept of “flag reliability”. If a particular user keeps moderator flagging for reasons that we consider invalid, their flag weight decreases. And for those users who continually flag reliably, their flag weight increases.

This change had a dramatic effect on how Stack Overflow was moderated. Prior to this change, flagging was mostly used for very serious situations: spam, blatantly-offensive content, and whatever problems a strongly-motivated individual might find worth typing up a note about (perhaps in verse). After the change, folks would flag for any problem they couldn't easily solve themselves. Motivated by a tantalizingly-visible "flag weight" score, many were even motivated to seek out problems, just so they could flag them. And the amount of work asked of elected moderators skyrocketed:

moderator-handled flags by month
Moderator-handled flags by month

Some of the moderators you elected rose to the challenge, using their experience and quick-thinking to handily evaluate and dispatch hundreds of posts a day, but the strain was immense: 2011 was the first - and to date, only - year in which we've run two moderator elections, purely to handle the increase in flags.

These moderators deserve your utmost respect and gratitude, not just for the work they did but also for the abuse they put up with. For though they acted as proxies for vast numbers of people, they shouldered all the blame for mistakes and oversight and inconsistency and lack of friendly hand-holding. This small band was - and to a large extent still is - tasked with protecting the community against the barbarian hordes, at any cost: rather a few borderline posts be deleted than the site be overwhelmed with garbage. With hundreds of flags every day, hand-holding and friendly close messages ain't gonna cut it; you need a tool that lets you dispatch crap quickly and with a minimal amount of fuss:

the final solution

These flaggers were invaluable in helping to point out problems, but next to worthless when it came to solving them. Indeed, the system rather perversely motivated them not to solve problems they came across: why fix a post or even down-vote it, when the only thing that would boost your Flag Score was to leave it ugly and flag it? Like the paranoid shut-in who calls 9-1-1 for everything from raging house fires to someone dropping a gum wrapper, these zealous souls solved one problem but created another.

It was crucial that we find a way to distribute the load more evenly.

Tiered support

When Sam implemented the first review system, the goal was simple: deputize members of the site and put this posse to work mentoring new users, guiding them in a more positive, constructive direction. Around the same time, we switched up the purpose of the flag queue shown to 10K users: instead of the (relatively-rare, often-misused) spam and offensive flags, it would be fed with the plethora of Not an Answer and Very Low Quality flags, and they would be withheld from the moderator queue for a day to give trusted members of the site a chance to handle them on their own.

These two changes helped... A little:

community-handled flags by month and type
Community-handled flags by month

The number of community-handled flags doubled, putting a nice dent in the moderator workload, which remained relatively stable... Until the next summer. By this time, it was becomming clear that these tools - as nice as they were - weren't actually doing very much to distribute the burden: the number of users willing to step up to be 2nd-level support was simply insufficient. It didn't help that for the vast majority of users on the site the tools were either inaccessible or toothless: only 10K users could say, "this flag is unwarranted" - and even when they did so, the flag still went into the moderator queue.

So we started handing out work to everyone who could do it, and gave them tools that actually did something.

Revamping review

Let's face it: this was a huge gamble. Suddenly, a bunch of folks who had barely ever used the moderation privileges they'd earned were invited - encouraged - to jump in and help out. We did our best to provide guidance, both in the instructions presented to folks opening each queue for the first time, and in the limited selection of tools available for each task... But no one knew for sure what would happen once it went live. When we turned on the first queues, each had a backlog of tens of thousands of tasks - we added a few at a time, expecting it to take months or years to bring them under control... But with one notable exception, it took weeks. Accuracy wasn't bad either. Unfortunately, it quickly got worse... And then it got a lot worse.

We spent the next year working out kinks in this system. We didn't want to make it too easy to delete things, but at the same time throwing out legitimate flags and leaving trash around is just replacing one problem with... Well, the original problem.

Lessons learned:

  • There's a "default" action in every queue that folks tend to gravitate to when in doubt. Make sure it's relatively harmless or requires multiple reviews to take effect.
  • Audits are critical as a teaching tool. Folks need feedback on how they're doing, especially right away.
  • Some folks see this as just another game to "win" and will do so at any cost. Many of these d00ds are also "cheating" elsewhere on the site. The system should be able to tolerate a few of these, but kick them out before they break it for everyone else.

After a number of tweaks to the behavior of the system (and a fair bit of helpful feedback from the moderator team), the false-negative rate for reviews is back down to a reasonable level. Which finally sets the stage for some actual progress...

Where we're going with all this

Last July, we finally rolled Close and Very Low Quality flags into the queues - you can see the effects on both moderator-handled flags and community-handled flags in the charts above. The next step is adding Not an Answer flags, which are pretty fast to process but make up for that in sheer volume. The Low Quality queue has been rigged such that flagged posts always require multiple reviews, and tasks completed without a clear consensus on what should be done are then forwarded to the moderator queue.

This should bring the moderator queue back to mostly exceptional situations, which is where we started: moderators as exception-handlers. Because you didn't elect them to handle the easy problems, right?

Of course, the devil's in the details: a lot of "other" flags aren't all that exceptional - they just represent situations we haven't developed tools to handle yet. The same could be said of probably 90% of comment flags - they're not really a good use of moderators' (or anyone's) time, we just haven't been able to figure out how they should be handled automatically. And for those truly exceptional situations, the UI for flag-handling leaves a lot to be desired - it could use some serious revamping with a "more meat, less busywork" design focus. To say nothing of the many, many pages of outstanding bugs and valuable suggestions for /review that would no doubt improve the ability of the greater community to find and lend a hand in areas that interest them.

Escape velocity, one step at a time...

Shog9
  • 159.4k
  • 177
  • 1.2k
  • 1.2k