Some of the Stack Overflow moderators were having (yet another) discussion about what makes "not an answer" fundamentally different from "very low quality", since the former implies the latter in a significant number of cases. Since Stack Overflow can easily hover at close to a thousand pending flags during peak use, moderators are always looking for new ways to let people in the community do more.
The "very low quality" flag has always been ambiguously defined because (as Jeff Atwood once put it) "toxic waste" comes in a variety of shapes and sizes. "You know it when you see it" I used to say. "It's incomprehensible gibberish" others have said. As a moderator, I'd often wonder if some folks had fallen asleep and just rolled their head around their keyboard while servicing these flags.
The thing is, it's really difficult to get more than two people to agree on something being "very low quality" in all but the most obvious of cases, and obvious cases aren't really the typical cases. In fact, this used to be my process for handling most of them:
- See the flag, get the full context (parent question if applicable, other answers, comments, tags, etc.).
- Make an edit (or maybe not).
- Dismiss the flag. Often, if I could sort of see why someone would have flagged it, I'd consider it a good-faith attempt and validate the flag.
By way of comparison, it was rare that I'd take the prescribed action, which is delete with extreme prejudice if there was any substance to the flag. In most cases, I just didn't know for sure, because while I was a pretty good objective judge of quality, I had little to no domain knowledge.
That got me thinking...
Why did we make this a moderator function, anyway?
Could gold and silver badge holders vested in these tags be handling these better than diamond moderators? Probably, because:
They know the system, and how it works. They know that Incorrect or sub optimal != Low Quality. They're in a better position to decline these flags as an attempt to use a moderator as a proxy to delete stuff that just mildly annoys someone for some reason.
They know the topic and can identify when something is really coming out of some strange place in the cosmos, and has no relevance to the question at all.
They know when a question is just an unanswerable mess that can't possibly be salvaged and might be able to help people avoid pile-on downvotes by quickly guiding users with something more helpful than "maybe try again".
What are the drawbacks of, say, opening at the silver tag badge level, allowing users to validate (somehow) or dismiss those flags? Should we open up a review queue just for this? How might that work? What should happen if two people with a silver badge agree on something being low quality and how do we surface it to them?
As we look through places where all of us would agree that users didn't get treated very well, there's this certain subset of questions where—well—there's just nothing nice or encouraging anyone could say about them. These questions turn out to bring out the worst in people and just make lots of ugly. Moderators hate stepping in when you really need domain knowledge to make a call, yet they hate seeing folks have this experience, and we've been debating how or if this flag should even exist for years.
How can we totally offload VLQ flags safely? What did we learn from the dupe-hammer initiative that applies here? Should we set up a review queue, revamp the helper queue to service this, or...?
Maybe just handle it from... something else? We need some input.
This is currently just blue-sky thinking.
But Shog and I always meant to do something with silver tag badges (in fact, I think he's typing his ideas on an implementation right now), but we wanted to get your ideas right now, as we do this, to see if there's any merit here in going further and writing an official specification.
Again, the goals (distilled):
- Let people with proven knowledge handle VLQ flags without moderator help. Part of this discussion is to define 'handle'. As they do that, let's decide if it's of any use, and how it could be better.
- Make sure any implementation improves the user's experience, even if it means deleting something quickly with some guidance (think outside of the box all you want here).
- Make sure there's a feedback loop so flaggers can hone their skills.
Anyone want to chew on it more? We're open to trying ideas and even getting a little nuts, but we need way more input than just Tim and Shog thinking about it in a back room. So, instead of getting an internal specification together to discuss externally, we're going to get an external spec together to discuss internally.
How could this work? What do you think? Is a cheese pizza with Canadian bacon really just very low pineapple? Let us know. We hope we can get enough ideas and concerns together to get a formal specification that we might be able to put into testing, but let's just start with the ideas part.
[list]
(or whatever that means)