46

I currently have 3 items in my flag queue, under review (All > 25 days old):

Old requests

But yet I'm noticing recent NAA deletions from mods like below:

Recent NAA deletions

To me, it seems like this content should be much lower priority in the queue, especially if it can be handled by the community. How is the current priority system implemented? I'm curious why recent NAAs that can be deleted via review, are even handled by mods (Or rather prioritized above custom flags) in the first place. Should the ♦ moderation queue priority system be reworked?

31
  • 22
    There isn't much of a priority system other than spam/abusive flags bumping to the top. Moderators will usually just work on particular flags that they feel they have the time to handle. Back when I was a mod, I would generally avoid custom flags unless I knew I had a good chunk of time to really focus on them. They can be very draining.
    – animuson StaffMod
    Commented Feb 27, 2018 at 17:16
  • 18
    @animuson Out of ignorance: aren't the custom flags the more relevant things for mods to handle? Many of the other flags and votes can be dealt with by the community, can't they? Or am I completely misreading mod duties?
    – yivi
    Commented Feb 27, 2018 at 17:21
  • 13
    The queue prioritizes posts with multiple flags. That non-answer was flagged by five other people and thus probably reached the top of the queue. Your custom flags haven't even sniffed the first page of the queue yet.
    – Michael Myers Mod
    Commented Feb 27, 2018 at 17:21
  • 5
    The mod queue (deliberately on SO at least) - doesn't expose the mods to the NAA/VLQ queue without an hour delay. If we then clean 'em up by delete votes so be it?
    – Jon Clements Mod
    Commented Feb 27, 2018 at 17:21
  • 3
    @MichaelMyers So there is some sort of prioritization there. Are these NAA flags mixed in with the custom flags, or is it a seperate queue all together? I'm asking: Because I feel like I'd rather have a moderator spend 5 minutes on the custom flag queue, than 30 minutes on the NAA queue. 25+ days is a bit unacceptable for handling flags.
    – Blue
    Commented Feb 27, 2018 at 17:27
  • 7
    @JonClements An hour doesn't seem like hardly enough time for 4 reviewers to cast "Recommend Deletion" votes in the queue.
    – Blue
    Commented Feb 27, 2018 at 17:35
  • 1
    @FrankerZ: As I understand it, the standard flags can be handled quickly in bulk. It's the custom flags that take "30 minutes". Commented Feb 27, 2018 at 18:04
  • 3
    Shoot this down if it's an unworkable idea, but could we allow 10K+ users the facility to specify that their custom flag is a quick one? That would be a sortable field to allow mods to get rid of easy ones. I don't have a strong need to have my flags acted on quickly, but if there was a general desire to get a higher throughput, it might be worth considering. I appreciate it is open to abuse, but I hope the rep restriction would help with that.
    – halfer
    Commented Feb 27, 2018 at 22:17
  • 3
    @DavidG, I will see your Jan 23rd and raise it with a Jan 17th. I have cut back on my moderator flagging as I don't want to be part of the problem.
    – Stephen Rauch Mod
    Commented Feb 28, 2018 at 3:47
  • 3
    @StephenRauch Please don't do that. Don't refrain from flagging if you find anything that needs our attention. We will get it to it eventually.
    – Bhargav Rao Mod
    Commented Feb 28, 2018 at 4:50
  • 3
    @BhargavRao That is nice and all but in all honesty, some custom flags actually are time sensitive. So if there is no proposed solution how the situation can be improved in the future, then being a bit more selective about what to flag is kind of the only option here.
    – poke
    Commented Feb 28, 2018 at 8:28
  • 5
    @BhargavRao I do disagree with that sentiment a bit. I do believe that improving the tools for that small subset of users would help to improve the overall experience for everyone. But I understand that there is enough other stuff to work on that would be directly more useful for those millions.
    – poke
    Commented Feb 28, 2018 at 9:19
  • 2
    @BhargavRao If flags are taking such a long time to handle, is it time to ask if we have enough mods to deal with them? It's either that or impove the mod tools somehow.
    – DavidG
    Commented Feb 28, 2018 at 9:55
  • 3
    I suspect I'm partially to blame for this diluting of the custom flag pool, I got 220 pending custom flags right now, some from the beginning of january still.
    – Magisch
    Commented Feb 28, 2018 at 12:22
  • 3
    Imgur is blocked in some places where stack.imgur is not, and also it's standard practice for off-site images to be moved on-site to prevent linkrot, which covers not only sites going down, but people removing images they uploaded later.
    – TylerH
    Commented Mar 3, 2018 at 14:52

2 Answers 2

46

The only priority ordering in the moderator flag queue at present comes from spam / offensive flags being shown above all the others and posts with multiple flags being sorted to the top.

"not an answer" flags have a tendency to build up on obvious non-answers, sorting them to the top. They are also very quick to judge, so you'll see flags like that getting acted on faster than others. I can delete dozens of clear non-answers in minutes. The community also handles a large chunk of these (in the last 24 hours, you cast 15 "not an answer" flags and moderators didn't even see 6 of them before they were handled).

Sorting of custom flags has always been a problem. Every issue under the sun gets mixed in there, and some of them can take hours to investigate. Important issues are mixed in with complaints about answers being downvoted without comments and demands that moderators answer their urgent question. Not to mention the piles and piles of migration flags to every site under the sun, few of which I'm comfortable handling. We need some kind of triage or keyword-based sorting of these, but at present the best we have are some quickly thrown together userscripts.

I've noticed that we tend to run into cascading problems with custom flags, where once a certain number of them have built up in the queue, the queue size tends to rise dramatically. We had a relatively low flag queue at the start of 2018, but things built up and got out of hand recently. My guess is that Stack Overflow will run another moderator election soon to account for this difference.

I do try to spend most of my time with custom flags, but the order in which I approach them can be fairly random. Sometimes I go to the newest flags and see if there are any pressing issues that can be dealt with quickly. Sometimes I go to the oldest flags to handle issues we've been putting off for a while. Sometimes I sort for all instances of plagiarism in the queue. There isn't any enforced ordering to this, so times will always be variable.

16
  • 8
    Have you considered more flag reasons to reduce the number of custom flags? Or aren't there really any clear candidates for reasons to add? Commented Feb 27, 2018 at 20:11
  • @Dukeling - This wouldn't help in many cases, since additional information is needed on top of the flag (migrations to sites not on the list, links to sources for plagiarism, descriptions of abusive behavior, evidence for voting rings). It also would add complexity to the flag dialog, something that SE has been trying to simplify over the years. I think the best solution is simple keyword sorting on the back end, ideally with keywords and search terms that moderators could customize. There's a long tail to these flags, and it might take some experimentation to see which ones work.
    – Brad Larson Mod
    Commented Feb 27, 2018 at 20:23
  • Well, there is already a text box in the flag dialog box... Commented Feb 27, 2018 at 20:37
  • The flag dialog already includes the duplicate close vote UI, so I don’t see why adding a bit complexity would hurt that much if it would allow mods to handle the flags more efficiently. For migrations, there could even be a dupe-like selection of the target site, so mods could filter based on the site (without having to do text scanning); or we could even involve the mods of the target site to offer some advice (this would reduce the need for mods to know about the target site’s rules)
    – poke
    Commented Feb 27, 2018 at 20:44
  • 20
    Thinking about how much time these custom flags could take up, and that moderators are doing this in their RL free time really makes you appreciate the work they're doing.
    – Tas
    Commented Feb 28, 2018 at 0:17
  • 15
    @Dukeling: We do need a dedicated plagiarism flag, and one has been proposed before. Assuming most plagiarism flaggers flag responsibly, including the link where required (as would be enforced by having a dedicated flag), it would significantly cut down on the effort needed to filter out the low-effort "this is copied from another answer" (flags that have merit but are unsourced or otherwise poorly written) and flippant "he copied my answer" (flags that have no merit and simply demonstrate poor sportsmanship) from flags that actually do require expedient attention.
    – BoltClock Mod
    Commented Feb 28, 2018 at 1:03
  • @BoltClock That would be a good idea. I have an open flag (detailed) since feb 3. I'm guessing since the queue is only growing, and that this flag will only ever get one "vote", it will silently age away.
    – TT.
    Commented Feb 28, 2018 at 7:38
  • 5
    @TT. Custom flags don't age away, so you can be assured that we'll get to it. (Also for plagiarism flags, try to add "plagiarism" in the flag text, it'll help us handle it faster).
    – Bhargav Rao Mod
    Commented Feb 28, 2018 at 8:52
  • @BhargavRao Aha, that's a good hint, thank you :)
    – TT.
    Commented Feb 28, 2018 at 9:01
  • 21
    Let's place another option in the flagging system for your: "complaints about answers being downvoted without comments and demands that moderators answer their urgent question". If someone chooses this then the flag is immediately declined without intervention. Commented Feb 28, 2018 at 22:29
  • 1
    @BhargavRao Care to share what's inside the deleted post you linked? Do you also check slight spelling mistakes of the plagiarism flags when you check for things inside the queue? Or does writing it wrong mean that someone can plagiarism your content for a long time instead of taken down quickly.
    – Ferrybig
    Commented Mar 1, 2018 at 10:34
  • Nothing much in that link @Ferry, it just tells to use "plagiarism" in the flag text. I usually don't check the spelling mistakes, but a few other moderators do.
    – Bhargav Rao Mod
    Commented Mar 1, 2018 at 10:37
  • 2
    @Ferrybig - Basically, some of us us a userscript that looks for "plagia" or "copied" in custom flags and groups those into a Plagiarism bucket. It's not an official thing, and it doesn't properly catch all these flags. Plagiarism can stand for a while not because of longstanding flags, but because it can take months for people to find and flag plagiarized content to begin with. Once found and acted on, dozens of posts can be removed in response to a single flag.
    – Brad Larson Mod
    Commented Mar 1, 2018 at 15:31
  • @BradLarson In interests of improving my process, does a similar bucket exist for serial voting? If so what are its keywords? Do you guys prefer voting rings and simple serial voting to have different keywords for easier sorting? Similar with serial up & downvoting
    – Magisch
    Commented Mar 2, 2018 at 12:28
  • 2
    Seems like it would be worthwhile to publish such keywords and make an unofficial set of additional flag reasons. I feel confident that most responsible flag users would be very happy do what they can to make handling easier for the mod team, in the absence of any support from SE.
    – jscs
    Commented Mar 2, 2018 at 18:07
7

Some moderators work "Bottom Up", others work "Top Down".

Here's what I mean:

enter image description here

No doubt, the "Other" flags are probably the most important; but there's a lot of chaff mixed in with the wheat. If you have between 10-30 minutes to devote; you may cover 30 flags, you may cover 10, you may cover 1.

If you work bottom up; the positive feedback mechanism of 'clearing out the queue' feels really good; even if the time is spent on lower priority things means not spending the time on (to brad's point) what could be higher priority flags.

For each type of flag in the picture; come up with what you think is an appropriate amount of time to spend processing that flag before it's just 'declined'. Based on your calculations, I'd be happy to ballpark how much time it'd take to clear out the queue.

2
  • I think custom flags are really hard to categorize
    – Magisch
    Commented Mar 3, 2018 at 9:14
  • @Magisch: Well, they are custom...
    – BoltClock Mod
    Commented Mar 3, 2018 at 16:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .