33

I personally like the "No longer needed" flag that was introduced. For me, it really simplifies the general reason for removing the comments and makes things easier when making that decision.

On the flip side, how has the introduction of this affected the moderators who have to review the non-auto-removed comments?

I remember having a conversation with a mod a while back on the same topic, but was curious about how it affect the mods generally when it came to their regular duties.

6
  • While I strongly doubt it, I wonder if this had an impact on the decision. Unifying flags on a smaller scale may be the dress rehearsal for a larger overhaul. But that's not confirmed in the slightest.
    – Makoto
    Commented Aug 24, 2017 at 22:05
  • Wasn't "not constructive" kind of being used as a catch-all reason anyway? Commented Aug 24, 2017 at 22:46
  • 8
    I like it, too. I think it makes a lot more sense. I can't in good conscience post an answer here, because I only became a moderator after this change took effect, so I can't really do a comparison between the two states. But I will say that I don't really see the downsides, or what benefit it would be to split these flags up into different little buckets.
    – Cody Gray Mod
    Commented Aug 24, 2017 at 23:44
  • Yeah @CodyGray I was particularly interested in the before and after comparison by the moderators. Whether you all experienced a consciously noticeable difference or if it was just more of the same.
    – Nkosi
    Commented Aug 25, 2017 at 0:08
  • For me personally I find myself using it a lot more than before. And I also noticed that my declined rate reduced given my now understanding of the reason behind the flag.
    – Nkosi
    Commented Aug 25, 2017 at 0:10
  • 1
    @EJoshuaS Yes; as SO has removed various catch-all flag and close reasons, users have migrated to the next most broad reason each time.
    – TylerH
    Commented Aug 26, 2017 at 21:34

2 Answers 2

33

Wanna see something cool?

chart of comment flags per week, broken out by handler

That's a chart of comment flags by week for the past two years (excluding the current week) by who handled them. Comment flags are handled by "community" when a sufficient number of flags are raised on the same comment, or the post is deleted, or the author deletes the comment, or the comment matches a pattern).

Observations:

  1. there are more comment flags being raised than ever before.
  2. the bulk of the increase is being handled without moderator intervention.
  3. this is without Andy's script running.
  4. moderators are still handling something like 4 thousand comment flags a week.

So... That's kinda nice. Folks are using the new flags and the volume isn't putting extra strain on the mod team.

What about the strain of handling a different set of flag types? Well, what about another line graph?

mod-handled flags by type

The volume of "other" flags (now "in need of moderator intervention") is about the same. Rudeness flags have increased pretty significantly, as I expected they would after the removal of "not constructive". But the vast, vast bulk of NC comment flags went straight into the "no longer needed" bucket, along with chatty and obsolete flags.

Now... I'll defer to the moderators actually handling the bulk of these flags to share their experience with sorting out any ambiguities.

But, in the interest of full disclosure, I wanna share a rough metric I've been keeping an eye on. When we rolled out the new flag types, I put together a query to estimate how much time moderators were spending handling flags. It's NOT precise by any means - I can't tell when a moderator sits and thinks for a half hour and then blows away 30 flags in one go, or pauses to enjoy a sip of Tab after every rude comment deleted... Nor can I truly attribute the time spent to a single type of flag when there are multiple types of flags handled in a single conversation. But, it's good enough to watch for signs of trouble. And here's what I got for comment flags over the past 82 days:

FlagType          Hours spent handling flags Avg seconds per flag 
----------------- -------------------------- -------------------- 
no longer needed  103.301111100              17                   
chatty / obs / NC 64.199444433               11                   
rude              41.825555550               43                   
other             15.485555550               35

For the time being at least, it appears that NLN flags are taking more time to handle than the flags they replaced.

11
  • 3
    Your last observation hardly seems fair. 11 seconds vs. 17 seconds does not seem like much of a difference, when you compare it to a whopping 43 seconds for rude flags. Actually, it's surprising to me that rude flags take the longest to handle, considering that whether or not something is rude should be pretty obvious from a cursory read of the comment. How, exactly, is your metric measuring this? When does it count that a moderator first "looks" at a flag?
    – Cody Gray Mod
    Commented Aug 24, 2017 at 23:43
  • If you open the page where the flag is active, the system actually records a "vote" that corresponds to your viewing it. That's a pretty inefficient way to handle comment flags though, so most comment flags won't have those. Instead, I mostly rely on a Lay's Potato Chip algorithm: assume you're going to handle more than one flag at a time and measure the time between flags (with some padding). It is, as I admitted, not a particularly precise measurement (nor is it wise to compare numbers between two different moderators, although you've probably seen me do that too...)
    – Shog9
    Commented Aug 24, 2017 at 23:47
  • @CodyGray if Shog really wanted to do a statistical comparison, he may do ANOVA or t test to confirm his hypothesis.
    – Braiam
    Commented Aug 25, 2017 at 0:12
  • 1
    (...but since when do I do statistical analysis of anything...)
    – Shog9
    Commented Aug 25, 2017 at 0:22
  • 1
    But but... you are learning R.
    – Braiam
    Commented Aug 25, 2017 at 0:44
  • 9
    You're thinking of someone else, @Braiam. I'm learning Aaaaaaaaaaaaarrrrrrrrrrrrgh! Come Sept 19, imma whip out some piratical analysis...
    – Shog9
    Commented Aug 25, 2017 at 0:54
  • 5
    Wanna see something cool? generally blue indicates cool and red indicates hot. red is trending in first graph so it's not cool. it's hot
    – Sagar V
    Commented Aug 25, 2017 at 6:49
  • @i-- I was thinking more on the lines of "I know you are playing me... and yes, I do."
    – Braiam
    Commented Aug 25, 2017 at 12:20
  • 1
    What caused the increase of the flags "handled by community" in the last few weeks?
    – Floern
    Commented Aug 25, 2017 at 14:15
  • 3
    That'd make a great question, @Floern!
    – Shog9
    Commented Aug 25, 2017 at 23:30
  • @Floern As the comment-flag system has a bit change, maybe people are more interested by this "new feature"?
    – Mistalis
    Commented Aug 28, 2017 at 12:49
13

I had mentioned this before that having a single flag in place of three has made it a bit harder for us to process the flags. Quoting a part from that:

Now, it's all in one huuuge pile (around 400 ATM). The "Thanks" and "I edited" comments are getting muddled between the other flags. Earlier they used to be in the TC and the Obs, respectively (and 5~6 seconds were enough to handle those). When you see a list of "Thanks" in one area, you can just delete them one after the other, which is easy. Whereas now, you're seeing a "Thanks" and then another "long nonconstructive comment". So the brain's taking a harder time to parse "Thanks" in between those.

Now that we've spent some time handling the new style of flags (and that we have had two new mods who have no idea about how easy it was before), as a mod team we have improved. Most of us are now used to the new system and the earlier issues which I had mentioned in my previous answer are no longer there. At the moment, we are quite happy with the new system.

Stuff which I think that are taking more time is mostly related to the previous obsolete comments. Earlier users used to flag comments as obselete when their post was edited, and hence we used to just check if the revisions has a change mentioned in the comment and delete the flag (which used to take 10 secs, mod dash > click the triangle to expand > middle click on revisions > delete comment). Now these are flagged as No longer Needed. Doesn't sound like it causes a problem, right? Wrong. People also flag irrelevant comments as No Longer Needed which used to be flagged as Non Constructive before. (An example would be, "your answer doesn't work for X" is actually an irrelevant comment when the post asks only about Y). This would need us to just check at the answer and the question to determine if it is right or a wrong flag. (A wrong flag here would be on a comment which brings out a fault with the answer). Now we are not sure what the flagger thought while flagging, (We aren't shown the flagger's name, so we don't know who's flagging to assume anything), hence we need to check if it's either obsolete or non constructive. This implies we have to do both the checks.

One another possible factor why we're taking more time to handle NLN than the earlier ones is that we're having very few comment flags in the queue at a given point of time (again thanks to the election). So we're spending a bit more time in not only cleaning the particular comment but also all the other comments on that post (Undo has written a very useful userscript to do this). This certainly would take more time than just deleting the comments on the mod dash.

For the first issue, I think once we get used to the flow which we are doing now, we'd be able to get back on track. For the second, I don't think spending a couple of seconds more to cleanup more would be an issue.

All in all, it's just a matter of time that we are back to the normal pace and forget the fact that there was once different flags (or perhaps say, "Hey I can't believe that the users then had so many choices to flag, and mods had to handle them separately"). Shog mentions that we're still lagging behind what we used to do, so we will certainly improve and try to handle them at the usual pace. :)

2
  • 9
    I think the fundamental issue with "no longer needed" is that it indicates that a comment is no longer needed, but doesn't explain why. It might as well be called "delete this comment" and we'd take the exact same amount of time to process these flags - because "no longer needed" tells us nothing except that the comment probably needs to go away (and that's assuming people use it correctly - which isn't always the case).
    – BoltClock Mod
    Commented Aug 27, 2017 at 6:15
  • @BoltClock So... you're saying we should just remove all the flag reasons except the custom 'in need of moderator attention' one, and require a description? :-P
    – TylerH
    Commented Aug 27, 2017 at 16:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .