415

TL;DR: Yes it can.


Background

On June 27, 2014 Skynet awoke. It looked at Stack Overflow and thought "Why are all these people being so chatty and talking about obsolete things? I should nuke them all!" Fortunately, Skynet was a baby and only had access to my 100 comment flags a day.

Prior to this activation date, the system was fed with 10,000 "Good Comments", "Obsolete" comments and "Too Chatty" comments. These comments were taken from the Stack Exchange Data Explorer. The "Obsolete" and "Too Chatty" comment types had to meet the following criteria:

  • Total comment length of less than 100 characters
  • Comment has a 0 score
  • Had variations of the following phrases:

Phrases

'%mark%answer%'
'%mark%accept%' 
'%accept%answer%'
'%lease%accept%'
'%mark%answer%'
'%thank%you%'
'%thx%you%'
'%.....'
'+1%'
'-1%'

"Good Comments" were assumed, initially, to be anything that didn't fall into the above criteria

This provided a base of 30,000 comments that were roughly categorized into 3 distinct groups. Manually scanning the classifications took several weeks, and through this some of the groupings were changed to reflect a more appropriate classification. Not all comments less than 100 characters starting with "Thank you" are "too chatty", just as not all comments over 100 characters are good comments. I reclassified these comments as if I had encountered them on Stack Overflow.

My next step was to train a classifier. I had initially assumed that I'd start with a Naive Bayes to get a baseline and then work to something more complicated from there. Perhaps, extract text features, user information, etc. and build a fancy classifier. My initial tests showed that the Naive Bayes was accurate 80-90% of the time with test data.

I combined the classifier's certainty of classification with an acceptable threshold of when I'd allow a flag to be issued in my name. Tuning these threshold took a few weeks but eventually I determined the following thresholds were appropriate for my use:

Type Threshold Flagging Enabled
too chatty 0.9997 True
obsolete 0.99 True
good comment 0.9999 False

When a comment is classified, if it exceeds the threshold for one of the above, it is recorded into my database for future retraining. If flagging is enabled, the API is utilized to issue an appropriate flag. Obviously, I don't want to flag good comments, but I do want to record them so that I can reuse the data in a later training step.


Results

What have the results of this experiment been? From my point of view, I'd venture that it's been successful. I have automatically flagged over 17,000 comments. As of December 17, 2014, the process has been running for 173 days. My comment flagging stats are currently:

Count Type
26885 comments flagged
26714 deemed helpful
171 declined

Started at (approximately):

Count Type
9885 comments flagged
9847 deemed helpful
38 declined

This gives me an overall accuracy of 99.36%. Down from 99.61% when no automated process was involved.


There are pictures that help tell this story too. In this first one, we see that the rolling 10 day average for the number of declined flags has stayed below two flags a day. In October, there was a two week period where the rolling average was 0 and nearly a month long period where the system did not make any mistakes.

Flags per day with rolling 10 day average

Since November, the number of mistakes has climbed slightly. The biggest number of mistakes it has made was the opening day of Winter Bash 2014. Purely speculation, but I believe this was the moderators being protective of content and not wanting people to farm the Resolution hat.

Delete 10 last-year comments

Of course, I don't know this. Another theory I have about this uptick since November is the adjustment to day light saving time. My process starts 10 minutes after UTC. It is possible that this earlier hour has caused my flags to be processed by a different moderator, or a moderator that is more awake/less hungry/in a different mood than previously at this point in the daily rotation cycle or because they lost their keys that day.


Total flagged vs Total Declined

Except for 3 days, since June 27th, the process has flagged 100 comments a day. In this chart, you can see the number of declined comment flags along the bottom.


Number of comments saved per day

Finally, this chart shows the number of comments that the system wanted to act on (and a rolling 5 day average). When the system was brought online, it was acting on 700-800 comments a day (saving to my local database). Many of these were being classified as "Good Comments". You can see the day that I adjusted the threshold for "Good Comments" to be acted upon (saved). The drop in the number of comments the system saved is dramatic. Instead of saving 700-800 comments daily, the system now averages about 150 comments to save. Since I don't flag "Good Comments", I feel this is the appropriate action to take.


Comments that were flagged but declined

As shown above, I've had comments flags declined. Some of these obviously should have been and required a retraining or threshold adjustment on my part. Others, in my opinion, should have been removed as noise. Below is a small sampling of both types of comments.

Recent comments that I feel are noise:

Here are some comments that were incorrectly flagged:

Other comments are flagged but then edited prior to a moderator seeing the comment. The edit adds information to the post, thus the declination is justified:

It's also worth noting that despite getting flags declined, some comments do eventually disappear. This is due to either flags raised by other community members putting the comment back in front of a moderator or by simply accumulating enough community flags for the system to act automatically. In either case, the desired result of removing noise has been accomplished.


Lessons and Observations

  • Replication to other sites would depend on site culture

As a (fairly) non-subjective site, Stack Overflow made a good test case for this. On a site like Community Building, Pets, Parenting or other site that accepts subjective answers, "too chatty" would be much harder to classify.

The observation I made on my own that comments with this type of content were distracting has been noticed by others as well. This was actually a very nice validation of my own process and some of the results posted on that thread show many such comments continue to be noise. Of course, this change did also force users to modify their content and may have added new patterns that can be utilized in future training.

The process of checking that my flagging history remains accurate is time consuming. The status of a flag can't be acquired via the API. I've submitted a feature request for this information to be added to the API. With this information, flagging can be paused or stopped if X number of flags are declined.

  • Stack Overflow's volume of comments is a crutch.

Due to the high volume of comments and limited number of comment flags my account has available, I can afford to be picky on which comments I want to act on. The classifier itself is about 85% accurate in determining the type of comment. However, I artificially increase my accuracy by only acting on comments that have a very high classifier certainty by forcing this certainty level to meet or surpass my threshold values from above. Smaller sites, with a lower volume, don't have the benefit of having enough comments to be this picky. It is on these sites that a more feature based classifier would be important.

  • The human element is still unpredictable.

My classifier was trained utilizing my idea of how comments should be flagged. Prior to automating this, I was not 100% accurate. Additionally, moderators are not 100% accurate in their processing of flags. Users disagree on how these rules should be implemented, but are willing to assist in keeping the site clean. With more than 175K comments a week, every little bit helps.


Discussion

As my title states, my original question was whether or not I can teach a machine how to flag comments as I would. The answer to that is yes. The next question is whether this type of system would be helpful in cleaning up comments across Stack Overflow. My system works only on new comments created around each new UTC. Once my 100 flags are hit (or the API tells me to stop), it shuts down for the day. Having something automated go through historical comments or that can run all day would be beneficial.

Finally, now that I've admitted that I've been automatically flagging comments, can I continue to do so?

47
  • 23
    @JamesDonnelly because once you've left that comment and the OP reads it, it ceases to be of value. It does not need to live forever. Commented Dec 17, 2014 at 15:34
  • 30
    You write a program that votes to close obvious off-topic content, like recommendations, tools, libraries, book requests, etc., and you'll be my hero. As it stands, this is fascinating. That reminds me, I've been meaning to write a utility that crawls the tag wikis and looks for plagiarized content. I should get around to that this winter. Commented Dec 17, 2014 at 15:34
  • 56
    My issue with this is that mass flagging of comments does require a mod to review the flags thus takes our attention away from more pressing flags. While comments should be cleaned up, flagging them in blocks of 100 can and will be distracting if this were available to more users. We have other things to deal with. There is no way we could manage this if there were 10+ users doing this daily.
    – Taryn Mod
    Commented Dec 17, 2014 at 15:37
  • 21
    Thoughts: Just because a comment flag was helpful and the comment was deleted does not neccessarily mean that it should have been deleted or that it was disruptive in the first case (meaning you've maybe caused more work by flagging the comment than by just leaving it be). As mods currently spend very little time on each comment flag (due to volume), don't have a good interface to handle them, and have the guidance to err on the side of deletion, I'd be very cautious with flagging comments just because they're not useful - the commented-on post might not be worth the effort to begin with.
    – l4mpi
    Commented Dec 17, 2014 at 15:38
  • 98
    This is both terrifyingly impressive, and generally terrifying.
    – Compass
    Commented Dec 17, 2014 at 15:39
  • 7
    @l4mpi not only that, but we already deal with complaints when we "incorrectly" delete a comment that was flagged - imagine that amplified.
    – Taryn Mod
    Commented Dec 17, 2014 at 15:40
  • 11
    Sometimes we decline comment flags when we notice that someone is just repeatedly flagging the same type of comment. We do it because it's noise to flood the comment flagging queue with the same type of comment flag in an automated fashion. Other times, we really don't care because we just stand on the delete button.
    – George Stocker Mod
    Commented Dec 17, 2014 at 15:40
  • 11
    @Compass, Remember, I've trained mine to flag based on my habits. Three identical processes using the same data sounds like sockpuppeting to me.
    – Andy Mod
    Commented Dec 17, 2014 at 15:49
  • 57
    You may want to consider adding an age check. I don't know about the other mods, but I know I tend to decline flags if the comment is extremely new. We don't need to stamp down on people saying "thanks" or asking for an accept 5 minutes after it happens. If the OP has had plenty of time to read it, flag away. Other than that, great work!
    – Bill the Lizard Mod
    Commented Dec 17, 2014 at 16:15
  • 16
    Thank you very much for your explanation, you rock dude !!!
    – Jongware
    Commented Dec 17, 2014 at 16:30
  • 74
    It's a sad day when an expression of appreciation is considered to be a broken window. When someone thanks me for spending my time, gratis, to help them solve their problem, it reinforces my desire to contribute. An upvote or accept doesn't carry the same positive reinforcement that a simple "thanks" can supply.
    – tvanfosson
    Commented Dec 17, 2014 at 16:36
  • 16
    This is the most impressive post anyone has ever written about comments. Great work Andy! Commented Dec 17, 2014 at 17:50
  • 33
    Any moment now Andy's algorithm is about to flag over half the comments on this post Commented Dec 17, 2014 at 19:26
  • 15
    @canon It's not released, so I find minor bugs acceptable. Also my comment hits %lease%accept%.
    – corsiKa
    Commented Dec 18, 2014 at 16:54
  • 8
    I read almost every comment and answer on this page, and I can't tell if the mods agree with your and give you permission or are telling you your doing to good of work and stop flagging as much. Nobody seems to disagree with your idea or you actually doing this automated, the problem is your finding to many matches? Commented Dec 18, 2014 at 22:48

4 Answers 4

153

To be honest, we figured someone was doing this, based on the similar kinds of comments being flagged. It's interesting work, but I wonder if it's being applied in the right direction.

It should be noted that the baseline for accepting "obsolete" comment flags on Stack Overflow is 95%, so those are going to be deleted with a high frequency anyway. I bet the same could be said for pure "not constructive" comments, but Jon lumped in "other" comment flags with those in his analysis. I know that we decline "other" comment flags at a much higher rate, because many people use them improperly to flag comments they think are technically incorrect.

I think a better direction for this would be to try to identify "rude or abusive" comments with a high rate of success. I don't really care if a few extra "thanks, that worked well" style comments hang around, but I do want to know if people are being insulted and move on that right away. According to Jon, we have only an 80% accept rate on those, so it would be much easier to see if machine learning could flag those better than the average community member.

While I do appreciate the intent and design of your flagging system, going after slightly noisy but complimentary comments isn't my highest priority as a moderator. There has to be a better way for the community to handle these, or to even hide them intelligently, instead of requiring moderators to step in for each one.

10
  • 45
    At 99% accuracy, this thing can now replace us for handling the specific kinds of comment flags it looks for. Totally agree that our time is better spent elsewhere.
    – Bill the Lizard Mod
    Commented Dec 17, 2014 at 16:35
  • 45
    @BilltheLizard - The false positives do give me a little concern for completely automating removal, but I wonder if hiding comments intelligently based on this might be viable. Could give the community the ability to still see the ones mistakenly identified as noise and un-hide them if they're actually useful.
    – Brad Larson Mod
    Commented Dec 17, 2014 at 16:44
  • 13
    Tweaking the comment length term would probably reduce the false positives. But yes, hiding them instead of deleting them would definitely be preferable if this were automated.
    – Bill the Lizard Mod
    Commented Dec 17, 2014 at 16:52
  • @BilltheLizard just let the system handle those flags, no?
    – Braiam
    Commented Dec 17, 2014 at 23:46
  • 3
    @Bill Since they aren't really deleted, wouldn't it still be hiding? Or do you mean something like "don't appear at all for under 1k rep, shows a 'hidden comment, click to view' for under 1-10k rep, shows a greyed out comment for 10k+ rep where they can manually vote to eliminate it permanently"
    – corsiKa
    Commented Dec 18, 2014 at 17:06
  • @Braiam That is what I'm saying.
    – Bill the Lizard Mod
    Commented Dec 18, 2014 at 18:31
  • 3
    @corsiKa When moderators delete flagged comments, only moderators can see them to undelete. That might be too much power for an automated system. It would be fine for comments where 100% of the content is identified as noise, but we'd probably want people to review comments that aren't that clear cut. We currently hide comments in long threads that aren't upvoted. We could do something like that for "noisy" comments as well, and let people decide if they want to see them.
    – Bill the Lizard Mod
    Commented Dec 18, 2014 at 18:38
  • @Bill That is almost exactly what I suggested, with the exception that for anonymous and low rep users, we treat them exactly the same as deleted because we don't expect them to wade through the noise as easily. Although I don't think it would be too much power. I see comments as second class citizens. It's like if you went on a plane ride and had to stand, I'd be pissed, but if I didn't get my salted peanuts I'd be like "well that sucks" and move on. I admit that's might not be appreciative enough of users' attachments to their comments, though...
    – corsiKa
    Commented Dec 18, 2014 at 20:14
  • 1
    Having just crossed the 10K threshold, I think if the volume of flags is going to go up, you need community involvement in order to handle it. I'm alluding to the idea that 10K is a good threshold for that privilege.
    – crthompson
    Commented Dec 18, 2014 at 23:06
  • 4
    Wouldn't it be beneficial to alert the user of the comment to the deletion of their comment? That could easily outsource the problem to the user, where if they are told directly "Your comment was rude and was removed. If you feel this is in error, please click here" Then, you could easily run this with 99% accuracy, and those few comments that slip through can be remedied through the smaller number of "accidental deletion" tickets. I would assume that the 1% accidental deletion tickets would be a much smaller number than the current daily comment flags, meaning less moderator work.
    – user4979686
    Commented Nov 16, 2015 at 21:34
103

This post alone is an amazing amount of work, the amount of work it actually represents is admirable. It's not an odd itch to want to scratch, we've been putting a lot of thought and time into comments lately. While we did find a few nice things to say about how comments on our sites are actually used, the majority of the time we've spent has gone into talking about what we don't like, or basically the majority of our comment system and accompanying moderation tools.

Go back for a second on why we decided to implement them, which was a growing problem with litterbugs. As people on the Internet are so accustomed to being able to comment on practically everything, folks were using answers to write comments despite glaring UI fixtures asking them not to do so. Subsequently, Jeff cried caved, and we had comments, followed by a system that was designed to help elevate anything useful contained in them above the "+1 lol jQuery ftw" baseline.

10k users could also delete any comment in the system. We did our absolute best to make sure people didn't get too attached to comments because we correctly anticipated that which was inevitable, people were going to make a lot of noise using them. Mods were informed that mercy is completely optional when faced with cleaning up a noisy thread of comments, and users were warned that comments were at best ephemeral.

The noise comments generated soon surpassed the noise that we hoped they would keep out. We then put a rep requirement on them so that .. oh wait, yes, that's right, we ended up with:

  • People were using answers to write comments, so we should give them a place to write comments instead
  • We then restricted comments so that only people that know how our system works to some extent can use them
  • Many people that tend to write noisy comments come from searches, have no or little rep, so they tend to write answers inst...

Headdesk.

If you're ever a mayor of a city and find that you have a litter problem, we strongly recommend welding most of your trash cans shut; it's worked so well for us.

In the city analogy, you've found litter right next to a trash can, yet you have to call the sanitation department to send someone to come throw it in. That's not good for you, that's not good for them, in fact if it were any more inefficient we could make a board game out of it with the end goal being to trap mice. The work you've done and the effort is fantastic, but I think we're going about it the wrong way, and I think we've got to bite the bullet.

The chief problem with comments is that comment moderation tools stink.

That's right, they stink. Too many clicks, too many people involved, too much thinking and work over things that were never supposed to matter much to begin with. We've talked about algorithms to hide comments, different kinds of flagging tools, all kinds of stuff - here's what I think we need, and what I'm chewing on at the moment:

Just-in-time help

Because you know, please don't comment to show appreciation, click on the up-vote or (anon) helpful button instead. Comments that don't directly address points in the post just create more noise to read and are routinely removed. We have this ability now, sort of, and can probably expand it. Basically, the same matches you outlined.

Badge-based trust

Enhanced privileges for gold tag badge holders has worked out extremely well. In fact, I'm planning to expand that idea next year. Why can't we let folks with flag-based badges just delete noise when they see it?

I think we've complicated what was initially a very simple system enough, and need to get back to basics. No, I don't want any 10k user to be able to whack things from orbit on a whim, but I tend to trust those that our moderators trust.

There are issues and caveats, the chief one being that people tend to be very attached to everything they type on the Internet. But when you warn them based on what they're typing .. that's teaching.

I've got to chew on this a bit more, but this was a wake-up call. We've got to address the issue sanely, or better mousetraps start looking dangerously attractive as time goes on.

13
  • 1
    Really glad to know SE is going to take this seriously (it's about time).
    – gunr2171
    Commented Dec 18, 2014 at 18:03
  • 3
    @gunr2171 We've been taking it seriously for quite a while, it's just .. well, one of those really icky things to solve in a manner that everyone is happy with. I'm proposing that everyone isn't going to be happy and we need to be able to live with that :)
    – user50049
    Commented Dec 18, 2014 at 18:04
  • 1
    Also, there would have to be oversight and limits if this is implemented, e.g. no user could delete more than (x) comments in a day, and more. That's for fleshing out, this is just an idea, or rather the start of one. I don't see any 'flagging' system for these on the scale of Stack Overflow ever working, it's just too much, we have to be able to offload some trust along with these units of work we ask folks to do, based on what they've done.
    – user50049
    Commented Dec 18, 2014 at 18:07
  • 15
    Could also make it so that the comment would need to be flagged by at least one person, in order to be eligible for instant deletion by anyone with a Marshal badge. I'm curious how many of those flags would actually age away before anyone acted on them. There are some possibilities here, I need to run some queries.
    – user50049
    Commented Dec 18, 2014 at 18:26
  • 11
    "Why can't we let folks with flag-based badges just delete noise when they see it?" I have the Marshal badge so I'd probably be eligible for this privilege, but I'm against it as I don't think a single user should have the power to (irreversibly, non-transparently) delete content. Mjolnir is fine because it is transparent and can be reversed by others if it misses the target. The same is true for question/answer deletion. If this were implemented for comments, we would need a much more open handling of all aspects of comment deletion, including notifications for the poster, accountability,etc.
    – l4mpi
    Commented Dec 18, 2014 at 19:05
  • If you could share the results of those queries, I'd be very interested in seeing that data.
    – Andy Mod
    Commented Dec 18, 2014 at 19:09
  • 1
    That said, I'm still under the impression that we actually delete too many comments, or at least too many of the wrong type. Of course, complete noise and offensive comments have no place on SO, and comments should be cleaned up if important information is hidden under less useful content like "too localized" comments (e.g. a message directed at OP telling them how to post better questions), but I'm not sold on the idea that all hell breaks loose if we err on the side of non-deletion for everything in the grey area.
    – l4mpi
    Commented Dec 18, 2014 at 19:13
  • 3
    @TimPost what would be wrong with having the old 10k flag interface and allowing them to cast pile on flags that would have the system delete it through already existing processes (3+vote flags)?
    – user289086
    Commented Dec 18, 2014 at 19:40
  • "Could also make it so that the comment would need to be flagged by at least one person, in order to be eligible for instant deletion by anyone with a Marshal badge...." - so, wait: can I consider my proposal (meta.stackoverflow.com/q/278927/792066) pretty much 'status-under-consideration'? Commented Dec 18, 2014 at 23:13
  • 4
    This answer contains an amazing amount of insight.
    – Ben Voigt
    Commented Dec 18, 2014 at 23:19
  • 3
    "Let folks with flag-based badges just delete noise", if these comment flagging bots exist, 5 days and a user has Marshal. It's a very low bar for a malicious user to meet and would require a bit more validation.
    – Brad Koch
    Commented Dec 19, 2014 at 0:57
  • 5
    @BradKoch We'd need to make some changes. If this were implemented, mass flagging wouldn't help, as mods would only see them if they piled up on a comment and if they were close to expiring, for instance (not a fully baked criteria, just a direction). You couldn't game the Marshal badge by flagging a lot of comments with a script, and you'd only be able to delete (N) per day anyway - this could be made pretty abuse-proof without a lot of work.
    – user50049
    Commented Dec 19, 2014 at 6:40
  • A followup to some of the ideas outlined in this post that gives an update on what the current mentality is wrt comments and whether or not anyone’s working on implementing improvements would be really nice. Commented Nov 11, 2016 at 0:24
54

I think it would be ideal if script like that would be run by Stack Exchange automatically and feed it into a dedicated review queue where hundreds or thousands of reviewers would validate the results.

Another option is to set up a dedicated chat room and feed links to comments discovered by a script into it, for attention of users interested in cleaning up. This would be pretty much similar to the way how "SO Close Vote Reviewers" run and coordinate their efforts.


Have to admit, current way feels quite awkward. Diamond moderators, "human exception handlers", carefully picked and elected experienced users, assumed smart and credible enough to evaluate really tricky cases, are forced to decide whether "+1 thank you" comment is to be deleted. Give me a break.

Idea that 3 flags would automagically delete garbage comments doesn't feel right either. It seems to be based on a shaky assumption that hundreds... thousands such comments would eventually collect enough eyeballs from users concerned enough to flag. Famous complaint about explosion of comments on Stack Overflow suggests that this way simply doesn't work.

2
  • 6
    10k users could delete any comment in the system at one time. I don't think we should go back to that, but badge-based trust for deleting noise when you see it does help solve the not enough eyeballs problem.
    – user50049
    Commented Dec 18, 2014 at 17:59
  • 13
    @TimPost being a 10Ker myself at other site, I also don't think we should go back to that - I am uncomfortable with idea of a system where folks like me would be able to unilaterally remove any comment
    – gnat
    Commented Dec 18, 2014 at 18:58
7

There's another approach that can be taken with this that would serve to help delete poor comments faster and reduce the moderator workload.

The comment flagging system currently has it that if something is flagged a certain way and contains particular words (Without giving too much away, there are certain heuristics that make some types of comments a bit, shall we say, easier to flag away than others).

Extend this heuristic to the one flag removal. If the comment is above a certain threshold for deletion worthy based on the system mentioned above, just delete it. It seems to work fairly well currently.

So instead of just checking if someone is swearing and a rude flag is cast, check to see if its "thanks, that worked" which scores a 88 out of 100 points and is over the 85 threshold for one flag deletion. Poof, its gone and need not bother a moderator.

2
  • Not safe to have the "checking system" working in the same way the the "finding system". Commented Sep 22, 2016 at 11:42
  • Profanity is profanity, which is why that works. When you introduce context, you introduce a lot of room for error. Commented Jun 10, 2019 at 14:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .