32

We currently have tons of s that some day someone needs review*, and they keep arriving. Stack Overflow is filled with bad tags.

Our guidance What is the process for tag removal (burnination)? states four criteria for burnination.

  1. Does it describe the contents of the questions to which it is applied? And is it unambiguous?
  2. Is the concept described even on-topic for the site?
  3. Does the tag add any meaningful information to the post?
  4. Does it mean the same thing in all common contexts?

But it also has another quote from Shog9:

If it looks like pointless busywork, it probably is pointless busywork

I have been participating in burnination of multiple tags as , and trust me having 5 people to review 1000 of questions to burninate 1 tag is a lot of effort. Should the request for community effort be considered when posting burnination request?

Is time to change the criteria to something like:

  1. What harm/confusion does this tag actually do? Why is this not pointless busywork?

The current 1,2,3,4 would already be included in this. Point 3 maybe even is an indication that it's pointless work to burninate the tag (it's just useless).

Should we also consider to open a new tag , in which people can request for tags just to be removed by staff, hence they don't expect community to review all the questions.

The , would be used for tags that are bad/useless, but do not need community effort to sort out the problem, review questions to re-tag to correct tag, close and delete off-topic question. Staff can simply delete the tag in 6-8 somethings.

*Jon Clements has an on-going Trogdor effort to review old burnination requests, in which I'm participating, but they keep arriving, which led me to post this meta question.

27
  • 8
    @Braiam or it would concentrate effort on critical issues? Commented Apr 18, 2018 at 10:33
  • 1
    What more critical issues than seeing answerable questions without answers? Or in dire need of editing without being edited? Or needed to be closed so we could prevent bad answers and the OP fixed them?
    – Braiam
    Commented Apr 18, 2018 at 10:43
  • 2
    In fact removing a tag that is tag is probably one of our last concerns Commented Apr 18, 2018 at 10:45
  • 1
    Well, ineffective tags are the main cause of above concerns... so, what are you getting at?
    – Braiam
    Commented Apr 18, 2018 at 11:24
  • 3
    It looks like pointless busy work Commented Apr 18, 2018 at 11:56
  • 1
    "looks" doesn't mean that it is.
    – Braiam
    Commented Apr 18, 2018 at 12:56
  • 6
    Feel free to answer and explain why we should put effort into it Commented Apr 18, 2018 at 13:18
  • 9
    @Braiam How does the existence of a bad tag result in questions not being answered, edited, or closed, as appropriate? The biggest problems with bad tags are tags that are ambiguous, and result in people constantly needing to re-tag questions asked in that tag to the one(s) that are actually appropriate, and making it hard to find the questions that actually belong to that topic. There is perhaps a lesser problem of tags for off topic concepts resulting in people asking off topic questions, but I suspect most people asking such questions would do so even if there was no such tag.
    – Servy
    Commented Apr 18, 2018 at 14:01
  • 7
    Let me turn this around - do you have any examples of tags that you think would pass this test? What's the answer to this question for those?
    – Undo Mod
    Commented Apr 18, 2018 at 15:27
  • 1
    As it is now, people posts request, we all agree it the tag is bad (we don't really consider if it's actually worth the time) and then we carry on, we feature some and it gets upvote, a few brave (that are always less, try to clean it up, while another 10 arrives). At least lets conclude that the current situation is not ideal. Commented Apr 18, 2018 at 16:03
  • 1
    due note that my indicated tags are only some examples of what I think can be worth the effort, lets not turn this questions into which tags are important to deal with, this discussion should instead be present on every burnation request The purpose of question instead is: Should we discuss and ask what is the harm?, is it worth the effort? on burniation request? Commented Apr 18, 2018 at 16:11
  • 5
    One of the criteria to be answered by the OP should be: Are you willing to do the closing/re-tagging/deleting/editing of those x questions. If the answer is no, don't bother posting.
    – rene
    Commented Apr 18, 2018 at 16:24
  • 3
    The amount of work taken to remove the take is irrelevant when assessing whether a tag is useful or not. Having said that, if the burn todo list is long well, it stays long until it can be addressed.
    – user3956566
    Commented Apr 18, 2018 at 17:31
  • 3
    @YvetteColomb To me both moderator time and community time is valuable and limited (even if it is free of charge) so the conclusion is the boring cost vs benefit. To me tag is useful is not enough to merit hundreds of hours of community work, we need to choose and directed or resources (yes it's sad and boring, we should edit and improve all tags.. but that's utopian and not realistic). I have taken "a stand" on a highly upvoted and featured meta, at least tell me why I should spend hours reviewing all questions with tag [multiple]?, Commented Apr 18, 2018 at 19:00
  • 3
    @BJMyers The problem here isn't that there are tons of people doing tag cleanups and people want to stop them from doing it, the problem is that there are tons of people suggesting tags get cleaned up and then those tags not actually getting cleaned up. If tag cleanup requests were more limited to requests that would be useful, then perhaps more people would see a reason to participate, or at a minimum, those that do choose to will be spending their time on work that adds more value.
    – Servy
    Commented Apr 18, 2018 at 22:02

3 Answers 3

10

Note that Shog on the same post, also includes the instructions to start, namely:

  • find closed questions and delete them,
  • find downvoted questions and close them (if they warrant it),
  • find poorly-written questions and re-write them (if they deserve it). You get the idea
  • sometimes it makes more sense to just replace it with another tag, or one of a number of other tags

This cleanup is easily the hardest, most time-consuming part of the process

In no moment, Shog specifies that the community shall retag all the questions to remove the tag, in fact, he says that the process that takes most time is basically cleanup... something that we should strive for in any circumstance anyways1.

In other words, all tag burnination efforts could be done single-handily by the asker if that is the only problem. What the burnination process tries to address is that such effort done by a single individual cause the following:

  • Lack of knowledge by other members of the community, which may be stakeholders of the tag
  • Depending the size of the tag and the incidence in new questions, it's very difficult to remove the tag since there would be more tags removed rather than added
  • Usually tags being burninated have more than one problem, and some of those problems (closing, deleting) are more useful to solve for SO as whole but needs more than one user to take place
  • It's demotivating when you are the only person cleaning the beach
  • Takes too much time, like cleaning the beach alone

Note that of above, most of those are considered shores or busywork... stuff nobody really wants to do, yet they should be done if we want to keep Stack Overflow being useful for the generations to come.


  1. Personally, I consider that SO, since it's already 10 years old, is in a dire need of actually more cleanup efforts. It doesn't make sense that we make the curators debt grow more, as it would reduce the usefulness of SO as whole.
5
  • 1
    Thanks for you answer, make the curators debt grow more, insignificant burnation requests makes it grow since we are not able to focus community resources. Last tag that we voted for burnination was [user], SE (I'm not sure who) basically decide to just remove it this is Shog9 view on this tag. Braiam we have problem with the all the burnation requests, by now we spend time classifying them, is it not curious that we are not even reviewing the questions anymore, but instead the burnation requests?. Commented Apr 18, 2018 at 20:04
  • I have a feeling that we need a strong hand starting to decline all non useful burnation request, since Shog9 does not have time for that anymore. My attempt for now is to fix our FAQ and raise some awareness in the community Commented Apr 18, 2018 at 20:08
  • 1
    @PetterFriberg as long as you have "some vague threshold" you will always perceive a problem. Don't worry about it. As I said, burnination can be done by a single user, that that user is Shog or Petter Friberg or someone else is inconsequential.
    – Braiam
    Commented Apr 18, 2018 at 20:17
  • 2
    most often they are just requested and not done by anyone. Do note, that I have no frustration over this, I do what I think is fun, my meta is only an attempt to discuss; should we upvote "Yeah tag seems bad" or should we upvote "Yes let us do this it's worth our time, we will significantly improve SO". The difference between these two are the final outcome. Commented Apr 18, 2018 at 22:37
  • @PetterFriberg you should upvote if the action will make Stack Overflow better. That is.
    – Braiam
    Commented Apr 20, 2018 at 10:07
6

Whilst burnination is certainly a important part of keeping Stack Overflow tidy, it is only a curative tool. Why don't we take things to the root of the problem:

The creation of tags.

At this point in time creation of tags is easy and straightforward, as per the help page: https://stackoverflow.com/help/privileges/create-tags

It is open to anyone with 1500 reputation points, and only requires those users to enter the tag to be created into the tag box. This creation system could be improved if you ask me.

Currently:

  • on some sites, new tags will be automatically culled and removed from the system if they are not used by at least 1 other question in a 6 month period.
  • meta tags, tags that cannot stand alone as the only tag on a question, are not allowed.

If a tag is only used once in half a year is a very loose criteria if you ask me, if we could reduce this period to a month, or maybe 3 months, I wonder how many tags get removed by the system (Moderator tools/report, anyone?).

Also you can read that Meta tags are not allowed, however no-one will stop you from creating the [gaming] tag.

Improvements

I think the best way would be a thorough redesign of the process of creating tags.

  • Require a tag to have complete info (summary etc.) when being created
  • Have a review like system on tag creation where users can approve, edit, or reject a new tag. Here would be an option to indicate that there already is a tag describing the subject, a tag is too localized, too broad etc.
  • Whilst tag creation is pending allow a tag to be used, this will prevent tags from being requested twice at the same time. However don't allow it to be used as the sole tag on a question. Also make it distinct from accepted tags, either by using a different fill color on the tag, or a distinct symbol pre/super-seding the tag.
  • If a tag is not accepted onto the network, remove all instances of that tag, or retag them with the review concensus on the alternative.
2
  • I'm strongly in favor of a review queue for new tags, with proper guidance on when tag creation is appropriate. I don't know about culling tags, though, since some programming languages may just be so obscure that they might just not get that amount of attention (such as code golfing languages).
    – Erik A
    Commented Apr 20, 2018 at 9:40
  • 1
    @ErikvonAsmuth Ideally culling of tags wouldn't be required anymore when tags are only accepted upon review if I come to think of it.
    – Luuklag
    Commented Apr 20, 2018 at 9:46
-3

I was reading this as I prepared to ask my first meta question on stack overflow. I am kind of a bit naive to say this, but I was wondering with all the coders we have out there why cannot we build support or analysis tools to help people detect tags for burnination?

I am naive because other than the se quality/deduplication project, I don't know how much has been done to develop monitoring systems. Below, however, are my specific ideas:

What harm/confusion does this tag actually do? Why is this not pointless busywork?

The problem with this and the original phrasing is that at least in my mind, we need to make this mathematically or programmatically objective. In other words, could we rephrase this in a way that can be coded?

First, we need a way to get and structure that data. To do this we first would need a way of web-scraping (like Parsehub), monitoring (like Splunk), or API access to the text and metadata of StackOverflow. Edit: We basically need either the server logs (if we go the Splunk route) or each webpages metadata (the text of the question, vote score, timestamp, and text of the tags). I believe this might have already been done from the conversation from here and metastack exchange; however, other then web-scrapping bots I am not sure how much data preparation has been done.

From Petter Friberg: Thanks for you answer, make the curator's debt grow more, insignificant burnation requests makes it grow since we are not able to focus community resources. Last tag that we voted for burnination was [user], SE (I'm not sure who) basically decide to just remove it this is Shog9 view on this tag. Braiam we have a problem with the all the burnation requests, by now we spend time classifying them, is it not curious that we are not even reviewing the questions anymore, but instead the burnation requests?

I think we can possibly reduce the burden through some kind of automation.

Going back to the original quote

What harm/confusion does this tag actually do? Why is this not pointless busywork?

How many time have we received burnination request specifically about this harm/confusion? Because I worry it might be actually rather small, it might be nice to explore the data. We could do this by a kind of weighted (by votes) clustering through k-means (with specifically cosine similarity, jacarrad similarity, or divergence distance metric so we fall into the euclidian error trap of text mining). We set number of clusters rather high when we do this though.

Then after we do an initial clustering. Why can we not build a 1) decision tree or bootstrapped tree classifier and 2) sentiment analysis on the harm/confusion of a question? Then humans review the results of that classifier 50% of there time while the other 50% is dedicated to classifying questions themselves.

We have a training data set already, based on how people have classified and modified questions before. The difference here is that we are building a monitoring system to alert users rather than a bot system that does things automatically.

Does it describe the contents of the questions to which it is applied? And is it unambiguous?

Entity extract might be able to help to objectively monitor this or even something like a TL;DR summarization. We set thresholds based on the residuals or recall of this text mining.

Is the concept described even on-topic for the site?

Anomaly detection and outlier detection like from program like Splunk could be used here as a monitoring system. We do anomaly detection for Stackoverflow but then when its an outlier we do anomaly detection to see what questions might fight in other stack exchanges. The thing would report if it thinks a question that meets anomaly should be moved from one side to the other. We would have to set the threshold of (key statistic) pretty high though.

Does the tag add any meaningful information to the post?

I think that if it's in the summary of a post or is clusters well based on text to other posts with the same tag, then it would be a good way to see if it has meaningful information.

We could also do a separate classifier her based using the work people have done as a training data set.

Does it mean the same thing in all common contexts?

This is one I am not sure how one might define or monitor, but I do think if i understood the question better, it might be easy to define a text mining principle to code.

Finally, we allow the user to set thresholds for these monitoring systems by defining bins (usually quartile bins) of where previous questions were. Splunk actually already does a little of this monitoring in its machine learning packages.

3
  • 1
    Bots are great fun and can be useful, but I'm not really sure we need to add more burnation requests, my discussions was rather to limit them. Also to clarify yes I build bots but the classification of burnation is done by "hand", involving multiple community users, reading and understanding requests, trying to evaluate priority and most appropriate action. My experience tells me that a bot doing this would be near to skynet, but you are free to try. Commented Apr 18, 2018 at 21:20
  • @PetterFriberg I was hesitant to call this a bot (other then webscrapping), because it's more of a monitoring system similar to how splunk works to monitor all web and machine data. Think of it more as a Wall-E net rather than a Skynet because of the lack of the ability to take action only show the results. ISTI already does anomaly detection: splunkbase.splunk.com/app/1841 splunk.com/blog/2017/02/07/… (crap posted early)
    – mlane
    Commented Apr 18, 2018 at 22:15
  • @PetterFriberg ... So just to give an example of how this might be done in Splunk: ISTI already does anomaly detection: splunkbase.splunk.com/app/1841 (monitoring dashboard) splunkbase.splunk.com/app/2890 (Sentiment analysis) docs.splunk.com/Documentation/Splunk/7.0.3/SearchReference/… Kmeans Would your bots be able to get the data? The problem with splunk is, it really would require the webpage or server logs of stackoverflow. And it's not like the webadmin of stackoverflow is going to magically give me .0001% of the server logs for development.
    – mlane
    Commented Apr 18, 2018 at 22:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .