I was reading this as I prepared to ask my first meta question on stack overflow. I am kind of a bit naive to say this, but I was wondering with all the coders we have out there why cannot we build support or analysis tools to help people detect tags for burnination?
I am naive because other than the se quality/deduplication project, I don't know how much has been done to develop monitoring systems. Below, however, are my specific ideas:
What harm/confusion does this tag actually do? Why is this not pointless busywork?
The problem with this and the original phrasing is that at least in my mind, we need to make this mathematically or programmatically objective. In other words, could we rephrase this in a way that can be coded?
First, we need a way to get and structure that data. To do this we first would need a way of web-scraping (like Parsehub), monitoring (like Splunk), or API access to the text and metadata of StackOverflow. Edit: We basically need either the server logs (if we go the Splunk route) or each webpages metadata (the text of the question, vote score, timestamp, and text of the tags). I believe this might have already been done from the conversation from here and metastack exchange; however, other then web-scrapping bots I am not sure how much data preparation has been done.
From Petter Friberg: Thanks for you answer, make the curator's debt grow more, insignificant burnation requests makes it grow since we are not able to focus community resources. Last tag that we voted for burnination was [user], SE (I'm not sure who) basically decide to just remove it this is Shog9 view on this tag. Braiam we have a problem with the all the burnation requests, by now we spend time classifying them, is it not curious that we are not even reviewing the questions anymore, but instead the burnation requests?
I think we can possibly reduce the burden through some kind of automation.
Going back to the original quote
What harm/confusion does this tag actually do? Why is this not pointless busywork?
How many time have we received burnination request specifically about this harm/confusion? Because I worry it might be actually rather small, it might be nice to explore the data. We could do this by a kind of weighted (by votes) clustering through k-means (with specifically cosine similarity, jacarrad similarity, or divergence distance metric so we fall into the euclidian error trap of text mining). We set number of clusters rather high when we do this though.
Then after we do an initial clustering. Why can we not build a 1) decision tree or bootstrapped tree classifier and 2) sentiment analysis on the harm/confusion of a question? Then humans review the results of that classifier 50% of there time while the other 50% is dedicated to classifying questions themselves.
We have a training data set already, based on how people have classified and modified questions before. The difference here is that we are building a monitoring system to alert users rather than a bot system that does things automatically.
Does it describe the contents of the questions to which it is applied? And is it unambiguous?
Entity extract might be able to help to objectively monitor this or even something like a TL;DR summarization. We set thresholds based on the residuals or recall of this text mining.
Is the concept described even on-topic for the site?
Anomaly detection and outlier detection like from program like Splunk could be used here as a monitoring system. We do anomaly detection for Stackoverflow but then when its an outlier we do anomaly detection to see what questions might fight in other stack exchanges. The thing would report if it thinks a question that meets anomaly should be moved from one side to the other. We would have to set the threshold of (key statistic) pretty high though.
Does the tag add any meaningful information to the post?
I think that if it's in the summary of a post or is clusters well based on text to other posts with the same tag, then it would be a good way to see if it has meaningful information.
We could also do a separate classifier her based using the work people have done as a training data set.
Does it mean the same thing in all common contexts?
This is one I am not sure how one might define or monitor, but I do think if i understood the question better, it might be easy to define a text mining principle to code.
Finally, we allow the user to set thresholds for these monitoring systems by defining bins (usually quartile bins) of where previous questions were. Splunk actually already does a little of this monitoring in its machine learning packages.