25

During the discussion about whether to burninate the Kali Linux tag, several people extended the argument that people would continue to ask off-topic questions regardless of whether there was a tag for it or not. In other words, the existence of a low-quality tag doesn't really directly influence how many low-quality questions there are.

I extended the argument in the comments that that's not true: if there's a tag for something, many people (especially new users) will use it as evidence that their question is on-topic here, especially since most people don't read tag usage guidance.

Is there concrete evidence as to who's correct here? Do people use the existence of certain tags as evidence that their low-quality question is on-topic (and therefore post more off-topic questions)? Or would they just find a different tag to use instead?

Just to make this more concrete: the site still has a tag, even though Software Engineering SE is the designated site for methodology questions. Predictably, the tag receives numerous off-topic questions. If this tag didn't exist, would people ask fewer methodology questions here? Or would they just find a different tag and ask their question anyway? Do people see the existence of this tag as proof that methodology questions are on-topic here?

Granted, in this case, the tag usage guidance is very specific that this kind of question is off-topic here. (In fact, I wrote the usage guidance myself). But, as I previously mentioned, the people that need to read tag usage guidance the most (e.g. new users) usually don't.

9
  • 26
  • Somewhat related: if someone attempts to tag a question with a blacklisted tag, do they get a warning that their question is likely to be off topic?
    – Raedwald
    Commented Sep 13, 2018 at 6:09
  • 3
    This also might be the case for theoretical machine learning questions. This recent example shows an OP misled by the existence of tags such as machine-learning, deep-learning and recommender-systems.
    – E_net4
    Commented Sep 13, 2018 at 6:27
  • It looks like that this question can only be answered by someone who can access deleted questions. Solution: Create a smart AI to detect whether a question is off-topic and related to the bad tag, count number of questions, make a graph.
    – user202729
    Commented Sep 13, 2018 at 7:06
  • 1
    I naturally don't have any research but considering that currently the last thing OP adds when asking is the tags, it's to late to give guidance. OP by then have already formulated question and it's really hard to make them choose "Trash this". Most user will probably will add whatever is needed to post the question. Lets hope SE finish the new ask question interface (the demo had tags in the beginning, hence guidance is to avoid question is more "welcoming") Commented Sep 13, 2018 at 7:30
  • Or would they just find a different tag and ask their question anyway? As someone who's relatively experienced in the [php] and [laravel] tags, I often see this. People who started building their first web app literally 5 minutes ago and then tag with every keyword they have ever seen. Their question is about why HTML looks like HTML, so naturally they tag [mysql], [php], [javascript], [css], [apache], [mvc] and so on.
    – Loek
    Commented Sep 13, 2018 at 7:54
  • 5
    @Loek they just tag it with php to warn everyone its a low-quality question ;-)
    – user5940189
    Commented Sep 13, 2018 at 8:07
  • @Orangesandlemons For some reason every programmer I meet thinks PHP is the worst thing ever designed haha. Ah well, everybody is allowed one big mistake in life ;-)
    – Loek
    Commented Sep 13, 2018 at 8:17
  • 1
    No, destroying a tag merely makes it harder to find off topic questions back. And, above all, eliminates my option to add the tag to my Ignored Tags section. That sucks. Commented Sep 13, 2018 at 12:52

2 Answers 2

11

From exhaustive testing and interviewing, we can confidently assert that tagging a question is a process that is very unintuitive to new users. Auto-completing helps people through the process, but taxonomy and nomenclature are the least of someone's worries when all they want to do is ask a question.

Additionally, we ask for tags as a final step in the asking process. This means, should someone not find an available tag that matches their intent, loss aversion alone can compel them to just find something that fits and see what happens. Abandoning efforts put into a title, body, any formatting done, etc would not seem optimal, to say the least.

With that said, it depends. We've had some success using much more descriptive messaging when blacklisting a tag, we've also seen success when warning using a tag, so there's at least some evidence that people are willing to abandon a question if the system makes it clear to them that the experience is not going to be great if they proceed. It's not always the case, though.

And then you have topics where a narrow subset is on-topic (anyone remember lots of discussions about Sharepoint?), questions with historical locks where people don't notice the 'locked' part (sometimes, just because all they see are the titles in search results) ... it's really hard to narrow the brunt of the blame on any single element there, tag included, and it's difficult to show just-in-time help for so many kinds of edge cases.

And then you've got folks that put tags in titles if they can't get them into the tag section, and those go a few hours without editing, get indexed, and 20 more come in. It's honestly a dice roll and the cost of 'doing business' with so many uncontrollable factors.

So the direct answer to your question: We have evidence that just-in-time warning can overcome the loss aversion that would otherwise prevent someone from abandoning a question, despite the existence (or lack thereof) a corresponding tag. However, it's just too hit-and-miss, with too many edge cases to build a solution around.

Does getting rid of bad tags help curtail off-topic questions? That's hard to answer because you can't document a negative. We know as much as we do through events we log when something is abandoned due to help being shown, which is expensive. Anecdotally? Yes. It does help, and certainly won't hurt.

The current strategy of inserting just-in-time help in known problematic tags while keeping them around just for the small subset of questions that are on topic, or just as a warning beacon is better than just accepting that it's just the cost of doing business.

More elegant solutions (like calculating the average score of questions in all applied tags and warning if foo looks like it's universally crapped on) is too much to do on-the-fly right now. But, eventually, the just-in-time help needs decoupled from the human dependency (perhaps offloaded almost entirely to mods and trusted users, but we'd have to overhaul how we implement the blacklist entirely to do that).

Sorry for the rambling response on this, it's something that I've been poking and peeking into more and more over the last few months as I was examining situations where questions were just kinda doomed no matter what happened to them.

5
  • 1
    "TL;dr: removing a tag wont hurt and might actually help"?
    – Braiam
    Commented Sep 13, 2018 at 12:23
  • @Braiam I think I've come to regret the last 11 minutes of my life. Yeah, essentially, but there are some caveats.
    – user50049
    Commented Sep 13, 2018 at 12:28
  • with regards to questions with historical locks where people don't notice the 'locked' part, I wonder how much dev time would it take to just fix their background to get rid of this problem, an hour, or two?
    – gnat
    Commented Sep 13, 2018 at 13:04
  • @gnat I prefer fixing the post, like Tim suggested all those years back.
    – Braiam
    Commented Sep 13, 2018 at 15:29
  • @TimPost, how about moving the tags just after, or even before, the title? Then there is far less too lose for users when they read warnings like: "[foo] looks like it's universally crapped on." and they might be more likely to take action upon that warning.
    – Luuklag
    Commented Sep 21, 2018 at 11:12
0

I think we need to take this discussion back one more step, and put it in the perspective of someone that has a question about something vaguely related to programming.

This user can either A, know about the existence of SO, or B is unaware of the existence of SO.

Case A:

Not much having a tag or not can do about this, these are just lost causes, IMHO. They will ask their question, and will just add (random) tags that vaguely border their case to get going.

Case B

How do these people arrive at SO? Most likely via a search engine. In this case having the tag can make the difference. And for that we should get rid of them when possible.


Side note: It would be interesting to have some statistics on how first-time askers came to be aware of SO.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .