3

As evidenced by the recent meta post: Introducing a new close reason specifically for non-English questions, a significant number of questions get asked in the wrong language.

Mike Nakis posted some good thoughts on why people would post in the wrong language in the first place: automatic browser translation is so good nowadays that one can perfectly use SO without speaking English.

In a quick test, I confirmed my suspicion that users are currently not made aware of the fact that they are writing in an unsupported language while asking the question in both question editors (traditional and wizard).

You can type a question in a non-English language, e.g. German (see below for demo), and will even get suggested duplicates in that language which in fact encourages an ignorant user to keep typing, as they see that others have asked/answered in their language before.

So here's the proposal: why don't we serve a suitable warning to users when they type in an unsupported language? Language recognition is a solved problem, it doesn't take many words to do it reliably. I could imagine that such a warning could reduce the number of closed questions a lot. At some point one could move from warning to not allowing questions in wrong languages to be asked at all. Like we do with questions that use certain words in the title: Stack Overflow forbids usage of the word `help` in question about CLI `--help` option.

Right now, one can easily type in a different language and not get any warning, why not prevent the user frustration of getting a question closed by hinting that one should write in English.

Sure it may not be that taxing to close these questions. But it's still wasteful for the community and the user formulating their question just for it to be closed immediately.

All sorts of heuristics block submission of questions, like ending a question title with "help?". I can't see how prompting a user that they are writing in a non-accepted language is less useful.

Addressing comments

Ivar pointed out that there was an experiment for something like this in 2016. The logic used back then was extremely simplistic, probably oversimplistic, like using particular Spanish characters and words like "un" with high false positive rate. There have been significant advances in natural language processing since then.

Rene commented that closed questions are not a problem. Closed questions are unnecessary work for both the community (close voters) and OP who wrote a question just for it to be closed. With the same reasoning, there should be no auto-blocking of potentially bad titles/questions.

Andrew T. pointed out that there is prior art here, a community project to detect non-English questions so that they can be closed. Automation is nice, but wouldn't it be better to avoid the need for it by making users aware of the requirement as soon as possible?

MisterMiyagi and Makoto asked about mixed content: English language prose and non-English code/warnings/characters. Any decent classifier will allow tuning of when it fires. One can always start conservative: only show a warning if there is no or almost no English in the question. Also, it's not terrible to get a very occasional false positive warning saying "It looks like you are asking a question in a language other than English. Questions must be asked in English, click here for more details". Just click it away. We don't seem to have a problem with hard blocks of questions with "bad" titles, like if they end in "help?".

enter image description here

20
  • 24
    Language recognition is a solved problem .... I didn't get the memo.
    – rene
    Commented Jan 25, 2023 at 13:18
  • 3
    Closed questions are not a problem. Once closed there is plenty of time to do The Right Thing ™. Getting questions closed in time, is the problem. I don't think adding language warnings is the breakthrough we need to improve new question quality.
    – rene
    Commented Jan 25, 2023 at 13:21
  • 3
    There has been a feature (or at least an experiment) for this in the past: False Positive Spanish Language Detection when Asking Question
    – Ivar
    Commented Jan 25, 2023 at 13:21
  • 2
    @rene There are dozens of libraries/APIs that will tell you the language of even a short snippet of text. It's even inbuilt in most browsers: developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/…. There are also dozens of libraries e.g. github.com/pemistahl/lingua-py, pypi.org/project/langdetect, Commented Jan 25, 2023 at 13:22
  • 1
    Kind of related: Why does Stack Overflow allow to post non English content?
    – Andrew T.
    Commented Jan 25, 2023 at 13:25
  • 1
    @Ivar sure, but that was June 2016. For reference, the paper for GPT-1 was published June 2018, and DeepL has only existed since August 2017. Commented Jan 25, 2023 at 14:59
  • 3
    @rene I kind of wonder if you ever get any memo's at all. You may have an email inbox somewhere that is overflowing.
    – Gimby
    Commented Jan 25, 2023 at 15:01
  • 1
    That said: when I tried to reproduce the scenario in the image, the two questions I saw asked in German had both been swiftly closed (and are from before this meta post). Commented Jan 25, 2023 at 15:02
  • 5
    Also, a related community project: Non-English Language Question Analyser
    – Andrew T.
    Commented Jan 25, 2023 at 15:06
  • 3
    Yep, they were closed, but as a user, I still get the misleading feedback that others have asked in German before. The point is: if we have invested into a custom close reason and documentation, foreign language questions must happen sufficiently often that it may be worth investing a little into a warning. Questions are already rejected based on very simplistic heuristics, I can't see how adding language detection is less helpful than triggering a warning/error on ending a question with "help?" Commented Jan 25, 2023 at 15:06
  • 1
    Re "I can't see how adding language detection is less helpful": Agreed, but it may be anticipation of the company doing the work (it will never happen no matter how simple it would be to implement) vs. the community (unpaid volunteers). Commented Jan 25, 2023 at 15:12
  • 3
    "Language recognition is a solved problem," Is that also for mixed language content? We often get English "prose" and non-English code. That's explicitly allowed (via Meta I think, possibly the help center even). Commented Jan 25, 2023 at 15:27
  • 2
    It would probably be best to wait for the staging ground release to see whether or not this even remains a concern after.
    – Kevin B
    Commented Jan 25, 2023 at 15:48
  • 1
    Related: How do I deal with non-English content? Commented Jan 25, 2023 at 15:54
  • 3
    shock collars maybe?
    – Kevin B
    Commented Jan 25, 2023 at 16:05

1 Answer 1

6

So here's the proposal: why don't we serve a suitable warning to users when they type in an unsupported language?

Because you can't tell if someone is asking a question in Japanese or if they're dealing with a data set that just so happens to contain ひらがな or カタカナ. The amount of false positives that you'd get would far outweigh the benefits of trying to selectively get a user to the "right spot".

Then there's the matter of someone writing their code in their native language, but asking their question in English. That has happened quite often and that would mean that you're going to get some seriously angry folks who are asking about their code - which happens to have Dutch variables - but are writing about their problem in perfectly legible English. That kind of false positive would be highly impactful and detrimental to question askers.

14
  • 3
    The number of false positives can be tuned to be acceptable (look up Receiver operator curve). One can show the warning when there is almost no English in the text. It shouldn't fire as soon as there are a few non-English characters/words. It should fire when it's mostly non-English. Commented Jan 25, 2023 at 15:44
  • 1
    @CorneliusRoemer: The problem is that from the perspective of everyone else, we're already very restrictive when it comes to people asking questions. A lot of this is justified - you can't just dump your work or your assignments here - but this would add a barrier that would need to be tweaked until it is "acceptable", and by that point, the damage would be done.
    – Makoto
    Commented Jan 25, 2023 at 15:46
  • 2
    Is a note that questions must be in English restrictive? If less than 20% of the post are in English, what's the probability it is acceptable? Isn't it more of a barrier if you put in effort and your question gets closed than if you get a gentle note that one must ask in English? If one starts firing only rarely there will be little damage. Commented Jan 25, 2023 at 15:52
  • @CorneliusRoemer While it is possible to tune, the quality of the outcome will depend on the data. Many good questions are primarily code (above 50%, sometimes into 90%) so a filter would need to be very lenient. Is there really a benefit if we only reject, say, questions with less than 10% English? That still leaves massive room for foreign-question-with-English-code to slip through. Commented Jan 25, 2023 at 15:55
  • 9
    The goal should not be to prevent other languages from being in a post. The goal should be to prevent posts from containing next to no English words and phrases. Remove all code blocks and inline code from the question, then check the proportion of remaining words. It wouldn't be perfect, but it'd be better than many tens of unsuitable questions being posted and then having to be closed every day. Commented Jan 25, 2023 at 16:03
  • @CertainPerformance: What questions have you seen that have been unsuitable for the site that are predominately in an unsupported language? Is there really a lot of these questions such that we need to put in heavy automation to deal with this?
    – Makoto
    Commented Jan 25, 2023 at 16:07
  • 5
    @Makoto there was enough of those questions to justify adding a close reason for them. So, I'd say - yes. It's enough to probably justify automation. I've no real expectation anything would happen but still.
    – VLAZ
    Commented Jan 25, 2023 at 16:10
  • 1
    @VLAZ: Sorry, going to disagree there. Of the custom close votes it was highest for sure, but of all close vote reasons, it's not even a tenth of a percent. I think it makes perfect sense to standardize the close reason, which is what I believe the intent of making a formal close reason for them was, but we're not getting so many of these kinds of questions that we have to have some kind of gate-keeping automation to deal with it for us.
    – Makoto
    Commented Jan 25, 2023 at 16:18
  • 4
    On the other hand, I don't really worry too much about displaying a warning to people who have code with variables named in a language other than English. While we do technically allow such posts, they're not ideal and should really be translated into English. A warning is not the same as an outright block.
    – Cody Gray Mod
    Commented Jan 25, 2023 at 16:19
  • 4
    Should we also remove all the other automatic quality controls? Well, whatever we had. I'm going to guess the answer is "no" but I'm not sure why detecting the language of a post is any different. We already don't allow, say, posts with a link to an external code service but no actual code in the question. And that blocks posting the question. Yet a warning that the post should be in English is "gate-keeping"? I really don't follow.
    – VLAZ
    Commented Jan 25, 2023 at 16:23
  • 1
    My bot has been running for months with a pretty high success rate. It does find false positives, but it's only a small percentage. Of course, it doesn't catch them all, but it finds ~20 per day.
    – Dharman Mod
    Commented Jan 25, 2023 at 19:38
  • 1
    I wonder if that is still a thing by the way, code being written in a native language. Especially Dutch. Having collegues from all over Europe is pretty much the norm now so everything has to be English, including documentation and meetings. Student code I guess would still be native.
    – Gimby
    Commented Jan 26, 2023 at 9:17
  • 2
    @Gimby from anecdotal experience - pretty much anybody in the industry writes code in English. The comments might be in a different language and maybe there is a method/variable named after a feature in the native language of the app but usually it's English. However, the questions that show up on SO with non-English code seem to be mostly from beginners. Not sure how many, or if any, of them are self-learners vs not.
    – VLAZ
    Commented Jan 26, 2023 at 9:58
  • 2
    I'm not sure why this would be a problem. If the question is properly formatted, the code and example data sets should be in a code block or snippet (or possibly a table). If those are excluded from the language detection so only the question text is analyzed (with the occasional inline variable or function name), wouldn't that mitigate this issue? Of course this assumes the question is properly formatted, but that could be included in the language warning hint. "Make sure any localized data sets and code are in a code block." or something. Commented Jan 26, 2023 at 10:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .