91

Articles on the misuse of bidirectional text have been making the rounds recently. Put simply, this abuses the concept of bidirectional Unicode text to compile something different from what it appears to be to a human.

For instance, the following little snippet:

#include <stdio.h>
int main() {
    int admin = 0;
    /*‮ } ⁦if (admin)⁩ ⁦ Verify admin: */
        printf("You're an admin.\n");
    /* End check ‮ { ⁦*/
    return 0;
}

When compiled and run this will show "You're an admin" despite looking like it shouldn't because of the if check. While there is a hint if you look closely at the syntax highlighting here, or if you drag a browser's highlighter over the text it might act odd, it's easy to miss.

I very much doubt this is a concern right now, but should Stack Overflow follow GitHub's actions and warn if a code snippet contains such control characters to prevent any possibilities of unexpected side effects of copy-pasting code?

20
  • 3
    Honestly, unless this becomes a big deal, I think our existing curation mechanisms do a fine job of downvoting and deleting answers with malicious (or just poorly formatted) code already. People that are in the know read it and vote accordingly; I don't think we need a special mechanism for taking care of these cases (even disregarding the dev hours creating a tool to do this would take, which would be very nontrivial; GitHub already has code-scanning mechanisms in place for a tool like this to plug into).
    – zcoop98
    Nov 1, 2021 at 21:42
  • 22
    On the other side, this would substantially increase the rate at which blatant copy-pasta is detected
    – Zoe is on strike Mod
    Nov 1, 2021 at 21:43
  • 22
    FWIW as soon as I copy and paste this, it makes it blatantly obvious that it would print "You're an admin" without me even having to run it... So your concern is really if people can't read, which isn't something we can do much about. An editor without proper unicode support makes it even more obvious Nov 1, 2021 at 21:45
  • 17
    Some editors also explicitly display codes like that, as opposed to hiding them (in which case it's abundantly obvious what's going on)
    – Zoe is on strike Mod
    Nov 1, 2021 at 21:48
  • 7
    Is there any reason to allow bidi characters at all? They are mainly useful for mixing LTR and RTL languages, which isn't very useful on an English-only site. Bidi characters should just be disallowed entirely.
    – loops
    Nov 2, 2021 at 1:16
  • 3
    Questions like this one seem perfectly fine to me. I don't know if the best way to answer it is with bidirectional markers, but it might be. Nov 2, 2021 at 1:27
  • 41
    @Nick: The fact that it looks obvious in two specific editors doesn't mean it's universally not a problem. This is especially true if the malicious segment is buried within a larger block, making it less obvious even in an editor that doesn't respect the control codes. Nov 2, 2021 at 1:53
  • 14
    @Smitop One might develop programs that are not English-only, and have to deal with these characters, for example in string literals. It's entirely imaginable someone could have a programming question about these characters.
    – Erik A
    Nov 2, 2021 at 8:09
  • 7
    @ErikA "It's entirely imaginable someone could have a programming question about these characters." A warning wouldn't take away anything from this. Basically a warning about funny characters in text that might not be visible right away or misleading would be nice, I think. Nov 2, 2021 at 11:58
  • 5
    @ErikA if someone were to have a question about these characters, wouldn't it make more sense to show the characters in some escaped form? there's no way in current form one could see if those characters are in use or not, and I don't see why we should support questions where the point would be about testing stackoverflow rendering support for these characters
    – eis
    Nov 2, 2021 at 12:01
  • 3
    @Trilarion I think ErikA was commenting on Smitop's question why we should allow them at all
    – eis
    Nov 2, 2021 at 12:01
  • 2
    Even though detecting these characters might be trivial for a savvy user, I think it would be a disservice not to warn visitors that a code snippet contains characters that can change the obvious meaning of the code. I think either displaying a warning or literally displaying the characters rather than interpreting them would suffice. I don't see a reason these characters need to be in code outside of being present in a string. Any non-english comments should be translated or removed from code on this site in my opinion.
    – spicy.dll
    Nov 2, 2021 at 15:31
  • 2
    Reminds me of meta.stackoverflow.com/q/361390/3702797 I guess a solution for one would also apply for the other.
    – Kaiido
    Nov 3, 2021 at 0:18
  • 1
    @ErikA that doesn’t require supporting bidi (and other Unicode features) outside of string literals or comments. As the term embedded suggests, an embedded r2l sequence should not span multiple tokens.
    – Holger
    Nov 3, 2021 at 8:25
  • 2
    @ErikA since stackoverflow does already have syntax highlighting for a lot of programming languages, it also has an understanding of the tokens. However, I see an even deeper design issue of the programming languages itself to allow such confusing Unicode features. Like Java’s infamous \u.... character references which can change the semantics of the tokens itself like with "a string\u0022 \u002f* a comment */
    – Holger
    Nov 3, 2021 at 11:35

3 Answers 3

44

Rather than a warning – which might be gamed by some bogus reason why it's there in prose – or outright blocking – which might block valid questions or answers – invisible unicode characters, I suggest we display code in some way that makes the presence of invisible unicode characters obvious1, like the common editors do.

That might even help the readability of some posts, as it makes it more obvious to the naked eye what exactly the code contains, even in legitimate use cases.

1Except for tabs, spaces, line breaks of course

16
  • 4
    In legitimate questions this turns into noise real quick.
    – Makoto
    Nov 2, 2021 at 18:21
  • 41
    @Makoto Does it? Are there really that many legitimate questions involving invisible unicode where seeing the invisible unicode is actually noise?
    – Baum mit Augen Mod
    Nov 2, 2021 at 18:23
  • 1
    There are a few where seeing the invisible unicode could ruin the spirit of the question
    – Kevin B
    Nov 2, 2021 at 18:23
  • 34
    @KevinB "Spoiling" that sort of trick question is not that big of a deal IMO.
    – Baum mit Augen Mod
    Nov 2, 2021 at 18:28
  • 3
    In particular, I think it in addition to making their presence obvious, it should also nullify their effect (like less does), e.g., I'd want the malicious example to look like if (accessLevel != "user😈 😈// Check if admin😈 😈") { instead of if (accessLevel != "user😈") {😈 // Check if admin😈😈 . That way you obviously notice "hey, that thing that looks like a comment is actually in the string literal!" and not just "weird, there's a control character in the string literal and a few more control characters in the comment." Nov 3, 2021 at 0:33
  • 3
    I agree that these should be made visible but whatever solution is used to make these visible probably should not alter what is being copied. I.E this should only alter the "rendered" text, not the actual content.
    – Kaiido
    Nov 3, 2021 at 1:13
  • 1
    I disagree with this answer, I don't think Stack Overflow should invent a new way to handle these Unicode characters. Either there are allowed (with or without warnings), or not, but don't make Unicode even more complicated by inventing new behaviour for characters.
    – Flimm
    Nov 3, 2021 at 7:11
  • 14
    @Flimm I would not call displaying Unicode in code snippets in a way common code editors display Unicode in code inventing "a new way to handle these Unicode characters". Rather, I suggest we align our handling of Unicode with the existing, and IMO better, solutions.
    – Baum mit Augen Mod
    Nov 3, 2021 at 8:27
  • 4
    Some editors even show tabs, spaces, and line breaks which is not bad, when there are more than one white-space character in a row and it may be relevant to the program logic. When the marker characters are show with low contrast, it’s not distracting. If you want to be really good, you’d allow the interpretation as long as the r2l sequence is completely contained within a single comment or string literal token. This still doesn’t cover all Unicode tricks though. U+00C4 and U+0041U+0308 denote the same character from text processing (i.e. a brower’s) perspective but different identifiers.
    – Holger
    Nov 3, 2021 at 8:35
  • A much lower effort, I suspect, would be adding a warning banner to the top or bottom of the code block.
    – VGR
    Nov 3, 2021 at 20:29
  • 3
    @Makoto On the contrary. I have an answer that has a legitimate reason for using these characters and it's the fact they're invisible that creates noise real quick: lots of explanations of where the crucial invisible characters are. If SO's syntax highlighting made these visible somehow (wrapped in a span with a width and a background image?), the answer would be more readable. Nov 4, 2021 at 9:43
  • 5
    Seems reasonable, but I also hear you saying I̠̫̪ͩ̋̑ ̟̹̟͙̯̖̙̉̆̊̈͋̇̚ha̎t̠̹͚͈͉e ̯̹̺͖̻̰͉̌ͧ͛͂ͥ̂͗f̼̻̱͍̞̖̊ͭ̀ͥ͑͑u͓̙̪̒ͤ̈́n͕̗̦̯̙̟̏̇ͨ̏͛̅̾ͅ. Nov 4, 2021 at 15:41
  • 1
    We do, of course, @Michael. Nov 4, 2021 at 18:39
  • @user56reinstatemonica8 But couldn't that usage easily be replaced with &rlm; and &lrm; in a Stack Snippet? Nov 5, 2021 at 4:20
  • 1
    @JosephSible-ReinstateMonica Not really because those are HTML entities and that answer isn't about HTML, it's React Native which doesn't decode HTML entities (unless you manually run HTML entity strings through a third-party library). Maybe it would work for SO's UI, but then the answer would contain code that doesn't work in real usage. Nov 5, 2021 at 9:29
1

...no?

Going to go on a wildly controversial limb here.

  • We know that people copy and paste code from Stack Overflow. It's kind of an endemic at this point.
  • There's little that warnings actually do to prevent people from just plopping code from some site on the Internet right into their editor and shipping it out into production.
  • The developer is ultimately responsible for the quality of their code, and it falls on them - not us if they ship crappy code.
  • (mumbles something about legal and licensing here). I think there was an instance in which someone had to rewrite their entire Git history to rip that code out because it was so poisonous to their license and business model.
  • Oh, also valid questions about bi-directional text do exist. I didn't think this needed saying but I'll say it for completeness.

So personally I'm seeing this as something to use to our advantage, within reason.

Most modern editors do seem to come with something that detects this on the behalf of the user, and it's not unfair to expect some amount of testing - either unit or acceptance - on code that is written. Yes, even homework code is executed or tested.

We don't have a warranty or guarantee of fitness for the code that is written, and this would serve as a far louder wake-up call to the folks that want to copy-paste stuff rather than the easy-to-dismiss alert/warning notification that was built over April Fool's.

Yes, I know this is somewhat evil, but at some point, we have to let go of their hand.


A quick addendum: this exploit is reliant on potentially dangerous code near potentially innocuous code, like an in-line comment or code that isn't commented properly. I'm not saying this to minimize the threat, but I really want to highlight that the biggest attack vector of this seems to be the programmer themselves, as they may hastily copy code from one part of the Internet and another into their own source.

28
  • 12
    I found the Docker/Razor copypasta incident pretty funny too. But yeah, agree with this answer.
    – DavidG
    Nov 2, 2021 at 17:26
  • 4
    I don't think anything short of, say, a mass self-driving car hack that kills 50 million people, would be enough of a "wake-up call" to get people to stop copy-pasting off Stack Overflow (and a hack like that would trigger a lot more panic, security theater, and counterproductive overreactions than useful reactions). Nov 2, 2021 at 18:26
  • 9
    "it's not unfair to expect some amount of testing - either unit or acceptance - on code that is written" - unit tests are not going to catch a malicious backdoor. Nov 2, 2021 at 18:28
  • 1
    Also, the syntax highlighter will highlight it as code not comment.
    – Joshua
    Nov 2, 2021 at 20:35
  • 1
    @user2357112supportsMonica: I find it hard to believe that you couldn't write a unit test to validate the boolean logic that is used as an example for the exploit.
    – Makoto
    Nov 2, 2021 at 22:58
  • 10
    @Makoto: The example in the question is easy to detect because it's deliberately obvious. It's a demonstration, not a serious attack. A serious attack wouldn't just unconditionally produce obviously wrong output. Nov 2, 2021 at 23:03
  • 1
    I agree that editors should be displaying warnings when the user pastes code with these special Unicode characters in.
    – Flimm
    Nov 3, 2021 at 7:12
  • 2
    @Makoto: You're not being creative enough. An attack doesn't require an obvious "do bad things" instruction. Imagine someone using directional overrides to, say, subtly alter a computation that decides how much memory to allocate, resulting in an exploitable buffer overflow under circumstances unlikely to come up naturally or in testing, but easy to trigger if you know how. Nov 3, 2021 at 17:00
  • 1
    ...and all of those cases can and should have some kind of testing against it, @user2357112supportsMonica. It'd be different if this were some kind of remote code execution exploit or even an exploit that explicitly rewrote something to then cause permanent damage, but the damage is really limited to whatever's near the lexer's commands. I'm not saying it's not a problem. I'm saying that it's not our problem to fix.
    – Makoto
    Nov 3, 2021 at 17:03
  • 1
    Framed in another mentality @user2357112supportsMonica, I can't see this too much different from running an arbitrary base64-encoded string on your system, which does require some level of guarding against.
    – Makoto
    Nov 3, 2021 at 17:06
  • 3
    @user2357112supportsMonica: Yes, I intentionally put more stock and faith in others' ability to test their code. Not because I want to, but because that's really where it belongs. If another software developer legitimately cannot test their code, or a company legitimately cannot validate their own running application for correctness, accuracy or safety, it is not by any means or stretch of the imagination the responsibility of complete and uninvolved strangers on the Internet to then do that much for them.
    – Makoto
    Nov 3, 2021 at 17:18
  • 4
    @Makoto: A test case is not going to distinguish "this function properly initialized its random array" from "this function used random-looking but actually sensitive stack garbage". A test case is not going to find a backdoor deliberately designed to trigger under highly-specific conditions that are unlikely to be tested. Testing is not a panacea for malicious code. Nov 3, 2021 at 17:25
  • 1
    @user2357112supportsMonica I'm all for not allowing users to shot themselves in the foot, but they should bear some responsibility into understanding what they are doing and why. I can't know who will use my answers or for what purpose, but I try to write them correctly. If someone decides not to write correct answers deliberately we should deal with that user specifically, not just put a blanket solution for a very specific and out on the bunnies problem.
    – Braiam
    Nov 4, 2021 at 15:16
  • 1
    @Braiam: Even if people read and understand the code they're using, that's not enough, because the code they read and the code they run are different code with this attack. They read innocuous code, and then they run something malicious. It's not fair to users to expect them to know about and check for this attack, or to retype code they understand manually instead of copy-pasting - the problem with copy-pasting code you don't understand is the "don't understand" part, not Ctrl-C Ctrl-V. Nov 4, 2021 at 18:09
  • 2
    @user2357112supportsMonica: If you could actually link to an example repository that demonstrates this being an actively malicious attack that offers absolutely no conventional way for a developer to either validate or test against it, or offers absolutely no conventional way for a QA person to perform acceptance testing against it, that would bolster your claims. Without that, I'm really reluctant to see this debate with you continue, since the premise you give us is that the burden of security and correctness with code copied and pasted from here is that it's fit for purpose.
    – Makoto
    Nov 4, 2021 at 18:24
-4

The lowly built-in notepad editor for windows shows this: (screen scrape instead of copy/paste)

enter image description here

I believe it would be wise for stackoverflow to sanitize code pastes since most* programming languages are latin and left-to-right read by the compilers...

5
  • 2
    If I write my code in a non-Latin language that does require RTL or LTR markers (like Hebrew), then "sanitization" may result in wrecking the source code I'm posting.
    – Makoto
    Nov 4, 2021 at 16:50
  • Further note to other commentators - the question and the problem statement have to be written in English as this is an English speaking site, but we've never encountered an issue with code being written in a non-English native language before.
    – Makoto
    Nov 4, 2021 at 16:50
  • 4
    While interesting, that is not a reasonable sanitization of the code. It doesn't show some bidirectional markers, and rendering the text backwards for some reason. Nov 4, 2021 at 17:30
  • 3
    "we've never encountered an issue with code being written in a non-English native language before." I'm sorry, what? I encounter this issue on a regular basis. So does everyone in Charcoal. @Makoto Nov 4, 2021 at 18:40
  • @CodyGray: Is that actually an issue? As in, someone posts code in Dutch or French (or Hebrew), but their question otherwise is in English? I've only happened by it maybe once or twice and I don't really recall a lot of fanfare around it.
    – Makoto
    Nov 4, 2021 at 18:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .