23

This seems like a bug to me...

enter image description here

...not because I was able to type 14 characters and use the RTL override to add the last character (a long-standing "thing" that has been around for a while but likely doesn't work anymore)...but because it impacted everything else (except for the timestamp - kudos on that at least!) on the line.

The override should be overridden beyond the actual text; that is, don't allow the RTL override to dominate the rest of the comment line.

The web fix (for the curious): add the dir="ltr" attribute to just the comment-copy-denoted span. We're already doing it for comment-date.

I've also observed the same behavior in the Stack Exchange app, with the caveat that the entire line is jacked up.

enter image description here

12
  • 51
    ‮PLEASE DON'T FIX THIS
    – BoltClock
    Nov 13, 2015 at 17:46
  • 9
    If only my username were palindromic.
    – ryanyuyu
    Nov 13, 2015 at 17:51
  • 26
    @BoltClock My mind wanted to read “Please don’t fix shit” there…
    – poke
    Nov 13, 2015 at 18:22
  • @poke: This is because you read from the left first. ;)
    – Makoto
    Nov 13, 2015 at 18:23
  • @BoltClock: Darn it, this makes me not want to get this bug fixed now if only for the hilarity...
    – Makoto
    Nov 13, 2015 at 18:23
  • I wonder if it works on answers...
    – Stryner
    Nov 13, 2015 at 18:24
  • I thought I had seen something about the RTL messing up comments on MSE. The post I was thinking of RTL text can mess up comment timestamps which is not exactly the same issue, but has a few other fun examples of what you can do with this. :)
    – Kendra
    Nov 13, 2015 at 18:24
  • 5
    @Stryner: You can place those characters in answers, absolutely. There are scenarios in which that sort of character is even justified or required, specifically on sites which have RTL character sets (Jewish, Arabic, etc).
    – Makoto
    Nov 13, 2015 at 18:24
  • Wow, people are talking siht all over the place in that other question! ;)
    – Alan Moore
    Nov 14, 2015 at 14:21
  • Comment rather than answer since it doesn't address the bug. But have you thought about changing your behaviour to make the RTL unnecessary? Something like 'What commands? There are no commands here.' or 'What commands are you referring to?'. The whole point of the min size is to ensure comments are substantial enough to be useful so bypassing the rules with trickery seems ... wrong to me.
    – paxdiablo
    Nov 14, 2015 at 21:43
  • @paxdiablo: You're right in that it's a bit of an underhanded move when it comes to comments, but there are those select and rare occasions when a terse comment or answer is justified. I just so happened to come across one of those scenarios. In either event, someone was bound to stumble on this issue, and it is one that should be fixed.
    – Makoto
    Nov 14, 2015 at 22:58
  • Related problem when mentioning rtl username in comment: meta.stackoverflow.com/questions/304127/… Nov 15, 2015 at 19:24

2 Answers 2

7

This issue has been previously reported on MSE, and was (briefly) fixed by applying the unicode-bidi: isolate style on the comment text. Unfortunately, it seems that this change triggered a bug in Safari and was reverted.

An alternative solution would be to reset the text direction by automatically appending suitable Unicode BiDi control characters to the comment text. Alas, due to the incremental complexity of the Unicode standard, this is not quite as simple as just appending a single BiDi reset character. That said, while I have not fully tested this, I believe the following rules should work, with minimal side effects, on all browsers:

  1. If the comment text contains any unclosed directional isolates (i.e. LRI (U+2066), RLI (U+2067) or FSI (U+2068) not followed by a corresponding PDI (U+2069)), append enough PDI characters to close them.

  2. Iterate over the comment from start to end. At each PDI character, and also at the end of the comment, if there are any unclosed directional embeddings or overrides (i.e. LRE (U+202A), RLE (U+202B), LRO (U+202D) or RLO (U+202E) not followed by a PDF (U+202C)) within the current isolation level (i.e. after the corresponding LRI / RLI / FSI character, or after the beginning of the comment, if there isn't any), insert enough PDF characters to close them. Remove any extra PDF characters that don't have a corresponding LRE / RLE / LRO / RLO within the current isolation level.

  3. After the previous step, remove any PDI characters that don't follow a corresponding LRI / RLI / FSI character. (Running step 2 first ensures that the secondary effect of PDI, i.e. terminating any open embeddings and overrides, is preserved.)

  4. Finally, to ensure correct directionality of the dash, timestamp and username after the comment, a Unicode Left-to-Right Mark (U+200E, HTML ‎) should be inserted after any comments that may contain strong right-to-left characters. (Note that all Unicode characters with codes below the Hebrew block, starting at U+0590, have left-to-right or weak / neutral directionality, so all this processing may be safely skipped for comments containing only those characters.)

The reason for the complex rule 2 is that, while the directional isolates (LRI / RLI / FSI / PDI) introduced in Unicode 6.3 are supposed to take precedence over the embedding and override characters (LRE / RLE / PDF / LRO / RLO) from earlier Unicode standards, not all browsers necessarily support them yet. The rules above should ensure that any embedding and overrides are consistently terminated on all browsers. The rules above also ensure that no directional isolates or embeddings / overrides will be introduced unless the comment already contains them, as browsers that don't support those control characters might render them in unwanted ways, e.g. as ugly boxes.

The following JavaScript code should implement steps 1–3 of the rules described above. While the rules above are formulated as several passes over the comment text, the JS code processes the text in a single pass, using a stack of active directional control characters.

// ensure that all BiDi embeds / overrides / isolates are properly closed
function sanitizeBiDi (str) {
    var PDF = "\u202C", PDI = "\u2069";  // Pop Directional Formatting/Isolate
    var stack = [];  // stack of pending PDF / PDI marks

    str = str.replace( /[\u202A-\u202E\u2066-\u2069]/g, function (chr) {
        if (chr === PDF || chr === PDI) {
            var rv = "";
            // PDI always terminates all unclosed embeds / overrides
            if (chr === PDI) {
                while (stack.length > 0 && stack[stack.length-1] === PDF) rv += stack.pop();
            }
            // skip this PDF/I unless we've seen the corresponding opening mark
            if (stack.length > 0 && stack[stack.length-1] === chr) rv += stack.pop();
            return rv;
        } else {
            // push corresponding closing mark onto the stack
            stack.push( /[\u202A-\u202E]/.test(chr) ? PDF : PDI );
            return chr;
        }
    } );
    // close any remaining open embeds / overrides / isolates at the end
    if (stack.length > 0) {
        stack.reverse();
        str += stack.join("");
    }
    return str;
}

This code could be optimized further, but I believe it should be more than fast enough already, and further optimizations might hurt clarity. (In particular, the stack.length > 0 checks are technically unnecessary in JavaScript, but may be needed in other languages, and including them hopefully makes the intent clearer.)


Ps. I've included this code in my SOUP user script, as of v1.42. SOUP already includes a CSS fix to isolate the directionality of comment text from surrounding content, but this code further ensures that comments posted by SOUP users cannot accidentally mess up the display for others.

The code currently in SOUP also appends a Left-to-Right Mark (U+200E) to any comments that might end in a strong RTL character, to ensure that the trailing spaces and dash will be rendered correctly. The regexp check used to detect this is somewhat crude, and may have some false positives, but that's OK — needlessly appending the LTR mark should have no harmful effects, except for wasting a few bytes from the comment length limit.

The specific regexp used is:

/([\u0590-\u08FF\u200F\u202A-\u202E\u2066-\u2069\uD800-\uF8FF\uFB1D-\uFEFE][^A-Za-z\u200E]*)$/

That is, it matches any UTF-16 code point in the following ranges:

  • U+0590..08FF: Hebrew, Arabic, Syriac, Thaana, NKo, Samaritan, Mandaic
  • U+200F: Right-to-Left Mark
  • U+202A..202E: BiDi embedding / override
  • U+2066..2069: BiDi isolation
  • U+D800..F8FF: Surrogates and Private Use Area
  • U+FB1D..FEFE: Hebrew and Arabic presentation forms (excluding U+FEFF = Byte Order Mark)

that is not followed by an ASCII letter (A to Z) or the Left-to-Right Mark itself (to avoid duplicate LTR mark insertion when editing comments). Obviously, this may occasionally match when it really shouldn't, e.g. for comments containing Arabic text (RTL) followed by Cyrillic (LTR, but not detected as such by the regexp), but in practice this should be rare and, as noted above, mostly harmless.

Obviously, this behavior may need to be adjusted if SE ever launches a site with an RTL interface (Stack Overflow al-'Arabiyyah, anyone?), but that'll have to wait until it actually happens, and we see how comment styling works on such a site.

2
  • 2
    This seems like it could help others with a similar problem. Consider posting a self-answered question (or answering an appropriate existing question) on the main SO site. Nov 14, 2015 at 21:00
  • ‮This is a test.‬‎ Dec 2, 2015 at 13:38
3

For the mobile apps, I'm doing the lazy fix of just inserting 'POP DIRECTIONAL FORMATTING' (U+202C) between the comment and the suffix. Per Ilmari's answer this is probably an incomplete solution if someone were to insert multiple unmatched RTL characters, but it's probably "good enough" for the stray use case.

  • This will be fixed in 1.0.84 of the Android app.
  • This will be fixed in 1.4.5.0 of the iOS app.
7
  • Does this also impact the main site? Or has that already been addressed? (It's been a while since I've bothered checking.)
    – Makoto
    Mar 11, 2016 at 17:44
  • @Makoto Looking at BoltClock's first comment, the web appears to be fine. Mar 11, 2016 at 17:46
  • Well, somewhat. It says "tide" instead of "edit" Mar 11, 2016 at 17:49
  • 1
    ‮Disregard this text; this is only a quick smoke test.
    – Makoto
    Mar 11, 2016 at 17:51
  • Nope - main site still shows "tide" when using that form. At least the date is fine, but neither the edit text nor my name are correct.
    – Makoto
    Mar 11, 2016 at 17:51
  • @Makoto Okay, I didn't realize this was directed at the whole site too. (When I saw the android-app tag, my brain automatically narrowed to that scope.) I'm going to remove android-app and status-completed so it gets the right attention. Mar 11, 2016 at 17:54
  • Well, the android app was an additional, "Oh hey, this doesn't quite seem right here, either." But alright, thanks for covering that.
    – Makoto
    Mar 11, 2016 at 17:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .