108

Update 2020-09-24

This is now live network-wide (including Stack Overflow).

Update

This is now live on Meta Stack Exchange and Meta Stack Overflow. Any bugs and feedback can be posted on the main post as an answer.


Note: This post is only an excerpt of the main Meta Stack Exchange post (MSE). See that post for the full details. Please report any issues as answers on the MSE post only.


Hey folks! I’m Ben and I’m a dev on the Teams team here at Stack Overflow - we're the team focused on building the private Teams experience on SO. I’ve recently been working on our post editing experience and I’d like to show off some preliminary work that’s coming to the network soon.

TL;DR

We’re switching our code block highlighting library from Google Prettify to highlight.js. All currently supported languages are still supported and you won’t need to change how you write posts at all. The only major change is how we render highlighted code blocks. In addition, we’re taking this opportunity to introduce our new highlighting theme as well. We’re rolling this out in stages, starting with MSE/MSO with other sites to follow. (See the main MSE post for details)

What’s changing about how I write posts?

Absolutely nothing :). There is absolutely no change to how posts are written. We still support all the existing Prettify language aliases, along with the new aliases from highlight.js.

So what is changing?

The “only” changes are visual. We are updating the client-side code block renderer that styles your code in posts (Questions, Answers, etc) and in the editor preview. Syntax autodetection when a language is not specified should be much better overall, along with syntax highlighting coverage in general. The biggest outward facing change for the typical user is going to be our new theme (see below for details).

How does this affect Stack Overflow specifically?

Code autodetection is a bit different

The most obvious change is that language autodetection is different from Prettify. In general, it will be much more accurate, but will possibly end up with a different result than what Prettify would give us. The original Prettify behavior was that if a language was not specified, it fell back to a generic “C like” implementation. highlight.js on the other hand, attempts to guess which syntax is most likely to apply and uses that one.

Personally, I find that the syntax guessing ends with a much nicer result, but there are some opportunities for false positives, especially in very small code snippets. For instance, the following Javascript code is actually detected as C++:

console.log("Is this cpp?");

(With images since highlight.js is not currently active, as of this post)

enter image description here

Despite this improper classification, the code snippet is still readable and honestly, not all that different from the correct syntax highlighting:

enter image description here

My recommendation: If you know the language, go ahead and explicitly set it. It makes the highlighting run a tad faster and you’re guaranteed to get the language you want.

One thing to note: Language detection based on tags or site-wide defaults will still continue to work exactly as before. This means that adding e.g. the tag to your post will continue to set all unspecified code blocks on that post to use python. Likewise, if the site has a site-wide default language (like https://mathematica.stackexchange.com/), then all unspecified code blocks will take on that site's default language.

Highlighted content is a bit different

Something else to note is that highlight.js tends to not highlight punctuation, which makes it a bit less colorful than other highlighters. This is considered a feature. Not a deal breaker by any means, but something I should mention regardless.

FAQ

  • Q: When is the rollout happening?

    A: Check the FAQ section of the MSE post for the most up-to-date info.

  • Q: What if I find a bug?

    A: Report bugs in an answer (one per answer) on the MSE question. Reporting them there will allow me to keep track of them all in one place and makes triage a lot easier on our end. See the FAQ entry on the MSE post for more details.

9
  • 6
    Is there any awareness as to whether this will also decrease the lag in the Code Snippet editor? Occassionally I've noticed lag spikes which I assumed were related to the syntax highlighting synchronously blocking the UI. Not sure if anyone else has ever experienced this issue, it was relatively minor though. Sep 8, 2020 at 19:14
  • 7
    @PatrickRoberts highlight.js is a bit faster than prettify (see the main post for details), but not significantly so. There is a lag time between keystrokes and preview updating that is up to 2 seconds long, if I recall correctly. This is so the preview doesn't continually try and update itself as you type. There's also an ajax call in there too if you updated any tags since the tag itself may change the default highlighting language. If you are able to consistently repro a lag spike after the rollout, I'd love to hear about it so we can make sure to improve the experience.
    – Ben Kelly StaffMod
    Sep 8, 2020 at 19:27
  • 15
    I'm curious why the most common/frequently-used single-line function in web history (console.log()) is somehow detected as C++ of all things instead of JS. Especially when this is a JS library we are talking about. Isn't the equivalent in C++ cout, anyway?
    – TylerH
    Sep 8, 2020 at 20:40
  • 5
    Hopefully, @TylerH, the move to Highlight.JS will address one of the major failings in the previous use of Prettify, which was the absolute lack of updates (both on Prettify's side, in the failure to accept merge requests for years, and on SO's side, in the failure to ever update to newer versions). Since Highlight.JS is open-source, its implementation can be improved, and doing so can improve Stack Overflow, too. Sep 8, 2020 at 20:44
  • 25
    @TylerH: Because we are talking about syntax highlighting, not semantic highlighting, and name binding is part of semantic analysis. IOW, for the syntax highlighter, console.log and foo.bar are the exact same thing, namely just "someName dot someName". (Actually, most programs that claim to be syntax highlighters are actually only highlighting lexemes, so they should really call themselves lexical highlighters.) Sep 8, 2020 at 21:30
  • 2
    "If you know the language, go ahead and explicitly set it." How do we do this? Using tags? Will that be effective when you have multiple languages in a post?
    – ouflak
    Sep 10, 2020 at 8:43
  • 7
    @ouflak Using the existing mechanism. Sep 10, 2020 at 13:30
  • Comments are not for extended discussion; this conversation has been moved to chat. If you have feedback, complaints, gripes, praise, wishes, bug reports, questions, concerns, or anything else of relevance to say, please post an answer. Comments get too unwieldy. Oct 20, 2020 at 4:13
  • Is it possible or maybe in the future possible to choose out a code-color-theme out of a list of color themes?
    – Aalexander
    Dec 15, 2020 at 12:18

3 Answers 3

53

My recommendation: If you know the language, go ahead and explicitly set it.

Even on posts where the question includes exactly one language tag? So far, language tags overrode the heuristic for detecting the code highlight language. Did that change?

7
  • 21
    I could have been more specific here. Language auto-overrides will continue to work exactly as before - it short-circuits the autodetection feature, essentially setting the language for you. I'll update the post to reflect this. Thanks for pointing this out edit - see my latest post edit
    – Ben Kelly StaffMod
    Sep 10, 2020 at 13:55
  • 4
    So I've tried letting highlist.js autodetect, and explicitly setting the language tag, but for the julia language highlight.js keeps treating it as YAML. Is this issue known already and where should I bring it to your attention? Sep 26, 2020 at 8:24
  • 1
    @RobinDeSchepper A bit late, but you should bring it to the folks over at highlight.js, not to us.
    – 10 Rep
    Oct 8, 2020 at 14:28
  • @10Rep No, that’s a Stack Overflow issue, because Julia is currently not a supported language. Oct 8, 2020 at 14:31
  • @KonradRudolph But the syntax highlighter is what controls the supported languages, no? I'm talking about the auto-detection, not explicitly setting the language, by the way.
    – 10 Rep
    Oct 8, 2020 at 14:34
  • @10Rep Highlight.js supports Julia, but Stack Overflow doesn’t (because while they use highlight.js, they haven’t enabled all language definitions). Oct 8, 2020 at 14:36
  • @KonradRudolph Ok, that makes a little more sense. So a post here only if we don't support the language definition, but a post there if the actual highlighting is wrong.
    – 10 Rep
    Oct 8, 2020 at 14:38
20

This has already been asked on MSE1 but I think it belongs here, given how important this is for SO:

What will the update cycle be?

I’m asking because the current status of highlight.js language coverage seems to be focussed on breadth rather than depth: lots of supported languages, but individual language support of very variable quality.

I’ve just submitted a major PR for the R language lexer. The previous lexer for R wasn’t bad but it was rather threadbare and had a few bugs, and it’d be a shame if Stack Overflow settled for that instead of getting the shiny new R lexer in the near future.


1 If the duplication is considered a nuisance feel free to downvote/delete!

3
  • 6
    You can refer to my comment on that post for my current official answer. To expand on that, we'll only be updating to tagged releases of highlight.js. This is important to note because if (for instance) a major language parser update was to land and a request was created on SO to update, but there wasn't another highlight.js release for another n weeks that included it, then the soonest it would land on SO would be after the overall release, not immediately on request.
    – Ben Kelly StaffMod
    Sep 11, 2020 at 17:43
  • 6
    @BenKelly Thanks for the reply. Just for the record, the policy is obviously totally fine — the idea isn’t to constantly update, just also not to get stuck at one version when there are significant updates. Sep 11, 2020 at 21:01
  • @BenKelly For sure I'd recommend using release versions. :-) I think the hope is that you would simply update soon after we make new minor releases... perhaps even without a specific request... our goal is typically a 6 week release cycle, though 2020 has been a bit off. Oct 19, 2020 at 20:59
11

Help (https://stackoverflow.com/editing-help#syntax-highlighting) has not been updated.

enter image description here

Also What is syntax highlighting and how does it work? could be updated.

6
  • 8
    It's only live on two sites, so we haven't updated it yet. We'll be sure to update once the changes are out of testing.
    – Catija
    Sep 16, 2020 at 12:52
  • 3
    Has there been a change in how we specify a language for a code snippet?
    – einpoklum
    Sep 21, 2020 at 11:04
  • 1
    @einpoklum The old abbreviations still work, but new ones from highlight.js should work as well (github.com/highlightjs/highlight.js/blob/master/…), I guess. Sep 21, 2020 at 11:49
  • 3
    @Trilarion: And these all need a lang- prefix, right? Sorry for making you speel this out. TBH, I usually forget to specify the language and just leave it the automatic magic to do highlighting.
    – einpoklum
    Sep 21, 2020 at 12:09
  • 1
    @einpoklum Yes, and there is a noticeable delay until the highlighting appears. I also usually leave it to the automatic deduction from the tags. That should work in almost all cases. Sep 21, 2020 at 13:50
  • 3
    @Trilarion So are we losing the ability to specify language without lang-? Or will it just be required for the new ones?
    – zcoop98
    Sep 22, 2020 at 22:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .