This appears to be an issue exclusively on Chrome for Mac OS X, but for code blocks that contain a single line of code larger than the viewport, when I try to scroll them and the large scrollbar appears, when I mouse away, the bar does not disappear.
I cannot reproduce the problem in Firefox; it uses the same system scrollbars, but they do disappear properly.
This is a usability issue because it renders the code block unreadable until a page refresh.