Compressor Should Ignore Private-Use Unicode Ranges

[bdkjones]

Consider this CSS:

.fa-spinner:before {
    content: "\f110"; 
}

Compiling this with the compact or compressed output style results in this:

.fa-spinner:before{content:""}

Because the code point \f110 falls into the "private use; reserved" range, the CSS compressor substitutes a "missing glyph" symbol. It's not visible on GitHub, but it looks like this in an IDE:

Why This Matters

For better or worse, libraries like Font Awesome are using the private-use Unicode range to let users put custom icons in their CSS. (Font Awesome translates the code point into an icon.) Dart Sass's compressor currently breaks this.

It's not just Dart Sass—every CSS compression tool I can find behaves the same way.

Proposed Solution

The CSS compressor should ignore any code points that fall into the "private use; reserved" range so that they are carried through as-is rather than replaced with the "missing glyph" symbol. This better matches the intent of the private-use range, where anything goes—there are, by definition, no defined glyphs that can be substituted for the code point.

[Goodwine]

Sass is not modifying the glyph. Actually if you use an online decoder and copy-paste the same codeblock you shared above you'll see that the glyph actually has been \f110 all along.

The thing you are seeing in your IDE is what the IDE renders that particular character as. In other words everything is working correctly (even your IDE), this is just the way it looks.

[bdkjones]

@Goodwine the issue is the replacement of the string literal "\f110" with the glyph. That breaks in the browser--the Font Awesome icons aren't properly substituted and therefore aren't rendered on the page. The Font Awesome library is looking for that string literal.

This is NOT an issue when the Output Style is set to Expanded, since the compressor does not perform the glyph replacement and the string literal "\f110" appears in the CSS as-is.

My suggestion is that when the string literal is a Unicode code point that falls into the private range, the literal should be left as-is, just as if the Output Style were set to Expanded.

I don't really have a dog in the fight, I'm just tired of people emailing me to complain about Sass breaking their websites and blaming me for the problem.

[bdkjones]

@Goodwine if you're still not going to change the behavior, I'd maybe update the docs with a warning about how this replacement can affect libraries like Font Awesome.

Short of that, I'll just send the angry-email people to this thread and tell them I tried to get Sass on board, but the feature was rejected. Font Awesome and similar libraries are pretty popular and this issue crops up a lot.

My proposed workaround is to use a build tool to manually add these style rules to the compressed CSS file once Sass is done creating it. That's very clunky, but gets the job done as long as you don't have to worry about the cascade and can simply append the rules to the end of the compressed CSS file.

[bdkjones]

@Goodwine we might be getting hung up on terminology, but if you process that snippet of CSS using Dart Sass with Expanded and then Compressed output styles, you'll find that the output is indeed different--Expanded will open in an IDE (and the browser) with the ASCII string literal in place. The compressed output will not. I understand that the character in the compressed output has the correct code-point, but my suggestion is that there should be 5 ASCII characters in the output: [\, f, 1, 1, 0] instead of the single Unicode glyph associated with that code point.

[nex3]

Can you provide a reproduction where rendering is broken in an actual browser?

For additional context: when Sass encounters a non-ASCII character (whether it was originally written as an escape sequence or as a literal character), it will emit that character as the literal character in compressed mode (because that's shorter) and add a UTF-8 byte-order mark at the beginning of the document. The CSS spec mandates that a byte stream beginning with a UTF-8 byte-order mark is always decoded as UTF-8 even if the server is incorrectly configured, and if it's decoded as UTF-8 the literal character is semantically identical to the escape sequence. The only case where this might fail is if some post-processing step strips the UTF-8 BOM—but that's a bug in the post-processor, not in Sass.

We've heard about issues like this periodically for years, but in all that time we haven't seen a single working reproduction. I believe this is because all of those cases were caused by other (broken) tools incorrectly processing the CSS after Sass generated it. #568 tracks the addition of an option to generate ASCII-only output to help work around those broken tools, and if this is a major issue for you you're encouraged to help contribute a fix there. But ultimately, unless you can provide a reproduction, I'm fairly certain Sass is working as intended here.

[bdkjones]

@nex3 Sorry, I missed your response back in December. That explanation makes sense. The trouble is that fontawesome (and similar libraries) don't look for the actual unicode character. Instead, they're scanning the page for the literal sequence of ASCII characters: \f110 and then dynamically replacing those characters with whatever icon has been assigned by the fontawesome people.

It's not that Sass is doing anything "wrong". It's that folks who want to use both compressed output and fontawesome can't do so because Sass won't carry the escape sequence through the compressor.

Because the private Unicode range has no assigned glyphs by definition, that means a browser won't render anything for the literal character that Sass outputs in compressed mode anyway. There is nothing to render. Because of that, my suggestion was: if someone uses the private Unicode range in a Sass file, the compressor should carry through the exact way they used the private range. If they supplied the literal character, bring that through. But if they supplied the escape sequence (\f110), bring that through instead and assume they had a good reason for expressing the character that way. This neatly solves the issue.

I proposed this behavior for only the private range. For other Unicode ranges where there does exist a glyph that a browser would render for a given code-point, the compressor can maintain the current behavior.

[nex3]

@bdkjones Thank you for clearing up exactly where this problem comes from. It makes sense that it wouldn't be a browser issue at all, but actually an issue with another tool.

That said, it's pretty clear that this is a bug in fontawesome. Sass's policy has always been to emit CSS that's spec-compliant, up to the point where that's supported by real browsers. The CSS spec defines \f110 and the literal character U+F110 as as semantically identical, and all browsers support that definition. If fontawesome is parsing CSS and fails to support this equivalence, that's a bug in its parsing, and I recommend filing an issue with that project asking them to better support the CSS spec.

In compressed mode, Sass will always emit the smallest CSS that's equivalent (again according to the spec and what browsers support) to expanded mode—that's the whole point of that mode. Emitting escape sequences may help fontawesome users work around a bug, but for anyone else it'll produce larger CSS files for no benefit. That's not something we're interested in doing, particularly just because of a bug in another tool.

[bdkjones]

Fair enough! It's not my issue; I just deal with the angry support emails.

I do agree that the ideal solution is to have fontawesome recognize both strings. Unicode is hard.