Stop emitting non-standard "; charset=binary" suffix - #1383
Closed
silverbucket wants to merge 3 commits into
Closed
Conversation
WireClient.put and Dropbox.put appended "; charset=binary" to the
Content-Type header for ArrayBuffer / binary uploads. The string "binary"
is not a registered IANA character set (RFC 6657), and the suffix has no
effect on transport — XHR/fetch sends the body bytes verbatim regardless.
The suffix is, however, sticky: servers (5apps, Armadietto, anyone storing
the request Content-Type verbatim) write it into stored metadata and echo
it back on GET. Some browsers, including Chrome, refuse to render an
<img> whose Blob type carries the "; charset=binary" suffix, producing a
valid-looking blob: URL that never paints. Apps that pass getFile()'s
contentType straight to a Blob constructor hit this.
Fixes the emit side; leaves the read-side detection alone:
- WireClient.put: drop the suffix-append (and its now-orphaned
isArrayBufferView import).
- Dropbox.put: same removal.
- shouldBeTreatedAsBinary: keep the charset=binary check with a
backward-compat note. Files uploaded with releases <=2.0.0-beta.9
still carry the suffix in their stored Content-Type and need to be
detected as binary on read; the existing control-char heuristic
handles new uploads.
- BaseClient docstring: update example folder listings to show clean
Content-Type values (image/png, not image/png;charset=binary).
- Tests: rename the two PUT charset assertions to verify the new
behavior (no suffix added; caller-supplied charset preserved).
Closes remotestorage#1379
…tType Stops the suffix from leaking through getFile().contentType to consumers when the server still has it baked into legacy file metadata. Files uploaded with releases <=2.0.0-beta.9 carry "; charset=binary" in their stored Content-Type forever — servers (5apps, Armadietto) write the request header verbatim and echo it back on GET. Apps that pass that contentType straight to a Blob constructor hit the same Chrome <img> rendering issue described in remotestorage#1379, even after upgrading to a fixed library version, because the data on disk is the legacy shape. Sanitize on read so callers see clean Content-Type values regardless of when the file was written. Internal binary-vs-text detection still uses the original (pre-strip) header, so shouldBeTreatedAsBinary keeps firing correctly on legacy responses. - util.ts: add stripLegacyCharsetBinary helper. Removes only the `; charset=binary` parameter — legitimate ones like `charset=UTF-8` on `text/html` are preserved. - wireclient.ts, dropbox.ts, googledrive.ts: apply the strip at the response boundary in each remote's GET handler. - tests: update the existing wireclient/Jaribu assertions to expect the cleaned contentType, add a regression test that legitimate charsets are not stripped. Refs remotestorage#1379
Member
|
@silverbucket Could you reopen the PR from this repo, please? I have added the same fix for cached documents to |
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1379. Supersedes #1380 (closed because the head branch was renamed to follow the project naming convention; no review comments to lose).
WireClient.putandDropbox.putappend; charset=binaryto theContent-Typeheader for ArrayBuffer / binary uploads. The stringbinaryis not a registered IANA character set (RFC 6657), and the suffix has no effect on transport — XHR/fetch sends the body bytes verbatim regardless of thecharset=parameter.The suffix is sticky: servers (5apps, Armadietto, anyone storing the request
Content-Typeverbatim) write it into stored metadata and echo it back on GET. Some browsers — Chrome among them — refuse to render an<img>whoseBlobtype carries the suffix, producing a valid-lookingblob:URL that never paints. Apps that passgetFile()'scontentTypestraight to aBlobconstructor hit this.Approach
Two commits, two halves of the fix:
1. Stop emitting the suffix
src/wireclient.tsput()(and the now-orphanedisArrayBufferViewimport).src/dropbox.tsput().src/util.tscharset=binarybranch inshouldBeTreatedAsBinarywith a comment explaining it's there for backward compat with files uploaded by<=2.0.0-beta.9. The existing control-char heuristic handles new uploads.src/baseclient.tsContent-Typevalues.test/unit/wireclient.test.mjs2. Strip the suffix on the read side too
Stops the suffix from leaking through
getFile().contentTypefor files that already exist on servers with the legacy shape baked into their stored metadata. Without this half, apps would still see broken<img>rendering for old files even after upgrading to the fixed library version.src/util.tsstripLegacyCharsetBinaryhelper. Removes only the; charset=binaryparameter — legitimate ones likecharset=UTF-8ontext/htmlare preserved.src/wireclient.ts,src/dropbox.ts,src/googledrive.tsshouldBeTreatedAsBinarykeeps firing correctly.test/unit/wireclient.test.mjscontentType; add a regression test that legitimate charsets are not stripped.test/unit/node-wireclient-suite.jsGET-side mock-server responses in tests still send the suffix (tolerance input) — the assertions verify the library sanitizes it before returning to the caller.
Backward compatibility
; charset=binaryby the storage backend, but the library now strips it before exposingcontentTypeto callers.getFile().contentType: previously got; charset=binaryfor legacy files; now get clean mime. If anyone wrote code that expects the suffix, that breaks — but the library already exposesbodyasArrayBuffervs string for the binary distinction, so the suffix as a signal would be redundant. Risk is low.shouldBeTreatedAsBinary: still public API, still recognizes the suffix when called with the raw header value. Only what's returned fromget()is sanitized.same/type; charset=binaryand read them back. These exercise the local cache, not the WireClient response path — unaffected.