-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add auto_encoding
and improve send_compressed
behavior
#2058
Open
m62624
wants to merge
10
commits into
hyperium:master
Choose a base branch
from
m62624:feature/compression-support
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…vior The service was initially configured to send compressed data only in Gzip format. Different clients requested varying compression algorithms (e.g., Gzip and Zstd). Logs showed the following behavior: 1. Client 1 requested Gzip: ``` Server received request: data="gzip", headers=MetadataMap { headers: {"te": "trailers", "content-type": "application/grpc", "grpc-encoding": "gzip", "grpc-accept-encoding": "gzip,identity", "user-agent": "tonic/0.13.0"} } Server sending response: gzip Responding with gzip compression ``` 2. Client 2 requested Zstd: ``` Server received request: data="zstd", headers=MetadataMap { headers: {"te": "trailers", "content-type": "application/grpc", "grpc-encoding": "gzip", "grpc-accept-encoding": "zstd,identity", "user-agent": "tonic/0.13.0"} } ``` It is clear that clients requested different encoders using `grpc-accept-encoding`. **Expected behavior:** - The server should select a compression algorithm from the intersection of the supported algorithms from the server and the client. - If the client requests a compression algorithm not supported by the server, the server should either use `identity` (no compression) or return a `UNIMPLEMENTED` status. **Actual behavior:** If any `send_compressed` configuration was set for the service, the `from_accept_encoding_header` method was triggered regardless of the `grpc-accept-encoding` header provided by the client. The function responsible for compression selection on the server did not respect `send_compression_encodings`. The server defaulted to the first supported compression algorithm from `grpc-accept-encoding` without checking if the algorithm was allowed by `send_compression_encodings`. **Resolution:** - If `send_compressed` is not specified, the server defaults to `identity`. - If `send_compressed` is specified, the server only compresses responses for clients explicitly requesting a supported algorithm. **Feature:** The `auto_encoding` mechanism was added. This allows the server to automatically select a compression algorithm supported by the client without requiring explicit configuration of `send_compressed` on the server. The server dynamically adjusts compression behavior, ensuring compatibility with client preferences for efficient communication.
Did you use AI to generate the description? If so, could you please share it? |
Starting from the "New Test Suite:" section, I generated the content using GitHub Copilot directly on the PR submission page. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
name: 💡 Feature Request
about: feat: Add
auto_encoding
and improvesend_compressed
behaviorFeature Request
Crates
tonic
Motivation
When using
send_compressed
, the server sends compressed data in a format that may not be explicitly allowed for the service, even if the format isn't listed as supported by the server. This means that the client receives the desired compression result, even if the server has not explicitly enabled that compression algorithm for the service.The server should only send compressed data using algorithms that are supported by both the client and the server. If the client requests a compression format that the server does not support, the server should either send the data uncompressed (using
identity
) or return an error (Status
).Currently:
send_compressed
is set for the service, thefrom_accept_encoding_header
method executes regardless of the providedgrpc-accept-encoding
.send_compression_encodings
configuration.This means that when processing a response, the server uses the
grpc-accept-encoding
header sent by the client and chooses the first supported compression algorithm, ignoring whether that algorithm is explicitly allowed insend_compression_encodings
.This results in unexpected behavior, as the server should only use compression types explicitly allowed via
send_compressed
.Proposal
The logic has been refactored to address this issue:
send_compressed
is used and the server-client compression formats do not overlap, the server sends data without compression (identity
).auto_encoding
method has been added to automatically select the correct compression format based on the intersection of client and server-supported formats. That is, the server sends the desired format to each client.This change ensures alignment with expected gRPC behavior.
Alternatives
Drawback: It continues to cause unexpected issues when formats don't align.
Drawback: It might unnecessarily interrupt communication where uncompressed data could suffice.
New Test Suite:
Cargo.toml
: Added a new test directorytests/various_compression_formats
to the members list.tests/various_compression_formats/Cargo.toml
: Created a new Cargo.toml file for thevarious_compression_formats
test suite.tests/various_compression_formats/build.rs
: Added a build script to compile protobuf definitions.tests/various_compression_formats/proto/proto_box.proto
: Defined protobuf messages and services for the test suite.tests/various_compression_formats/src/lib.rs
: Included the compiled protobuf module.tests/various_compression_formats/tests/auto_encoding.rs
: Added comprehensive tests to verify the behavior of different compression formats and automatic encoding detection.Automatic Encoding Feature:
tonic-build/src/server.rs
: Introduced theauto_encoding
flag and logic to automatically determine the encoding based on request headers. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]tonic/src/codec/compression.rs
: Modified thefrom_accept_encoding_header
function to support theauto_encoding
flag.tonic/src/server/grpc.rs
: Added theauto_encoding
flag to theGrpc
struct and implemented the logic to handle automatic encoding. [1] [2] [3] [4]