Avoid extra copy in protobuf rust codec

[arjan-bal]

Feature Request

protobuf-rust Message trait don't support encoding into a user provided buffer. The reason is that the generated messages may be using either of the following C++ protobuf implementations (called kernels):

• upb (micro protobuf)
• C++ protobuf

This means that the trait must provide a common subset of features supported by both kernels. upb performs a single pass (opposed to two passes) over the message by encoding the message in reverse. It starts serializing from the end of the message and works its way backward in memory. This approach means that by the time it needs to write the length of a sub-message, that sub-message has already been fully encoded, and its length is known. The encoding buffer is filled from left to right and there may be an empty space in the front of the buffer.

Even if upb adds support for encoding into a user provided buffer, it may be slower than copying the bytes.

Crates

tonic-protobuf

Motivation

Proposal

Alternatives

I can think of the following ways to avoid a copy here:

1. Request the protobuf crate to expose an API that allows deserialization into a destination buffer.
2. Change the codec API to return a Vec<u8> instead of accepting the destination as a parameter. Tonic can then create an EncodedBuf from the returned Vec<u8>.
3. Make the constructor of EncodedBuf public. This would allow a codec to create its own EncodedBuf and std::mem::swap it with the one passed to the encode method, avoiding a copy.