Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Protobuf with Manual Packing #489

Open
patrick-ogrady opened this issue Feb 12, 2025 · 4 comments
Open

Replace Protobuf with Manual Packing #489

patrick-ogrady opened this issue Feb 12, 2025 · 4 comments
Assignees
Labels
research More work required
Milestone

Comments

@patrick-ogrady
Copy link
Contributor

When first implementing the p2p primitive, I decided to use Protobuf and represent PublicKey/Signature with Bytes (which prost could automatically use). However, the migration to Array (#447) makes Protobuf a lot less enjoyable to use (it was already less performant but is also even more so now).

We now need to parse fields from proto and keep them around with the proto object:

We should consider dropping protobuf altogether in primitives for manually packed byte payloads (which can parse generic types directly).

Related: #454

@patrick-ogrady patrick-ogrady added the research More work required label Feb 12, 2025
@patrick-ogrady patrick-ogrady self-assigned this Feb 12, 2025
@github-project-automation github-project-automation bot moved this to Backlog in Tracker Feb 12, 2025
@patrick-ogrady
Copy link
Contributor Author

This change may pair very well with #434 (we could parse messages + verify signatures concurrently and then pass the formatted messages to the application).

@patrick-ogrady
Copy link
Contributor Author

A convenient byproduct of this would also be removing this constant:

const PROTOBUF_OVERHEAD: usize = 64;

@patrick-ogrady
Copy link
Contributor Author

May make sense to add a Version/Flag type that has 4 bits dedicated to version and 4 bits dedicated to flags (for including/not including different object components).

@patrick-ogrady
Copy link
Contributor Author

I'm not sure if there is any analysis on "worst case" parsing of adversarial Protobuf payloads of different sizes but that would be very interesting to dig into 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
research More work required
Projects
Status: Backlog
Development

No branches or pull requests

1 participant