Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize batch reconstruction in Trantor's availability module #503

Open
matejpavlovic opened this issue Jul 28, 2023 · 0 comments
Open
Labels
enhancement New feature or request Trantor

Comments

@matejpavlovic
Copy link
Contributor

The batch reconstruction currently requests a missing batch directly from all signers of the corresponding certificate. This is EXTREMELY inefficient especially at scale. E.g., with 32 nodes, at least 21 nodes sign the certificate. For a 1MB batch, a node ends up uselessly downloading 20MB of data.

Implement a smarter way of downloading the data, e.g., periodically picking a random signer of the certificate and requesting the data only from that one signer. Even better, the batch can be split into chunks and downloaded in parallel from all signers, or even erasure-coded.

https://github.com/filecoin-project/mir/blob/201930e724deee5036dfe7b4ea5051d709e8f8dd/pkg/availability/multisigcollector/internal/parts/batchreconstruction/batchreconstruction.go#L95-L99C26

@matejpavlovic matejpavlovic added enhancement New feature or request Trantor labels Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Trantor
Projects
None yet
Development

No branches or pull requests

1 participant