You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The VmLocalProver can have a configuration flag for memory mode such that in high memory mode where there is no risk of OOM, it should parallelize maximally every chance it gets using rayon for work stealing. This means:
generate_air_proof_input should be parallelized across segments. There is a stupid Send+Sync stopping it because we have some RefCell inside ExecutionSegment, but we can make a wrapper type around ExecutionSegment and unsafe impl Send + Sync just for this purpose because we know it's safe.
proving can be parallelized across segments as well, but Engine is not threadsafe because plonky3 also uses RefCell in some DFT. If we give a Fn() -> Engine to reconstruct the engine from FriParameters in each thread, then we can parallelize.
These risk OOM since it requires much more memory, but it could improve single machine proving time.
The text was updated successfully, but these errors were encountered:
The
VmLocalProver
can have a configuration flag for memory mode such that in high memory mode where there is no risk of OOM, it should parallelize maximally every chance it gets using rayon for work stealing. This means:generate_air_proof_input
should be parallelized across segments. There is a stupid Send+Sync stopping it because we have someRefCell
insideExecutionSegment
, but we can make a wrapper type around ExecutionSegment andunsafe impl Send + Sync
just for this purpose because we know it's safe.Engine
is not threadsafe because plonky3 also usesRefCell
in some DFT. If we give aFn() -> Engine
to reconstruct the engine fromFriParameters
in each thread, then we can parallelize.These risk OOM since it requires much more memory, but it could improve single machine proving time.
The text was updated successfully, but these errors were encountered: