Experiment: flatten plain state for MemoryStateProvider

[mediocregopher]

Describe the feature

When using MemoryStateProvider we pass it an array of ExecutedBlocks, which it then iterates over to serve requests:

reth/crates/chain-state/src/memory_overlay.rs

Lines 111 to 115 in b79c58d

	for block in self.in_memory.iter() {
	if let Some(account) = block.execution_output.account(address) {
	return Ok(account);
	}
	}

reth/crates/chain-state/src/memory_overlay.rs

Lines 215 to 219 in b79c58d

	for block in self.in_memory.iter() {
	if let Some(value) = block.execution_output.storage(&address, storage_key.into()) {
	return Ok(Some(value));
	}
	}

reth/crates/chain-state/src/memory_overlay.rs

Lines 227 to 231 in b79c58d

	for block in self.in_memory.iter() {
	if let Some(contract) = block.execution_output.bytecode(code_hash) {
	return Ok(Some(contract));
	}
	}

We should see if we can flatten the BundleState of the ExecutedBlocks into a single state, which can be applied as an overlay instead. This can be done asynchronously in a similar manner to the trie input:

reth/crates/engine/tree/src/tree/payload_validator.rs

Line 1050 in b79c58d

fn spawn_deferred_trie_task(

Note that we can only use this flattened state if its anchor block shares the anchor of the block being validated. If not then we must either compute a new flattened state or fallback to the current behavior.

Additional context

No response

[gakonst]

I like the idea of "if there's no reorg then go to quick flattened state" optimization vs "current behavior fallback" idea. Maybe the right way to decide is whether re-computing the flattened state on reorg is more expensive than occasionally falling back to the current behavior? Could see either, but def want the flattened optimization in teh happy case.

[mattsse]

this doesnt really have much to do with re-orgs, we still need block level changes, so the flattened view would be extra