Store only 1/2 of sidecars on supernodes

[zilm13]

We need to decrease storage requirements for validator nodes with high count of validators and supernodes. This could be achieved by deduplicating sidecars data which is 2x extension over blobs. I suggest following changes which could be split into 2-3 PRs:

1. Add flag --data-column-sidecar-extension-retention-epochs, which I suggest to set by default to 512 epochs (a bit more than 2 days)
2. Improve DataColumnSidecar RPC handlers to have special logic when queried on supernode:
   a. Use dedicated thread pool with 2 threads for this kind of tasks
   b. If node is supernode, request asks for sidecars over retention period but in DA window (Defaults: 512 < epoch < 4096) and index is more than 63, we get 0-63 sidecars from DB and reconstruct everything else using thread pool. Data is not cached and used only for one request, but we should run only one extension for several indices in one request.
3. Add DataColumnSidecarExtensionPruner, which prunes 64-127 indices over data-column-sidecar-extension-retention-epochs if the node is supernode

NOTE: 72 blobs DataColumnSidecar's reconstruction for 1 slot is 1.2 ops/second on M4

[tbenr]

I like the idea. The approach makes sense to me.
What do you think about calculating the retention boundary as

oldestExtentionRetentionEpoch = Math.min(latestFinalizedEpoch, currentEpoch - sidecarExtensionEpoch)

So that we always keep all extensions in case of non finality. This way we better serve the RPCs in case network instability where probably several nodes are doing a checkpoint sync.

[zilm13]

I wanted to deduct it from last finalized, do we deduct 4096 from head epoch?? I thought we count only finalized, not?
I think your formula makes sense

[tbenr]

I wanted to deduct it from last finalized, do we deduct 4096 from head epoch?? I thought we count only finalized, not?

IIRC finalization is not part of the DA boundary calculation.

[zilm13]

Alternatively, if we want to get cheaper computation we could have separate table with proofs from pruned DataColumnSidecars and it will require only extension computation which is very cheap. But it's way more complex system for such feature and I'm not sure it makes sense to put that much in it at this moment.