Changing cubeSize should not create a new Revision

[Jiaweihu08]

Why

This feature will allow the adjustment of cubeSize over time without creating new Revisions.

Theory

In theory, the domain-driven maxWeight estimation allows changing cube sizes: Wc = Wpc + dcs / Dc without impacting the integrity of the index, as cube domains are not dependent on the cube size. However, existing files must be rewritten for better adherence to the index via qbeastTable.optimize.

A closer examination of Wc = Wpc + dcs / Dc reveals that Wc is directly proportional to dcs - any change done to dcs will be reflected in the cube maxWeight in the same proportions:

Index:
        r
      /    \
     b      a
              \
               aa

Wr = dcs / Dr;
Wa = Wr + dcs / Da;
Waa = Wa + dcs / Daa;

Waa = Wa + dcs / Daa
= (Wr + dcs / Da) + dcs / Daa
= ((dcs / Dr) + dcs / Da) + dcs / Daa
= dcs / Dr + dcs / Da + dcs / Daa
= dcs (1 / Dr + 1 / Da + 1 / Daa)

In other words, multiplying the desiredCubSize(dcs) by any factor f will result in a new weight Waa_new = Waa * f; halving the desiredCubeSize will divide all weights by 2.

Impact

If the cubeSize is reduced, weight ranges will shift to the left, increasing cube offsets - elements e | e.w > cube.maxWeight.

On the other hand, if cubeSize is increased, weight ranges will move to the right; some elements that before belonged to c will have to be relocated to some ancestor cube.

All these adjustments will be handled automatically when optimizing via qbeastTable.optimize().

[fpj]

We discussed in the product backlog curation call and determined that it needs further investigation to understand what the implementation entails.