You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@Smith-Cruise maybe i dont understand correctly, but won't using the combined row ranges from the compound predicate tree zone map evaluation that unions and intersects the ranges across columns (based on AND/OR), for selecting the offset index for each individual column (narrows down which pages need to be read per column) result in more IO than is necessary, since this calculation should only be narrowed down by conjunctions from other column predicates and never widened because of a disjunction from a different column. e.g. for compound predicate column_A = "rare" OR column_B = "common" the range / pages to read for A should not be increased because of the inclusion of a non selective predicate on column B.
seems like the parquet advanced zonemap filter wont fully narrow down using the offset index properly if the compound predicate also contains an OR with a subpredicate on a column that matches most of the rows
The text was updated successfully, but these errors were encountered:
Originally posted by @shaeqahmed in ab8abca
Enhancement
seems like the parquet advanced zonemap filter wont fully narrow down using the offset index properly if the compound predicate also contains an OR with a subpredicate on a column that matches most of the rows
The text was updated successfully, but these errors were encountered: