fix(query): avoid rewrites for Parquet-compatible decimal widening - #20418
Draft
youngsofun wants to merge 1 commit into
Draft
fix(query): avoid rewrites for Parquet-compatible decimal widening#20418youngsofun wants to merge 1 commit into
youngsofun wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cdc2c24bde
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
youngsofun
force-pushed
the
codex/metadata-only-decimal-widen
branch
2 times, most recently
from
September 1, 2026 03:49
4a72d5b to
024cb69
Compare
sundy-li
approved these changes
Sep 1, 2026
youngsofun
marked this pull request as draft
September 1, 2026 07:44
youngsofun
force-pushed
the
codex/metadata-only-decimal-widen
branch
2 times, most recently
from
September 2, 2026 14:33
2258cad to
25be845
Compare
Contributor
🤖 CI Job Analysis
⛔️ CANCELLEDHigher priority request detected - retry cancelled to avoid conflicts. |
youngsofun
force-pushed
the
codex/metadata-only-decimal-widen
branch
2 times, most recently
from
September 2, 2026 18:12
f5774f5 to
6407a05
Compare
12 tasks
youngsofun
force-pushed
the
codex/metadata-only-decimal-widen
branch
from
September 7, 2026 00:41
6407a05 to
801b58f
Compare
# Conflicts: # src/query/service/src/physical_plans/physical_top_n.rs # src/query/storages/fuse/src/operations/replace_into/mutator/replace_into_operation_agg.rs
youngsofun
force-pushed
the
codex/metadata-only-decimal-widen
branch
3 times, most recently
from
September 7, 2026 01:58
94bd908 to
dcce5b2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
ALTER TABLE ... MODIFY COLUMNcurrently rewrites the full table whenever a Decimal type changes, even when only the precision is widened and the persisted Parquet representation is unchanged. On large tables this turns a lossless schema change into a full scan and rewrite.This PR makes eligible Decimal precision widening metadata-only when:
FIXED_LEN_BYTE_ARRAYuses the same byte width.Changes that cross a Decimal kind or Parquet-width boundary, change scale, or narrow precision continue to rewrite data.
Schema Decimal variants are not stable across meta protobuf round trips, so the decision derives Decimal width from precision rather than enum discriminants. Persisted statistics also keep their original
DecimalSize; schema-awareColumnStatisticsandClusterStatisticsviews temporarily retag compatible bounds and fail open for incompatible statistics.Reducers, range pruning, TopN, REPLACE, partition Top-K, runtime scan ordering/filtering, and reclustering now consume the safe statistics abstraction. Compatible Decimal TopN values merge at the widest precision, while incompatible statistics are discarded without blocking writes. Missing statistics remain unknown instead of being represented by fabricated bounds.
The persisted statistics layout is unchanged. Existing segment and snapshot metadata remain readable, and metadata-only DDL preserves existing block locations.
Tests
Validated with:
cargo test -p databend-storages-common-table-meta --libcargo test -p databend-common-storages-fuse --lib cluster_statistics::testscargo test -p databend-query --test it parquet_rs::prune_pages -- --nocapturecargo test -p databend-query --test it parquet_rs::prune_row_groups -- --nocapturecargo clippy -p databend-storages-common-table-meta -- -D warningscargo fmt --all -- --checkThe logic test covers metadata-only ALTER, predicate reads, post-ALTER writes, TopN, REPLACE, ANALYZE, COMPACT, reclustering, time travel, Decimal physical-width boundaries, and preservation of existing block locations.
Type of change
AI assistance
This change is