Skip updating partition stats when adding empty stats partitions to Glue #25970
Conversation
chenjian2664
force-pushed
the
sep17064-sync-calls
branch
from
716ef21 to
87432b7
Compare
chenjian2664
changed the title
chenjian2664
changed the title
| @@ -1106,6 +1106,7 @@ public void addPartitions(String databaseName, String tableName, List<PartitionW | |||
|
|
|||
| // statistics are created after partitions because it is not clear if ordering matters in Glue | |||
| var createStatisticsTasks = partitionsWithStatistics.stream() | |||
| .filter(partitionWithStatistics -> partitionWithStatistics.getStatistics() != PartitionStatistics.empty()) | |||
There was a problem hiding this comment.
Does this impact cases where we actually want statistics to be reset to empty ?
E.g. Insert overwrite on a partition with hive statistics collection flag disabled
There was a problem hiding this comment.
The PartitionStatistics contains HiveBasicStatistics, this won't equal to the empty().
|
/test-with-secrets sha=87432b76aeba578904a9f573f8af77b960ec18ae |
|
The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/15571937661 |
chenjian2664
force-pushed
the
sep17064-sync-calls
branch
from
87432b7 to
9e0a289
Compare
| @@ -87,6 +95,40 @@ public void cleanUpSchema() | |||
| getQueryRunner().execute("DROP SCHEMA " + testSchema + " CASCADE"); | |||
| } | |||
|
|
|||
| @Test | |||
| void testInsertOverwriteStatisticsDisabled() | |||
There was a problem hiding this comment.
@raunaqmorarka I add the test to show case that the last commit won't influence the metastore operations
|
The test failures in https://github.com/trinodb/trino/actions/runs/15571937661 looks like not relates to the change |
chenjian2664
force-pushed
the
sep17064-sync-calls
branch
2 times, most recently
from
4ba1e08 to
2bd5ceb
Compare
testing/trino-testing/src/main/java/io/trino/testing/MultisetAssertions.java
Outdated
Show resolved
Hide resolved
Add Glue metastore access operations test for insert overwrite with stats disabled
Add Glue metastore access operations test for `sync_partition_metadata` procedure in `TestHiveGlueMetastoreAccessOperations`
chenjian2664
force-pushed
the
sep17064-sync-calls
branch
from
2bd5ceb to
bef0465
Compare
|
/test-with-secrets sha=bef046541465899f32b9300e7d041ec8e2c25c57 |
|
The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/15599241399 |
|
It's a known flaky test? https://github.com/trinodb/trino/actions/runs/15599241399/job/43936056919#step:18:44 |
...src/test/java/io/trino/plugin/hive/metastore/glue/TestHiveGlueMetastoreAccessOperations.java
Show resolved
Hide resolved
Description
Additional context and related issues
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: