Skip to content

Conversation

@NivinCS
Copy link
Contributor

@NivinCS NivinCS commented Jun 24, 2025

Description

Add support for ALTER COLUMN SET DATA TYPE in the Iceberg connector

Motivation and Context

To address the issue : #25417

Impact

Test Plan

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add support for ALTER COLUMN SET DATA TYPE in the Iceberg connector


@github-actions
Copy link

github-actions bot commented Jun 24, 2025

Codenotify: Notifying subscribers in CODENOTIFY files for diff 709047e...7b915f5.

Notify File(s)
@aditi-pandit presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@elharo presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@kaikalur presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@rschlussel presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jun 24, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: dmariamgeorge / name: Deepa Mariam George (7b915f5)

@NivinCS NivinCS force-pushed the icebrg_alter_table_set_datatype branch from 85658a0 to fb22b19 Compare June 24, 2025 11:30
steveburnett
steveburnett previously approved these changes Jun 24, 2025
Copy link
Contributor

@steveburnett steveburnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! (docs)

Pull branch, local doc build, looks good. Thanks!

@prestodb-ci
Copy link
Contributor

@imjalpreet imported this issue as lakehouse/presto #25418

Copy link
Member

@hantangwangd hantangwangd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work, the whole change overall looks good! Just a few things to address:
1. As I see, the ShortDecimalType -> LongDecimalType conversion (e.g. decimal(10, 2) -> decimal(24, 2)) might cause problems in PARQUET format Iceberg table.
2. Since this change primarily focuses on supporting ALTER COLUMN TYPE in Iceberg, we should add test cases in Iceberg covering all type conversions (int->long, float->double, decimal->decimal, v2-compatible referring here), verifying that both query results and SHOW CREATE TABLE outputs meet expectations. An example test case may look as follows:

    @Test
    public void testAlterColumnType()
    {
        testWithAllFileFormats(this::testAlterColumnType);
    }

    private void testAlterColumnType(Session session, FileFormat fileFormat)
    {
        String schemaName = session.getSchema().get();
        String tableName = "test_alter_column_type_" + randomTableSuffix();
        assertUpdate("CREATE TABLE " + tableName + "(a int, b real, c decimal(4, 2)) with (\"write.format.default\" = '" + fileFormat + "')");
        assertUpdate("INSERT INTO " + tableName + " VALUES(1, 1.1, 1.21)", 1);
        assertUpdate("INSERT INTO " + tableName + " VALUES(2, 2.2, 2.34)", 1);
        assertQuery("SELECT * from " + tableName, "VALUES(1, 1.1, 1.21), (2, 2.2, 2.34)");
        validateShowCreateTable(tableName,
                ImmutableList.of(
                        columnDefinition("a", "integer"),
                        columnDefinition("b", "real"),
                        columnDefinition("c", "decimal(4,2)")),
                getCustomizedTableProperties(ImmutableMap.of(
                        "write.format.default", "'" + fileFormat + "'",
                        "location", "'" + getLocation(schemaName, tableName) + "'")));

        assertUpdate("ALTER TABLE " + tableName + " ALTER COLUMN a SET DATA TYPE bigint");
        assertUpdate("ALTER TABLE " + tableName + " ALTER COLUMN b SET DATA TYPE double");
        assertUpdate("ALTER TABLE " + tableName + " ALTER COLUMN c SET DATA TYPE decimal(10, 2)");
        assertQuery("SELECT * from " + tableName, "VALUES(1, 1.1, 1.21), (2, 2.2, 2.34)");
        validateShowCreateTable(tableName,
                ImmutableList.of(
                        columnDefinition("a", "bigint"),
                        columnDefinition("b", "double"),
                        columnDefinition("c", "decimal(10,2)")),
                getCustomizedTableProperties(ImmutableMap.of(
                        "write.format.default", "'" + fileFormat + "'",
                        "location", "'" + getLocation(schemaName, tableName) + "'")));

        assertUpdate("ALTER TABLE " + tableName + " ALTER COLUMN c SET DATA TYPE decimal(24, 2)");
        assertQuery("SELECT * from " + tableName, "VALUES(1, 1.1, 1.21), (2, 2.2, 2.34)");
        validateShowCreateTable(tableName,
                ImmutableList.of(
                        columnDefinition("a", "bigint"),
                        columnDefinition("b", "double"),
                        columnDefinition("c", "decimal(24,2)")),
                getCustomizedTableProperties(ImmutableMap.of(
                        "write.format.default", "'" + fileFormat + "'",
                        "location", "'" + getLocation(schemaName, tableName) + "'")));

        dropTable(getSession(), tableName);
    }

Please feel free to adopt it or modify it to your needs.

Copy link
Contributor

@pratyakshsharma pratyakshsharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took first pass. One high level comment, we can add documentation as well around this feature on https://prestodb.io/docs/current/sql/alter-table.html and update Iceberg connector with the limitation clearly stating what data type changes are we supporting currently.

HiveColumnConverterProvider.DEFAULT_COLUMN_CONVERTER_PROVIDER,
context.getWarningCollector(),
context.getRuntimeStats());
if (isTableOwner(transaction, identity, metastoreContext, tableName)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the check should be reversed here I think.

.commit();
}
catch (RuntimeException e) {
throw new PrestoException(ICEBERG_UNKNOWN_TABLE_TYPE, "Failed to set column type: " + firstNonNull(e.getMessage(), e), e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we expecting one of them to be null here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ICEBERG_UNKNOWN_TABLE_TYPE ?
Should be unknown column name or incompatible column type.

return immediateFuture(null);
}

private static List<RowType.Field> getCandidates(Type type, String fieldName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method is not getting used and can be reverted.


protected ConnectorContext context;

protected QueryStateMachine queryStateMachine;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is not getting used.

return Optional.empty();
}

public int getCreateTableCallCount()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not getting used.

return connectorCapabilities;
}

public void setConnectorCapabilities(ConnectorCapabilities... connectorCapabilities)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this as well

protected WarningCollector warningCollector;

private Session testSession;
private TableHandle tableHandle;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be converted to a local variable.

{
private static final String CATALOG_NAME = "catalog";
public static final String SCHEMA = "schema";
private CatalogManager catalogManager;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check intellij suggestions here. There are few variables which can be converted to a local variable. Please do the needful

Copy link
Contributor

@PingLiuPing PingLiuPing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A general question, does Presto support alter column type for other connector without this PR?

Copy link
Contributor

@PingLiuPing PingLiuPing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add test case to cover the iceberg primitive type promotions.
See https://iceberg.apache.org/spec/#schema-evolution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants