Skip to content

[Iceberg]Error reading decimal columns when declared type is long decimal and the parquet file is short decimal #25149

@denodo-research-labs

Description

@denodo-research-labs

Error reading short decimal (precision <= 18) column when declared type is long decimal (precision > 18) and the parquet file is short decimal in an Iceberg table.

The parquet file of the Iceberg table declares the following type:
typedecimal: OPTIONAL INT64 L:DECIMAL(10,0) R:0 D:1

The metadata file of the Iceberg table generated by Snowflake declares the following type:

      "name" : "typedecimal",
      "type" : "decimal(19, 0)" 

Your Environment

  • Presto version used: 0.292
  • Data source and connector used: Iceberg
  • Iceberg table generated by Snowflake

Current Behavior

With vectorization enabled, hive.parquet-batch-read-optimization-enabled=true, throws this error:

java.lang.UnsupportedOperationException: com.facebook.presto.common.block.LongArrayBlock
        at com.facebook.presto.common.block.Block.getLong(Block.java:97)
        at com.facebook.presto.common.type.LongDecimalType.getSlice(LongDecimalType.java:142)
        at com.facebook.presto.common.type.LongDecimalType.getObjectValue(LongDecimalType.java:80)
        at com.facebook.presto.server.protocol.RowIterable$RowIterator.computeNext(RowIterable.java:77)
        at com.facebook.presto.server.protocol.RowIterable$RowIterator.computeNext(RowIterable.java:50)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
        at com.google.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1391)
        at com.google.common.collect.Iterators$1.hasNext(Iterators.java:140)
        at com.facebook.presto.server.protocol.QueryResourceUtil.prepareJsonData(QueryResourceUtil.java:333)

Without vectorization, hive.parquet-batch-read-optimization-enabled=false, throws this error:

Caused by: java.lang.UnsupportedOperationException: com.facebook.presto.common.type.LongDecimalType
    at com.facebook.presto.common.type.AbstractType.writeLong(AbstractType.java:110)
    at com.facebook.presto.parquet.reader.ShortDecimalColumnReader.readValue(ShortDecimalColumnReader.java:47)
    at com.facebook.presto.parquet.reader.AbstractColumnReader.lambda$readValues$0(AbstractColumnReader.java:169)
    at com.facebook.presto.parquet.reader.AbstractColumnReader.processValues(AbstractColumnReader.java:223)
    at com.facebook.presto.parquet.reader.AbstractColumnReader.readValues(AbstractColumnReader.java:168)
    at com.facebook.presto.parquet.reader.AbstractColumnReader.readNext(AbstractColumnReader.java:148)
    at com.facebook.presto.parquet.reader.ParquetReader.readPrimitive(ParquetReader.java:389)
    at com.facebook.presto.parquet.reader.ParquetReader.readColumnChunk(ParquetReader.java:550)
    at com.facebook.presto.parquet.reader.ParquetReader.readBlock(ParquetReader.java:533)
    at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:230)
    at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:208)

Changing the precision in the Iceberg table metadata file works:

      "name" : "typedecimal",
      "type" : "decimal(10, 0)" 

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    🆕 Unprioritized

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions