-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Open
Labels
Description
Error reading short decimal (precision <= 18) column when declared type is long decimal (precision > 18) and the parquet file is short decimal in an Iceberg table.
The parquet file of the Iceberg table declares the following type:
typedecimal: OPTIONAL INT64 L:DECIMAL(10,0) R:0 D:1
The metadata file of the Iceberg table generated by Snowflake declares the following type:
"name" : "typedecimal",
"type" : "decimal(19, 0)"
Your Environment
- Presto version used: 0.292
- Data source and connector used: Iceberg
- Iceberg table generated by Snowflake
Current Behavior
With vectorization enabled, hive.parquet-batch-read-optimization-enabled=true, throws this error:
java.lang.UnsupportedOperationException: com.facebook.presto.common.block.LongArrayBlock
at com.facebook.presto.common.block.Block.getLong(Block.java:97)
at com.facebook.presto.common.type.LongDecimalType.getSlice(LongDecimalType.java:142)
at com.facebook.presto.common.type.LongDecimalType.getObjectValue(LongDecimalType.java:80)
at com.facebook.presto.server.protocol.RowIterable$RowIterator.computeNext(RowIterable.java:77)
at com.facebook.presto.server.protocol.RowIterable$RowIterator.computeNext(RowIterable.java:50)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
at com.google.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1391)
at com.google.common.collect.Iterators$1.hasNext(Iterators.java:140)
at com.facebook.presto.server.protocol.QueryResourceUtil.prepareJsonData(QueryResourceUtil.java:333)
Without vectorization, hive.parquet-batch-read-optimization-enabled=false, throws this error:
Caused by: java.lang.UnsupportedOperationException: com.facebook.presto.common.type.LongDecimalType
at com.facebook.presto.common.type.AbstractType.writeLong(AbstractType.java:110)
at com.facebook.presto.parquet.reader.ShortDecimalColumnReader.readValue(ShortDecimalColumnReader.java:47)
at com.facebook.presto.parquet.reader.AbstractColumnReader.lambda$readValues$0(AbstractColumnReader.java:169)
at com.facebook.presto.parquet.reader.AbstractColumnReader.processValues(AbstractColumnReader.java:223)
at com.facebook.presto.parquet.reader.AbstractColumnReader.readValues(AbstractColumnReader.java:168)
at com.facebook.presto.parquet.reader.AbstractColumnReader.readNext(AbstractColumnReader.java:148)
at com.facebook.presto.parquet.reader.ParquetReader.readPrimitive(ParquetReader.java:389)
at com.facebook.presto.parquet.reader.ParquetReader.readColumnChunk(ParquetReader.java:550)
at com.facebook.presto.parquet.reader.ParquetReader.readBlock(ParquetReader.java:533)
at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:230)
at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:208)
Changing the precision in the Iceberg table metadata file works:
"name" : "typedecimal",
"type" : "decimal(10, 0)"
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
🆕 Unprioritized