Convert VARIANT to binary placeholder#15228
Conversation
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveSchemaUtil.java
Show resolved
Hide resolved
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveSchemaUtil.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
I think this is the right fix, although I believe we should document it. Another approach (if we are worried about this conversion) is to go to the actual caller of this code and just strip out variants before we pass in the schema. Since I don't think we actually use this hive schema anywhere for reads it's probably fine to just use a placeholder.
As @pvary noted we also have to take care of the other non-compatible types, (nanos) that will break this code.
I do want to make sure we get at least one Hive Integration expert to make sure we aren't breaking anything by doing this. (Since technically this will be a struct)
I am happy to open a new PR (or even do it as part of the same PR ) to address that once the community agrees overall that this is the correct approach. I'll also update the docs to reflect this change shortly |
|
CC: @deniskuzZ |
Fixes #15220