Parquet Encryption

[norberttech]

https://parquet.apache.org/docs/file-format/data-pages/encryption/

Parquet files containing sensitive information can be protected by the modular encryption mechanism that encrypts and authenticates the file data and metadata - while allowing for a regular Parquet functionality (columnar projection, predicate pushdown, encoding and compression).

The results of compression of column pages are encrypted before being written to the output stream. A new Thrift structure, with column crypto metadata, is added to column chunks of the encrypted columns. This metadata provides information about the column encryption keys.

That would make the parquet format even more attractive as it could significantly improve the security of data at rest