writerFooter is not supported for ORC file format

[rui-mo]

Description

ORC reader needs to parse footer as proto::orc::Footer.

velox/velox/dwio/dwrf/reader/ReaderBase.cpp

Lines 200 to 204 in 4adec18

	if (fileFormat() == FileFormat::DWRF) {
	footer_ = parseFooter<proto::Footer>(decompressed.get(), arena_.get());
	} else {
	footer_ = parseFooter<proto::orc::Footer>(decompressed.get(), arena_.get());
	}

Velox now only supports writing footer as the DWRF proto::Footer.

velox/velox/dwio/dwrf/writer/WriterBase.cpp

Line 23 in 4adec18

void WriterBase::writeFooter(const Type& type) {

When testing the ORC decimal read and write, we find that the ORC reader parses the precision and scale information from footer while the DWRF footer does not contain them. Long decimal type needs to be written as hugeint kind in the ORC footer while the DWRF format does not contain hugeint type.

[majetideepak]

@rui-mo We need to create/extend the FooterWrapper for the write path similar to the read path.
Orc Format must be written into proto::orc::Footer

[pedroerp]

Cc: @Yuhta

[rui-mo]

@majetideepak I will take further look on how to generate the correct one. Thanks for your suggestion!

[wypb]

@majetideepak @rui-mo It's not enough to just modify the write of orc file footer, there are other information such as statistics, Stripe Footer, index, etc. We implemented a complete ORC writer internally last year based on Velox's DWRF writer , and I could open source this code if needed.

[rui-mo]

It's not enough to just modify the write of orc footer

@wypb I recognized that when implementing the orc footer writing...

We implemented a complete ORC writer internally last year based on Velox's DWRF writer , and I could open source this code if needed.

Glad to hear about your open source plan, and that would be great!

[wypb]

I am in the process of organizing the code. There are many files involved (more than 40). I may be able to submit a PR as early as next week.