Performance difference with hyparquet ?

[AbdealiLoKo]

Curious if anyone has tried both parquet-wasm and hyparquet and found any performance difference

[kylebarron]

hyparquet is pure JS and loads to JS objects. So I'd expect:

• If you only need to load the metadata of Parquet files, or if you have very small Parquet files, hyparquet could be faster as hyparquet is smaller.
• If you need the absolute smallest bundle size, hyparquet may be better for your use case.
• Since parquet-wasm 1. uses a really high performance rust library, 2. is running in WebAssembly, and 3. converts to a high-performance binary memory format. If you have large files, and can use the resulting Arrow data directly without converting that to JS objects, I'd expect parquet-wasm would be significantly faster.

But I don't have hard numbers. If you do any benchmarking I'd love to hear!