Larger recordsize Causes Worse Compression

[inferenceus]

Based on extensive research across the internet, it seems that larger recordsize values allow for greater performance and efficiency due to less blocks to read/write, and less metadata overhead. The recordsize seems to be variable and data-blocks for files smaller than the recordsize are stored in the same-size recordsizes as the file or 1 larger, with tail-packing for the final recordsize.

It is also apparent that larger recordsize values allow for higher compression-ratios; however, this is not the case on my system. After hours of testing, I am able to reliably reproduce the following results (compression-levels above 1 do not improve compression-ratio in my dataset beyond around 0.03x):

• compression=zstd-1 + recordsize=256K = 1.23x
• compression=zstd-1 + recordsize=512K = 1.11x
• compression=zstd-1 + recordsize=16M = 1.01x

A new dataset was created to test different values, with it being completely erased of both files and snapshots before the recordsize value was changed. My dataset is comprised of image-files with an average size of 192 kiB. How is it that the results are as such?

[amotin]

For files smaller than recordsize zfs uses smaller blocks, so it does not depend on the tail compression. I wonder whether your data are compressible at all, or what you see are the tails.

[inferenceus]

@amotin If I set compression=none, the compression-ratio drops to 1.00x, so there seems to be some compressible data despite the images being pre-compressed. I am confused about why the compression-ratio is always worse with anything other than 256K recordsize; I expected there to at least be parity when using larger recordsize values.

[amotin]

Because files of only one block/record don't need tail compression. Of two or more -- do need, and it is this compression you see.

[inferenceus]

@amotin Okay. In that case, is there still an advantage to using 256K recordsize rather larger, such as 1M? Despite the lower compression-ratio, it wouldn't actually use more storage space, but I'd gain higher performance?

[amotin]

Smaller blocks may be beneficial for random access patterns.

[inferenceus]

@amotin Thanks. I've set my media to 1M recordsize due to it being sequential in nature, and LZ4 compression as it doesn't seem compressible but there may be some data and metadata which can be compressed; LZ4 is extremely cheap so I don't see a point of completely disabling compression.