Feature Request: Dedicated compression command

[Archmonger]

Currently users must run a defragment in order to retroactively force compression on existing files. There are many scenarios where the user may wish to force compression on existing files without the overhead of an defragment. For example, defrag isn't needed on a SSD mounted with compress, but the user may wish to force compression on uncompressed files during downtime hours.

[Zygo]

"defrag" is "atomically copy all of the data in this file to a new, hopefully contiguous, location, with some filters to try to avoid unnecessary work."

"compress" is "atomically copy and compress all of the data in this file to a new location, without caring too much about contiguity, with some filters to try to avoid unnecessary work."

There's not much difference between these--they're the same basic copy operation, but one has compression forced during the copy while the other has compression only if the mount or inode flags request it, and the unnecessary-work filters should be set differently for each case. On paper, it makes sense for one tool to handle both use cases, since they're the same fundamental operation with different parameters. So that's the way it has been implemented so far in btrfs.

In practice, defrag can handle neither case well. If the extent can't be compressed, then the defrag copy should be abandoned, but the current code keeps the bad copy and removes the original. e.g. if a large extent can't be compressed, the extent should not be modified at all, but the current code will make a copy with attempted compression, fail to compress, and end up carving the file into 512K uncompressed extents, even if it was originally 128M contiguous uncompressed extents. The existing filters are more or less orthogonal to the task of identifying what should and should not be rewritten for offline compression, so repeated defrags will keep moving data around and burning CPU on some portion of the data that should be excluded.

Ideally, a tool designed for offline compression would first test the data to see if it is compressible, then only issue defrag ioctls to rewrite the portions of the file that can be compressed, and only if they aren't already compressed in full-sized compressed extents. Bonus points for combining it with something like btrfs sub find-new so it only looks at new data in repeated runs (i.e. once it is determined that a big uncompressed extent cannot be compressed, the tool never looks at that extent again).