Irrelevant extents are not deleted

[DenisVorobyov74]

Hi! There is an Ubuntu and a partition with a BTRFS file system, which contains a *.mdf file. From this "template" file, reflink files (cp -a --reflink=always) are created for new databases (used for development).
Initially, the template file weighed approximately 252GB. Periodically, transaction log files from the product database are loaded into the template database using MS SQL. Naturally, the size increases. After three weeks, I deleted all the reflink files (leaving only one template *.mdf file) and used the command "btrfs filesystem du –s" to check the current size. The size remained the same (252GB), but when I used the command "sudo btrfs filesystem usage –T", the "Used" section showed 283GB.
How can I reduce the occupied space to 252GB?
P.S. The *.ldf file is located in another EXT4 partition...

[Zygo]

The phenomenon you're observing is due to bookend extents. TL;DR, when you write into the middle of any existing extent, btrfs doesn't free the space occupied by the original extent. Instead, it allocates more space to store the newer, overwriting data. The original space is not freed until every block of the original extent is deleted or overwritten.

This has nothing to do with reflinks. It occurs with any file that had been copied sequentially from somewhere else or a file that has been defragmented (which makes big extents), then partially overwritten with random access writes (small writes make bookends in the large extents).

You can free the extra space by rewriting partially overwritten extents, e.g. by:

1. cp -a --reflink=never foo.mdf clone.mdf && mv -f clone.mdf foo.mdf (always works because it rewrites every extent; does more IO than necessary; requires 252G of free space)
2. btrfs fi defrag -t 512M foo.mdf (rewrites most extents; less IO, some free space not reclaimed; requires a few GiB of free space depending on system RAM size; change the -t parameter for speed/size tradeoffs)
3. develop a garbage-collection tool for btrfs (rewrites only extents with bookends to cause btrfs to free the unreachable blocks; doesn't exist yet, you'd have to build it or wait for someone else to make one; requires minimal temporary space and IO)

You'll notice that the first two solutions recreate the situation that leads to the bookend extents. They are only a temporary workaround, and you'll have to repeat them every time bookend extents build up. Solution 3 could avoid this by maintaining the file in a fragmented state, but the tradeoff is much larger metadata size and much slower sequential scans of the database file.

btrfs filesystem du reports only the logical size of the file, not its physical size, so you cannot observe the issue with btrfs filesystem du. Use btdu or compsize to see physical file sizes. btdu will give you a separate "unreachable" category so you can see which files are most affected.

[DenisVorobyov74]

Thank you for your response! Is there any estimate when the garbage-collection tool might appear? Is there any development roadmap?

[DenisVorobyov74]

Is there an approximate time for when the bug will be fixed?

[kdave]

There isn't, sorry. The option 3 sounds best if implemented by an ioctl, similar to defrag, to free the ureachable blocks.