Reasonable Memory limits per container for rustic?

[mueckinger]

I started playing around with rustic in Kubernetes. For safe operation, it is necessary to set memory limits per container. While restic is doing well with a memory limit of 128Mi for restic init and restic backup, rustic gets OOM killed while running rustic init or rustic backup with a memory limit of 128Mi. This is not what I expected as rustic claims "Huge decrease in memory requirement".

Tested with rustic v0.6.1 (musl) on alpine 3.19.0

[aawsome]

Thanks @mueckinger for opening this issue! Can you please specify more what you are doing? What are the exact commands you are using and which data are you trying to back up? Do you also use rclone within the container? Can you also please give the rustic output you get before it aborts?

Besides this, 128 MiB is quite few and I fear you can't work with that memory with either restic or rustic when using a "normal" data size to backup like some 100 GiBs... And for larger repositories, yes rustic is expected to use less memory than restic. There can be cases like backing up to an empty repository where rustic uses slightly more memory, mostly because it uses a higher pack size than restic does and because it keeps packfiles in memory before sending them to the backend instead of writing temporary files.

[mueckinger]

Rclone is running in sidecar containers, so the 128Mi are for rustic only. Unfortunately the containers are killed before I can measure anything, but increasing limits to 256Mi works so far, at least for rustic init and rustic backup with backing up some small sample files.

What would be your recommendation regarding a memory limit for rustic?

[aawsome]

Actually with 256 or 512 MiB of memory you should get quite far. I have a repository with around 200GiB of sizes and didn't encounter more than 200 MiB of memory usage so far...

[aawsome]

Note that for restore rustic currently uses more memory, see #1067. There are already PRs open to fix that but for large repositories, it could be that you'll still arrive at 1.5 GiB memory requirement.