-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
restic backup should refuse to run without cache #4775
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Output of
restic version
restic 0.16.4 compiled with go1.21.6 on linux/amd64
What backend/service did you use to store the repository?
Infomaniak "Swiss Backup" S3 compatible
Problem description / Steps to reproduce
Short story: Now that I have found out, it is easy to describe the problem: When
restic backup
fails to identify the cache directory, a message stating this is printed tostderr
and the backup continues. The backup works, but is incredibly slow. When restic is run from a script that only capturesstdout
and ignoresstderr
, it is impossible to see the reason for the slowness.I would prefer if
restic backup
would abort without cache (and maybe a new option--ignore-no-cache
would be implemented to get back the old behavior). IMHO, runningrestic backup
without cache is something that should be avoided. Read on to understand where I am coming from.Long story
I run a dedicated backup server. This backup server uses dirvish (basically an
rsync
wrapper) to backup remote hosts. Dirvish uses the--link-dest
feature ofrsync
for file based deduplication (unchanged files are hard linked between backup generation directories instead of being transferred and stored a 2nd time) . The server is "woken on LAN" daily, the backup job which runs all backups (including the offsite ones) is run fromsystemd
as last task of the boot up sequence. The last statement of the backup job shuts down the machine, ready for the next wake on lan.I introduced
restic
as "secondary offsite backup", i.e. I wantedrestic
to copy the latestdirvish
backup generation to the cloud, so I simply added a few lines to my existing backup script: Do abind mount
so thatrestic
sees the same base directory name for the recentdirvish
backup (which has a timestamp as directory name), runrestic backup
and umount the bind mount again.After having set up (and manually tested) all this to my best knowledge according to the documentation, I realized that when ran automatically from
systemd
it was unbearably slow, but kind of worked too.Looking Back: Unfortunately, my existing backup script captured only
stdout
and notstderr
, sosystemd
, the error messages about the missing cache ended up injournalctl
(where I did not look at until doing this bug report) and not in my own logs (where I stared at for hours to see a meaningful difference).stderr
on the console, nothing was printed there to see.Having had no clue why
restic
was so fast in manual mode and so slow insystemd
mode made me so desperate that I started to raise the wildest (and wrong) ideas about the root cause, (e.g. ticket #4755). Introducing the explicitly setRESTIC_CACHE_DIR
in my backup script solved all problems and restic is now incredibly fast 🎉Some additional Info
Hint: The options in the examples above (
--ignore-ctime --ignore-inode --no-extra-verify
) were also a result of me having no clue what causes that slowness. As thebind
mount does not change the inode numbers, I don't need--ignore-inode
and I use--ignore-ctime
only now. This is another lesson learned: As thersync
feature--link-dest
updates the hard link count (among others), thectime
of all files and directories in my backup will always have the latest backup date asctime
, becausectime
cannot be "faked" bytouch
(or even a syscall).ctime
gets updated when a file is createdctime
gets updated whenever a file is linked to its cousin from the previous day.Expected behavior
restic backup
should refuse to run when cache directory not found (i.e. working without cache), maybe an option to override this (i.e. "--do-even-without-cache") needs to be implemented too.stderr
andstdout
.Actual behavior
restic backup
logs tostderr
only about the missing cache and then continues happily (but becomes very slow).Do you have any idea what may have caused this?
See above, the error message is clear.
restic
needsHOME
orXDG_CACHE_HOME
to be set to have a cache.Did restic help you today? Did it make you happy in any way?
I am still thrilled by the implementation of
restic forget
: the easiest way to manage staggered backup history I've ever seen :-)The text was updated successfully, but these errors were encountered: