backup: support reading changed files/dirs from a file #4469
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For `restic backup`, support new flags `--changed-files-from-verbatim` and `--changed-files-from-raw` to read the files/dirs that actually have changed from a file (or multiple files). Directories that don't (directly or indirectly) contain any changed files/dirs will reuse the corresponding subtree of the parent snapshot. This option is useful for higher-level backup tools which use restic as a backend but have their own mechanism of figuring out which files have changed (e.g., using zfs or btrfs diff tools). We require to explicitly pass `--parent` as a protection mechanism in order to make sure the higher-level backup tool and restic agree on the parent snapshot. Though the caller can circumvent this protection mechanism by passing `--parent latest`. Caveat: since device IDs are unstable (across reboots or across different zfs/btrfs snapshots of the same subvolume), the parent snapshot and current snapshot might have mismatching device IDs. In this case, the feature will still reuse subtrees of the parent snapshot (under the conditions mentioned above), so we end up with a snapshot that contains subtrees with different `device_id` values, even if there was only a single mountpoint in play. For now, we could simply document this caveat and discourage users who rely on correct restoration of hardlinks from using this feature. When restic#3041 is properly fixed in the future, then this caveat probably goes away, too. The idea for this feature emerged here: restic#1502 (comment)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For
restic backup, support new flags--changed-files-from-verbatimand--changed-files-from-rawto read the files/dirs that actually have changed from a file (or multiple files). Directories that don't (directly or indirectly) contain any changed files/dirs will reuse the corresponding subtree of the parent snapshot.This option is useful for higher-level backup tools which use restic as a backend but have their own mechanism of figuring out which files have changed (e.g., using zfs or btrfs diff tools). We require to explicitly pass
--parentas a protection mechanism in order to make sure the higher-level backup tool and restic agree on the parent snapshot. Though the caller can circumvent this protection mechanism by passing--parent latest.Caveat: since device IDs are unstable (across reboots or across different zfs/btrfs snapshots of the same subvolume), the parent snapshot and current snapshot might have mismatching device IDs. In this case, the feature will still reuse subtrees of the parent snapshot (under the conditions mentioned above), so we end up with a snapshot that contains subtrees with different
device_idvalues, even if there was only a single mountpoint in play.For now, we could simply document this caveat and discourage users who rely on correct restoration of hardlinks from using this feature. When #3041 is properly fixed in the future, then this caveat probably goes away, too.
Was the change previously discussed in an issue or on the forum?
The idea for this feature emerged here: #1502 (comment)
Checklist
changelog/unreleased/that describes the changes for our users (see template).gofmton the code in all commits.