shrink-backup is a very fast utility for backing up your SBC:s into minimal bootable img files for easy restore with autoexpansion at boot

I made this script because I wanted a universal method of backing up my SBC:s into small img files as fast as possible (with rsync), indepentent of what os is in use.

Autoexpansion tested on Raspberry Pi os (bookworm and older), Armbian, Manjaro-arm and ArchLinuxARM for rpi with ext4 root partition.
(Now also experimental btrfs functionality, please read further down)

Latest release: shrink-backup.v1.0.0
Testing branch: If you want to have the absolute latest version. There might be bugs.

Very fast restore thanks to minimal size of img file.

Can back up any device as long as root is ext4
Default device that will be backed up is determined by scanning what disk-device root resides on.
This means that if boot is a partition, that partition must be on the same device as root.
Backing up/restoring, to/from: usb-stick /dev/sdX with Raspberry pi os has been tested and works. Ie, writing an sd-card img to a usb-stick and vice versa works.

Ultra-fast incremental backups to existing img files.

See wiki for a bit more information about usage.
Ideas and feedback is always appreciated, whether it's positive or negative. Please just keep it civil. :)

Don't forget to make the script executable.

To restore a backup, simply "burn" the img file to a device using your favorite method.
When booting up a restored image with autoresize active, wait until the the reboot sequence has occured. The login prompt may very well become visible before the autoresize function has rebooted.

---

Usage

shrink-backup -h
Script for creating an .img file and subsequently keeing it updated (-U), autoexpansion is enabled by default
Directory where .img file is created is automatically excluded in backup
########################################################################
Usage: sudo shrink-backup [-Uatyelhz] [--loop] imagefile.img [extra space (MiB)]
  -U            Update existing img file (rsync to existing img)
                  Optional [extra space] extends img size of root partition
  -a            Autoresize root partition (extra space is ignored)
                  When used in combination with -U:
                  Expand if partition is >=256MiB smaller than resize2fs recommended minimum
                  Shrink if partition is >=512MiB bigger than resize2fs recommended minimum
  -t            Use exclude.txt in same folder as script to set excluded directories
                  One directory per line: "/dir" or "/dir/*" to only exclude contents
  -y            Disable prompts in script (please use this option with care!)
  -e            DO NOT expand filesystem when image is booted
  -l            Write debug messages to logfile shrink-backup.log located in same directory as script
  -z            Make script zoom at light-speed, only question prompts might slow it down
                  Can be combined with -y for UNSAFE ultra mega superduper speed
  --fix         Try to fix the img file if -a fails with a "broken pipe" error
  --loop [img]  Loop img file and exit, works in combination with -l & -z
                  If optional [extra space] is also defined, the img file will be extended with the amount before looping
                  NOTE that only the file gets truncated, no partitions
                  Useful for example if you want to manually manage the partitions
  -h --help     Show this help snippet
########################################################################
Examples:
sudo shrink-backup -a /path/to/backup.img (create img, resize2fs calcualtes size)
sudo shrink-backup -e -y /path/to/backup.img 1024 (create img, ignore prompts, do not autoexpand, add 1024MiB extra space)
sudo shrink-backup -Utl /path/to/backup.img (update img backup, use exclude.txt and write log to shrink-backup.log)
sudo shrink-backup -U /path/to/backup.img 1024 (update img backup, expand img size/root partition with 1024MiB)
sudo shrink-backup -Ua /path/to/backup.img (update img backup, resize2fs calculates and resizes img file if needed)
sudo shrink-backup -Ua --fix /path/to/backup.img 1024 (update img backup, automatically resizes img file if needed, fix img free space)
sudo shrink-backup -l --loop /path/to/backup.img 1024 (write to log file, expand IMG FILE (not partition) by 1024MiB and loop)

Zoom speed (-z)

The -z "zoom" option simply removes the one second sleep at each prompt to give the user time to read.
By using the option, you save 15-25s when running the script.
When used in combination with -y warnings will also be "bypassed"! PLEASE use with care!

Broken pipe (--fix)

Add --fix to your options if a backup fails during rsync with a "broken pipe" error. You can also manually add [extra space] instead of using -a to solve this.
The reason it happens is because rsync normally deletes files during the backup, not creating a file-list > removing files from img before starting to copy.
So if you have removed data from the system you backup from and added new data, the risk is rsync tries to copy the files before deleting other files from the img.
Using --fix makes rsync create a file-list and delete the files before backup. This also means the backup takes a little longer.
Having a "broken pipe" error during backup has in my experience never broken an img backup after either repairing or increasing space.

Loop img file (--loop)

Use --loop to loop an img file to your /dev.
If used in combination with [extra space] the amount in MiB will be added to the IMG FILE NOT any partition.
With this you can run for example run sudo gparted /dev/loop0 (if you have a graphical interface) to manually manage the img partitions in a graphical interface.
If you added [extra space] this will then show up as unpartitioned space at the end of the device.
This functionality works on any linux system, just use the script on any img file anywhere available to the computer.
To remove the loop: sudo losetup -d /dev/loop0, change loop0 to the correct dev it got looped to. lsblk /dev/loop* if you lost the information after using --mount

Log file (-l)

Use -l to write debug info into shrink-backup.log file located in the same directory as the script.

exclude.txt (-t)

The folder where the img file is created will ALWAYS be excluded in the backup.
If -t option is selected, exclude.txt MUST exist (but can be empty) within the directory where the script is located or the script will exit with an error.
Use one directory per line in exclude.txt.
/directory/* = create directory but exclude content.
/directory = exclude the directory completely.

If -t is NOT selected the following folders will be excluded:

/lost+found
/proc/*
/sys/*
/dev/*
/tmp/*
/run/*
/mnt/*
/media/*
/var/swap

Info

Rsync WILL cross filesystem boundries, so make sure you exclude external drives unless you want them included in the backup.
Not excluding other partitions will copy the data to the img root partition, not create more partitions so make sure to manually add extra space if you do this.
The script will ONLY look at your root partition when calculating sizes.

Applications used in the script:

• fdisk
• sfdisk
• dd
• parted
• e2fsck
• truncate
• mkfs.ext4
• rsync
• gdisk (sgdisk is needed if the partition table is GPT, the script will inform you)

---

Image creation

To create a backup img using recomended size, use the -a option and the path to the img file.
Example: sudo shrink-backup -a /path/to/backup.img

Theoretically the script should work on any device as long as root filesystem is ext4. But IMHO is best applied on ARM hardware.
Since the script uses lsblk to crosscheck with /etc/fstab to figure out where the root resides it does not matter what device it is on.
Even if you forget to disable autoexpansion on a non supported system, the backup will not fail. :)

Order of operations - Image creation

1. Uses lsblk & /etc/fstab to figure out the correct disk device to back up.
2. Reads the block sizes of the system's root (and boot if it exists) partition.
3. Uses dd to create the boot part of the system + a few megabytes to include the filesystem on root. (this can be a partition)
4. Uses df & resize2fs to calculate sizes by analyzing the system's root partition. (For btrfs btrfs filesystem du + 192MiB is used instead of resize2fs)
5. Uses truncate to resize img file.
6. Loops the img file.
7. Removes and recreates the root partition on the loop of the img file.
8. Creates the root filesystem on loop of the img file with the same UUID and LABEL as the system you are backing up from.
9. Creates a temp directory and mounts img file root partition from loop.
10. Checks if boot partition exists, if true, checks fstab and creates directory on root and mounts accordingly from loop.
11. Uses rsync to sync filesystems.
12. Tries to create autoresize scripts if not disabled with -e.
13. Unmounts and removes temp directory and file (file created for rsync log output).

Added space is added on top of df reported "used space", not the size of the partition. Added space is in MiB, so if you want to add 1G, add 1024.

The script can be instructed to set the img size by requesting recomended minimum size from e2fsck by using the -a option.
This is not the absolute smallest size you can achieve but is the "safest" way to create a "smallest possible" img file.
If you do not increase the size of the filesystem you are backing up from too much, you can most likely keep it updated with the update function (-U) of the script.
By using -a in combination with -U the script will resize the img file if needed. Please see section about image update further down for more information.

Smallest image possible

To get the absolute smallest img file possible, do NOT use -a option and set "extra space" to 0

Example: sudo shrink-backup /path/to/backup.img 0

This will instruct the script to get the used space from df and adding 128MiB "wiggle room".
If you are like me, doing a lot of testing, rewriting the sd-card multiple times. The extra time it takes each time will add up pretty fast.

Example:

-rw-r--r-- 1 root root 3.7G Jul 22 21:27 test.img # file created with -a
-rw-r--r-- 1 root root 3.3G Jul 22 22:37 test0.img # file created with 0

Disclaimer!
Because of how filesystems work, df is never a true representation of what will actually fit in a created img file.
Each file, no matter the size, will take up one block of the filesystem, so if you have a LOT of very small files (running docker f.ex) the "0 added space method" might fail during rsync. Increase the 0 a little bit and retry.
This also means you have VERY little free space on the img file after creation.
If the filesystem you back up from increases in size, an update (-U) of the img file might fail.
By using -a in combination with -U the script will resize the img file if needed. Please see section about image update below for more information.
Using combination-Ua on an img that has become overfilled works, or at least I have not gotten it to fail, but use at own risk.

---

Image update

To update an existing img file simply use the -U option and the path to the img file.
Example: sudo shrink-backup -U /path/to/backup.img

Order of operations - Image update

1. Loops the img file.
2. Probes the loop of the img file for information about partitions.
3. If -a is selected, calculates sizes by comparing root sizes on system and img file by using fdisk & resize2fs.
4. Expands filesystem on img file if needed or if manualy added [extra space] is used.
5. Creates temp directory and mounts root partition from loop.
6. Checks if boot partition exists, if true, checks fstab and creates directory on root and mounts accordingly from loop.
7. Uses rsync to sync filesystems.
8. Shrinks filesystem on img file if -a was used and conditions were met in point 3.
9. Tries to create autoresize scripts if not disabled with -e.
10. Unmounts and removes temp directory and file (file created for rsync log output).

Resizing img file when updating

If -a is used in combination with -U, the script will compare the root partition on the img file to the size resize2fs recommends as minimum.
The img file root partition needs to be >=256MB smaller than resize2fs recommended minimum to be expanded.
The img file root partition needs to be >=512MB bigger than resize2fs recommended minimum to be shrunk.
This is to protect from unessesary resizing operations most likely not needed.

If manually added [extra space] is used in combination with -U, the img file's root partition will be expanded by that amount. No checks are being performed to make sure the data you want to back up will actually fit.
Only expansion is possible with this method.

---

btrfs

ALL testing has been done on Manjaro-arm
THIS IS NOT A CLONE, IT IS A BACKUP OF REQUIRED FILES FOR A BOOTABLE BTRFS SYSTEM!

All options in script should work just as on ext4. The script will detect btrfs and act accordingly.
The script will treat snapshots as nested volumes, so make sure to exclude snapshots if you have any, or directories and nested volumes will be created on the img file. This can be done in exclude.txt, wildcards should work.
When starting the script, the initial report window will tell you what volumes will be created. Make sure these are correct before pressing Y

As of now, top level subvolumes are checked for in /etc/fstab and mounted accordingly, mount options should be preseved (for exmaple if you change compression).
Autoresize function works on Manjaro-arm.

---

Thank you for using my software <3

"A backup is not really a backup until it has been restored"