Currently, I experiment with a possibility to backup the whole filesystem to a single compressed file. During the backup process, I want to conveniently preview the directory structure which will be actually backed up.

Overview

I have prepared separate scripts for multiple stages of this process. While doing a backup, I run them in order.

The scripts have hardcoded paths - review and edit the scripts and files below before using them.

Stage 1: Mount a filtered filesystem

#/bin/bash

mkdir -p /tmp/to_backup
rofs-filtered /tmp/to_backup -o source=/ -o config=/home/user/bin_settings/rofs-filtered.rc

rofs-filtered is available at GitHub or as AUR package for Arch Linux.

The rofs-filtered.rc file contains the filter rules as regular expressions. E.g. the contents of my file are:

^/bin$
^/dev$
^/lib$
^/lib64$
^/lost\+found$
^/media$
^/misc$
^/mnt$
^/net$
^/opt$
^/proc$
^/run$
^/sbin$
^/sys$
^/tmp$
^/usr$
^/var/cache$
^/var/lib/docker$
^/var/lib/jenkins/caches$
^/var/lib/jenkins/\.gradle$
^/var/lib/jenkins/\.npm$
^/var/lib/jenkins/\.nuget$
^/var/lib/jenkins/jobs/[^/]+/builds$
^/var/log$
^/home/user/\.android/avd$
^/home/user/\.cache$
^/home/user/\.config/Code/User/workspaceStorage$
^/home/user/\.config/unity3d/cache$
^/home/user/\.local/share/NuGet$
^/home/user/\.local/share/Trash$
^/home/user/\.local/lib$
^/home/user/\.gradle$
^/home/user/\.npm$
^/home/user/\.nuget$
^/home/user/\.minikube/cache$
^/home/user/Unity/Hub$
^/home/user/VirtualBox VMs$
^/root/\.cache$
^/root/\.gradle$
^/root/\.npm$
^/root/\.nuget$
/\.metadata$
/node_modules$

Stage 2: Check what occupies the most space

#/bin/bash

filelight /tmp/to_backup

filelight is a GUI application which shows you a graphical representation of the space occupied by the parts of the filesystem tree. Mostly, this application can be obtained from standard Linux distribution repositories.

Stage 3: Check the backup size

#/bin/bash

tar -czf - /tmp/to_backup | pv > /dev/null

This only tries to create the compressed backup, without storing it anywhere. It is faster than Stage 4 and the result does not occupy any disk space.

If the result of this step shows that the backup is too large, I return to Stage 3 to see the largest directories, and then possibly return to Stage 2 to edit the filesystem filter.

Stage 4: Execute the backup

#/bin/bash

DATETIME=$(date '+%Y%m%d_%H%M%S')

tar -czf - /tmp/to_backup | pv > "/mnt/backup_server/zip/${DATETIME}_${RANDOM}.tar.gz"

This creates the same backup file as Stage 3 and actually stores it somewhere.

Stage 5: Unmount the filtered filesystem

#/bin/bash

fusermount -u /tmp/to_backup