It is often said, yet too often forgotten: taking backups (and verifying that they work). Taking backups is not purely for companies and organizations. Individuals should also take backups to ensure that, in case of errors or calamities, the all important files are readily recoverable.
For backing up files and directories, I personally use obnam, after playing around with Bacula and attic. Bacula is more meant for large distributed environments (although I also tend to use obnam for my server infrastructure) and was too complex for my taste. The choice between obnam and attic is even more personally-oriented.
I found attic to be faster, but with a small supporting community. Obnam was slower, but seems to have a more active community which I find important for infrastructure that is meant to live quite long (you don't want to switch backup solutions every year). I also found it pretty easy to work with, and to restore files back, and Gentoo provides the app-backup/obnam package.
I think both are decent solutions, so I had to make one choice and ended up with obnam. So, how does it work?
Configuring what to backup
The basic configuration file for obnam is
/etc/obnam.conf. Inside this file,
I tell which directories need to be backed up, as well as which subdirectories
or files (through expressions) can be left alone. For instance, I don't want
obnam to backup ISO files as those have been downloaded anyway.
[config] repository = /srv/backup root = /root, /etc, /var/lib/portage, /srv/virt/gentoo, /home exclude = \.img$, \.iso$, /home/[^/]*/Development/Centralized/.* exclude-caches = yes keep = 8h,14d,10w,12m,10y
root parameter tells obnam which directories (and subdirectories) to
back up. With
exclude a particular set of files or directories can be
excluded, for instance because these contain downloaded resources (and as such
do not need to be inside the backup archives).
Obnam also supports the CACHEDIR.TAG
specification, which I use for the various cache directories. With the use of
these cache tag files I do not need to update the
obnam.conf file with every
new cache directory (or software build directory).
The last parameter in the configuration that I want to focus on is the
parameter. Every time obnam takes a backup, it creates what it calls a new
generation. When the backup storage becomes too big, administrators can run
obnam forget to drop generations. The
keep parameter informs obnam which
generations can be removed and which ones can be kept.
In my case, I want to keep one backup per hour for the last 8 hours (I normally take one backup per day, but during some development sprees or photo manipulations I back up multiple times), one per day for the last two weeks, one per week for the last 10 weeks, one per month for the last 12 months and one per year for the last 10 years.
Obnam will clean up only when
obnam forget is executed. As storage is cheap,
and the performance of obnam is sufficient for me, I do not need to call this
Backing up and restoring files
My backup strategy is to backup to an external disk, and then synchronize this disk with a personal backup server somewhere else. This backup server runs no other software beyond OpenSSH (to allow secure transfer of the backups) and both the backup server disks and the external disk is LUKS encrypted. Considering that I don't have government secrets I opted not to encrypt the backup files themselves, but Obnam does support that (through GnuPG).
All backup enabled systems use cron jobs which execute
obnam backup to take
the backup, and use rsync to synchronize the finished backup with the backup
server. If I need to restore a file, I use
obnam ls to see which file(s) I
need to restore (add in a
--generation= to list the files of a different
backup generation than the last one).
Then, the command to restore is:
~# obnam restore --to=/var/restore /home/swift/Images/Processing/*.NCF
Or I can restore immediately to the directory again:
~# obnam restore --to=/home/swift/Images/Processing /home/swift/Images/Processing/*.NCF
To support multiple clients, obnam by default identifies each client through the hostname. It is possible to use different names, but hostnames tend to be a common best practice which I don't deviate from either. Obnam is able to share blocks between clients (it is not mandatory, but supported nonetheless).