Backup design

Keep independent copies

Protect backups from server loss, operator mistakes, account compromise, and ransomware with separate failure boundaries.

8 minute lesson

~~~

A second directory on the same disk is a copy, but it does not survive disk or server loss.

The question to ask of every backup copy is: which failures does this one survive? A copy on the same disk survives an accidental rm. It does not survive the disk dying, the server being deleted, or ransomware encrypting the filesystem. Each copy is only as good as the failure boundary between it and the original.

Layer the copies

Keep multiple copies on different media, with at least one off-site. The classic formulation is the 3-2-1 pattern: three copies of the data, on two different media, one of them off-site.

A local copy on a second disk handles the common cases fast:

sudo rsync -aHAX --delete /srv/data/ /mnt/backups/data/

An off-site copy over SSH survives losing the whole machine:

sudo rsync -aHAX /srv/data/ [email protected]:/srv/copies/web01/data/

Note the --delete flag appears only on the local mirror. On the off-site copy I leave it off, or better, use a snapshotting tool like restic or borg that keeps history. Here’s why that matters.

Replication is not backup

Replication alone can faithfully copy deletion and corruption. A mirror that syncs every hour will dutifully propagate tonight’s ransomware encryption, or this afternoon’s fat-fingered rm -rf, to every replica by morning. What saves you is history: older versions that no current process can rewrite.

Independent credentials

If one stolen SSH key or cloud token can delete both the server and its backups, they are not independent copies. Give the backup destination its own credential that can only append, and administer retention from the storage side. Use separate credentials and immutable or offline retention where appropriate — object-lock on a bucket, or a disk that spends most of its life unplugged.

Map three plausible failures against your current copies. For each failure, name the copy that survives and the credential needed to restore it.

failure: data disk dies       -> survives: off-site copy  (backup SSH key)
failure: rm -rf on /srv/data  -> survives: local mirror   (root on server)
failure: server account stolen -> survives: ???           <- fix this one

If any row ends in question marks, that’s your next piece of work.

Lesson completed

Take this course offline

Get every free book and course as PDF and EPUB files.

Get the download library →