The Value of Backups

All hardware has a finite life. Mechanical hard disk drives (HDDs) and solid-state drives (SSDs) don’t last forever. Accidents happen, too. Laptops can be lost, stolen, or dropped down the stairs.

It used to be said that the value of an effective backup system only becomes clear once you’ve lost data. When failures or losses occur, you need to have a fast and easy way to get your files and information back. If an organization loses data, the implications are severe. It could even jeopardize business continuity. Even in a domestic setting, data loss can be a painful experience. Backups are the only sensible safeguards.

And on top of everything else, the accidental—or malicious—loss of personally identifiable data is considered a breach under some data protection legislation, such as the General Data Protection Regulation (GDPR).

There’s a couple of considerations to observe when you choose backup software. Where do you want your backups to be kept? On a removable drive, on another machine across your local area network (LAN), or in cloud storage? Obviously, you need to use a backup program that can write and restore from the data storage you wish to use.

Backups should be encrypted, especially if they are going to be stored in off-premise locations or in the cloud. If they are encrypted, they cannot be read and restored by unauthorized people.

The program should be fast. You don’t want to wait all day for a backup or restore to complete. Some programs store a base backup image and then store the differences between the base image and the source machine for each subsequent backup. This speeds up the backup process considerably. It also uses less space for your backups.

restic does all of this. It is free, open-source, licensed under the 2-Clause BSD License, and under active development. The source code is on GitHub.

Where to Back up to

In this article, we’re going to be storing our backups on another computer across our network. That’s great, because that provides fast file transfers, and it’s easy to back up and restore from. In a real-world scenario, you really need to back up to another off-site location as well. If your live systems and backups are at the same location and a disaster occurs at that location—a fire, a theft, or a flood—your goose is going to be cooked unless you have an off-premise backup.

Unsurprisingly, restic can back up to an external drive that can be taken away from the premises, and—even better—it can back up straight to cloud storage.

Out of the box, restic can back up to:

A local directory or local removable drive. A networked computer via SSH File Transfer Protocol (SFTP). This, of course, requires Secure Shell (SSH). HTTP REST server. AWS S3. OpenStack Swift. BackBlaze B2. Microsoft Azure Blob Storage. Google Cloud Storage.

If you need to back up to a data destination that isn’t in that list, you can combine the power of rclone with restic and back up to any of the 40 or so destinations that rclone supports.

SFTP network backups require SSH to be installed and configured on the backup server. This is the machine that the backups will be stored on. If you set up SSH keys on the backup server and the machine you’re going to back up, you won’t be prompted for the SSH password every time you run a backup.

One way to automate your backups is to create short scripts or shell functions and use cron to run them at specified times. Using SSH keys will allow you to avoid the problem of providing a password for unattended backups.

RELATED: How to Create and Install SSH Keys From the Linux Shell

Installing restic

The restic application is in the repositories of the main Linux distributions, so installing it is a simple one-liner using the package manager of each distribution.

To install restic on Ubuntu, type:

The command to use on Fedora is:

On Manjaro we use pacman:

Setting up restic

Make sure you have SSH set up on the backup server machine, and that you can make a remote connection to it from the machine you’re going to back up. That’s the client machine. In our test network, the client is called “ubuntu-20-10,” and the server is called “backup-box.”

In restic’s terminology, backups are stored as snapshots in a repository. Each backup creates a new snapshot. We need to create a place for the repository on the server.

We need to create a directory on the backup server to hold the repository. Historically, services that were served up by a server were located in the “/srv” directory. So we’ll place our repository there.

On the backup server, issue this command. You can name the repository directory whatever you like. We’re using the name “restic” for the sake of simplicity.

We need to make sure that this directory is accessible to the person who will be handling backups. If it were multiple people, it would make sense to create a user group and give the group access to the directory.

Let’s check the settings on the directory:

Now we can move over to the client machine and create the repository on the server from there. Substitute the name of your user, the name of the backup server, and the name of the repository directory to match your choices. You can use the IP address of the backup server if you prefer.

We’re using the -r (repository) option to specify the path to the repository that we’re going to create. The restic initcommand initializes the repository.

You’ll be asked for the password for the user account on the backup server. If you’ve set up SSH keys between the server and the client, you won’t need to perform this step.

You’ll also be asked for the repository password, and then you’ll be asked to confirm it. This password must be used to interact with the repository in the future. Don’t lose it! You won’t be able to back up or restore the data if you lose the password.

It only takes a moment for the repository to be created and initialized.

Creating a Backup

Creating a backup is very simple. We use the backup command with restic, tell it what we want to back up, and which repository to send the backup to.

You need to provide the user’s password and the password for the repository. While the backup is running, the names of the files being copied are displayed, along with statistics telling you how many files are going to be copied in total, how many have been copied so far, and what percentage of the backup is complete. The snapshots are encrypted using the advanced encryption standard AES-256.

Because this was the first backup to this repository, all the files that were backed up were new. We did say that restic was fast—over 70,000 files were backed up in 23 seconds. That’s all of the source code for the Linux kernel.

On another test machine, I backed up over 350,000 files amounting to over 170 GiB in an hour and a half.

I created a new file on the client in the source directory and ran another backup. The command is the same as before.

The source directory tree was scanned for changes, the new file was detected, and it was backed up. That second tiny backup took three seconds to complete, including scanning the other files for changes.

Let’s take a look at the two snapshots we have in the repository. The restic command for this is snapshots.

Each snapshot has been given a hexadecimal identifier as a unique ID. The date and time when each snapshot was created are displayed. The name of the computer that the backup was made from and the path to the data that was backed up is also shown.

I then created a second new file and created another backup. Again, the command line is the same as before.

Just like with our earlier top-up backup, this tiny update took three seconds to complete.

By now, you’re probably tired of entering the repository password. We can address that before we use the snapshots command to look at our collection of three snapshots. Open an editor and type the repository password in it, then hit “Enter” to start a new line. Save the file as “.rest_pass” in your home directory.

To make sure that no one else can see the password, change the access mode bits of the file using chmod :

This means that no one but you can access the file.

Now we can pass this to the rest command line using the -p (password file) option. If you’ve also set up SSH keys between the client and the server, you won’t have to enter the user account password either. You can easily automate your backups with cron once the human interaction has been removed from the process.

We’re no longer asked for the repository password, which is great. We don’t need to remember it and we can’t mistype it.

Working With Snapshots

The restic diff command lets you see the differences between any two snapshots. Use the unique IDs of the two snapshots you want to compare. You can see the snapshot IDs when you use the restic snapshot command.

The differences between the snapshots are shown as columns of statistics.

The restic check command performs a verification test against all the snapshots in the repository.

To delete a snapshot, you must tell restic to forget it and to prune it. You must use the snapshot’s unique ID to identify which snapshot to delete.

Restoring Data

When the time comes to restore data from your backups, it’s as easy as it was to create the backup. You need to specify which snapshot you’re going to restore. You can use a snapshot’s unique ID, or you can use the latest label to use the newest snapshot in the repository.

You also need to provide a directory for the restored data to be copied to using the target option.

Restoring is as speedy as backing up. Checking in the target directory, we can see that the directory tree and files have been restored for us.

Make Backups, Sleep Easy

Data loss is a serious problem. A robust backup solution means that you have one less thing to worry about. With restic, you can automate your backups to local and cloud repositories and sleep easy.