Rclone: Back Up a Server to the Cloud and S3

Rclone is a command-line tool for working with cloud storage. It supports more than 70 services — S3-compatible storage, Google Drive, Dropbox, OneDrive, SFTP, WebDAV — and uses the same commands for all of them. For a server it is the most versatile way to send backups off the machine.

Installation

The official script installs the latest version:

sudo -v ; curl https://rclone.org/install.sh | sudo bash

Distribution repositories carry rclone too (apt install rclone), but the version there is often much older. Check the installation:

rclone version

Connecting storage

In rclone terms a storage location is called a remote. Start the configuration wizard:

rclone config

Press n, give it a name (for example backup), choose the storage type from the list and answer the questions. The settings are saved in ~/.config/rclone/rclone.conf.

Example: S3-compatible storage

Most S3-compatible services need an access key, a secret key and an endpoint. The same settings can be entered with a single command instead of the wizard:

rclone config create backup s3 \
  provider=Other \
  access_key_id=YOUR_KEY \
  secret_access_key=YOUR_SECRET \
  endpoint=https://s3.example.com

Your own Garage storage is connected the same way.

Example: a server over SFTP

Any server with SSH access can serve as storage:

rclone config create backup sftp host=203.0.113.20 user=backup key_file=~/.ssh/id_ed25519

Storage with browser login

Google Drive, Dropbox and OneDrive require authorisation in a browser, which a server does not have. The wizard asks you to run rclone authorize on your own computer and paste the resulting code on the server.

Essential commands

A path on a remote is written as name:path.

rclone lsd backup:                      # list directories
rclone ls backup:site                   # list files with sizes
rclone copy /var/www/site backup:site   # copy new and changed files
rclone sync /var/www/site backup:site   # make the copy identical to the source
rclone check /var/www/site backup:site  # compare source and copy
rclone size backup:site                 # total size and file count
rclone delete backup:site/old           # delete files

The difference between copy and sync matters: copy only adds and updates files, while sync also deletes everything on the remote that is not in the source.

Useful options

Option	What it does
--dry-run	show the actions without performing them
-P	show progress
--transfers 8	number of parallel transfers (default 4)
--bwlimit 10M	limit the speed
--exclude "cache/**"	exclude files by pattern
--max-age 24h	only files changed in the last 24 hours
--backup-dir backup:old	move deleted and replaced files to a separate directory
--log-file /var/log/rclone.log	write a log file

Encrypting backups

Backups in someone else’s cloud should be encrypted. Rclone does this transparently: you create another remote of type crypt that encrypts file names and contents and stores them in a regular remote.

Run rclone config, create a remote named secure of type crypt, set backup:encrypted as the target and choose a password. From then on, use secure: like any other remote:

rclone sync /var/www/site secure:site

The cloud will hold files with unreadable names. The encryption password is stored in rclone.conf in a reversible form, so keep a separate copy in a password manager: without it the data cannot be recovered.

Scheduled backups

Create the script /usr/local/bin/backup-cloud.sh:

#!/bin/bash
set -e
DATE=$(date +%F)
DUMP=/var/backups/db-$DATE.sql.gz

# database dump
mysqldump --single-transaction --all-databases | gzip > "$DUMP"

# site files: a mirror that keeps deleted versions
rclone sync /var/www secure:www \
  --backup-dir "secure:www-archive/$DATE" \
  --exclude "cache/**" \
  --log-file /var/log/rclone.log --log-level INFO

# database dump: upload it and delete local copies older than 7 days
rclone copy "$DUMP" secure:db --log-file /var/log/rclone.log
find /var/backups -name 'db-*.sql.gz' -mtime +7 -delete

# keep dumps in the cloud for 30 days
rclone delete secure:db --min-age 30d

Make the script executable and add it to cron:

chmod +x /usr/local/bin/backup-cloud.sh

0 3 * * * flock -n /tmp/backup-cloud.lock /usr/local/bin/backup-cloud.sh

Verification and restore

A backup you have never restored from is a hope, not a backup. Check integrity regularly:

rclone check /var/www secure:www

Restoring is the same command in the opposite direction:

rclone copy secure:www/site /var/www/site-restored -P

Mounting storage as a folder

sudo apt install fuse3
mkdir -p /mnt/cloud
rclone mount backup:site /mnt/cloud --daemon --vfs-cache-mode writes

The remote appears at /mnt/cloud as an ordinary directory. Unmount it with fusermount3 -u /mnt/cloud. Mounting is convenient for browsing and occasional operations; for backups, copy and sync are more reliable.

Choosing a backup tool

Rclone is excellent at moving files to the cloud, but it does not do deduplication or snapshots with history by itself. It is often paired with another tool: Borg Backup creates a local repository and rclone ships it to the cloud. For copying between two servers, rsync is simpler.

You can also keep the copies on separate UnixHost backup storage — rclone connects to it over SFTP.