Backups

Backups are a critical part of your infrastructure management. Your business records, transactions, and configuration are all incredibly valuable - and ideally you invest a regular part of your schedule into ensuring that those components are salvageable.

Stuff happens. Back it up!

In every case described below, SHUT DOWN CAS before performing the backup. Data is constantly being recorded by CAS, and no matter which method you choose - you risk a corrupt (and unusable) backup if CAS is running during that backup.

If you’re going to take the time to backup - do it right - shut down CAS before beginning.

Test your backups from time-to-time. I can sarcastically assure you - there’s no greater joy than discovering your 7 years of backups are completely unusable after your system crashed.

Scripting Notes: Security

When uploading backups/files to an offsite location, keep security in tight focus. While the suggestions here are not the only possible approach, they should be a baseline.

Your backups are more important than your CAS host. Script accordingly.
Example: dial IN to your CAS host (for SCP/RSYNC) using a restricted account.
1. Permitting your backup script to dial OUT may expose your backups to an attacker!
2. The restricted account should definitely not have sudo access to your CAS server, and
3. the backups should be stored in a folder that the restricted user can access.

Digital Ocean Snapshot

Useful before a major CAS upgrade or OS update. Minimal effort. Fast results.

Requires Digital Ocean, but most cloud providers have something similar.
See: https://docs.digitalocean.com/products/snapshots/

The droplet state can be imaged (and quickly restored) with just a few clicks.

Individual files and settings cannot be cherry-picked - the entire droplet must be restored.
The snapshots are stored on your cloud account, further reducing your effort & increasing security.

Digital Ocean Backup

See: https://cloud.digitalocean.com/images/backups

Useful on a routine basis where reduced configuration and automation is desired.

Better than no backups at all, but any automation with this process cannot shutdown CAS - so there is a tiny risk of backup corruption - which means you may be paying for (and relying on) something worthless.
The most expensive option.
Options are: daily & weekly. Backup cannot be performed on-demand.
Cannot be downloaded for long term storage.

Server Backup

This scripted option can automatically backup the important folders on your host.

Includes CAS and any other components (e.g. NGINX) that you wish to include.
Maximum initial effort. This process can provide the most robust protection, but requires debugging and occasional monitoring to ensure that it is doing exactly what you believe it to be doing.
Depends upon additional offsite (off-Droplet) storage because the archives may be large and will eventually (quickly?) deplete your Droplet storage space.

Benefits:

Offers the finest file-level control for restoring only necessary files (or folders).
Free (except for off-Droplet storage).
A more comprehensive backup than using batm-manage backup alone.
Backups are saved locally on the Droplet, and can be downloaded offsite on-demand.

Drawbacks:

The backups will have to be periodically (and manually) pruned off the Droplet to conserve space.

Suggested approach:

https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories

# shut down CAS
sudo /batm/batm-manage stop all
# if scripting offsite storage, change the folder to one that the user has permission to access
BACKUP_FILE="/var/backups/server-backup-$(date +%F).tar.gz"
# modify as needed to include any other critical folders
BACKUP_FOLDERS="/batm /etc/nginx /etc/openvpn /var/www /var/log/batm /root /home"
sudo tar -czvf "$BACKUP_FILE" $BACKUP_FOLDERS
# start CAS
sudo /batm/batm-manage start all

NOTE: remove the /rootfolder from the list OR be sure to remove any sensitive data in /root/.ssh (such as private keys) that might enable an attacker with access to your backup to get your keys.

To password protect the backup (strongly encouraged) using ZIP, modify with this:

PASS="ThisPasswordForBackupsONLY"
# change the backup filename to ".zip" so that the decompressor can properly recognize it, as so:
BACKUP_FILE="/var/backups/server-backup-$(date +%F).zip"
zip -y -r -P "$PASS" "$BACKUP_FILE" $BACKUP_FOLDERS

You may have to install ZIP to use this (sudo apt update && sudo apt install zip).
This password implementation is only useful when the archive is stored off-Droplet. Anyone with access to the Droplet could uncover the password - but it should protect the archive when moved off-Droplet (downloaded to another system) and that is the intended security applied here.
When password protected, you can safely backup the /root folder (which can be important).

CAS (only) Backup

Backup CAS as often as practical. Since the backup requires that CAS be shut down, it may not be prudent to perform this more frequently than daily. If you can recreate the transactions from the past week (using the logs), then backup weekly. Monthly? Whatever time frame is practical for you - but definitely make backups a regular routine.

batm-manage

These individual components can be archived on demand, and are saved to a new folder in /batm/backup.

Example 1: sudo /batm/batm-manage backup

All components will be backed up if no components/parameters are specified.

Example 2: sudo /batm/batm-manage backup “data db”

only archives your data & database.

Component	Backup details

Component	Backup details
config	This folder: `/batm/config` which contains your CAS startup configuration, and some 3rd-party API settings (e.g. Telegram, Onfido, etc).
app	This folder: `/batm/app` which includes the CAS binaries, modules, schemas, audit files, extensions, acceptor packages, and heap dumps.
data	This folder: `/batm/data` which includes selfies, custom graphics, fingerprint hashes, screenshots, server dumps, captures, and videos.
vpn	The CAS VPN configuration files. Does NOT include a full backup of OpenVPN - just the files relating to the CAS configuration when using OpenVPN. To perform a FULL backup of OpenVPN, see: Backups \| OpenVPN
db	The MySQL database (only).
batm-manage	Exclusively this file: `/batm/batm-manage` which is not backed up by any other method.

Custom Backup Scenarios

OpenVPN

Backup OpenVPN before cleaning the VPN configuration using batm-manage vpn-clean.

The files created by OpenVPN are signed and only valid locally - that is: they cannot be transferred to another computer or restored to a new OS image. They must be created by the OpenVPN setup process. If you merely want to backup in the possible event of a minor misstep or error, then the backup created by this command will work. It can be restored if something explodes (just a little).

sudo tar -czvf /batm/backup/openvpn-backup.tar.gz /etc/openvpn

Logs

Logs contain enough information to clearly recreate most transactions, and should be stored for appropriate time periods as specified by your region.

Your logs are stored in /var/log/batm and they are automatically compressed and archived weekly.