Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: setup borg cli #30

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Refactor: setup borg cli #30

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
40b4acb
refactor borg note
sayem314 Aug 24, 2023
180818f
update config comments
sayem314 Aug 24, 2023
b3ba897
refactor intro texts
sayem314 Aug 25, 2023
593d677
refactor prerequisites texts
sayem314 Aug 25, 2023
d4e4266
remove dollar sign
sayem314 Aug 25, 2023
c8aa63d
update install step
sayem314 Aug 25, 2023
86ca4b4
update tabs name
sayem314 Aug 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 72 additions & 49 deletions setup/borg/cli.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ Reviewed in August 2023
You should be comfortable using the command line. If you prefer a graphical, client, look into our [Vorta Desktop Client](/setup/vorta) instead. These instructions should work on macOS and popular Linux flavors, like Debian, Ubuntu, as well as Red Hat, Fedora and CentOS.

{: .note }
Borg will get a [major upgrade](https://borgbackup.readthedocs.io/en/2.0.0b3/changes.html#change-log-2-x) to version 2 later this year, which brings many improvements and removes legacy code- and features. Most commands stay the same. Where there is a difference, you can choose the right version using the tabs above the code snippet. _BorgBase_ itself supports all the latest release of every major branch and you can choose the preferred version for each repository.
Borg will receive a [major upgrade](https://borgbackup.readthedocs.io/en/2.0.0b6/changes.html#change-log-2-x) to version 2 later this year, bringing many improvements and removing legacy code and features. Most commands will remain the same. In cases where there are differences, you can select the appropriate version using the tabs above the code snippet. As for _BorgBase_, it supports the latest release of every major branch, allowing you to choose the preferred major version for each repository.

### Introduction
## Introduction

In this article we will set up a backup software called Borg Backup (or Borg for short). There are other backup solutions for Linux or macOS, but Borg has all the features we would expect from a proper backup, like encryption, compression and deduplication.
This guide focuses on configuring Borg Backup (referred to as Borg). While there are alternative backup solutions for Linux and macOS, Borg excels by offering a comprehensive array of features, including encryption, compression, and deduplication.

## Prerequisites

This tutorial assumes you are using a recent Linux, \*BSD or macOS machine. You don't need to be a command line expert. All commands can be copy & pasted.
Before proceeding, ensure that you have access to a modern Linux, \*BSD, or macOS system. You don't need to be a command-line expert, as all commands can be copied and pasted for simplicity.

It also assumes that you have server-based version of Borg available. The easiest option is to sign up for [BorgBase.com](https://www.borgbase.com), which offers a free tier of 10 GB. Other options are your own server with SSH access or one of the providers linked [here](https://www.borgbackup.org/support/commercial.html).
Additionally, please note that you should have a server-based version of Borg available. The most straightforward option is to register on [BorgBase.com](https://www.borgbase.com), which includes a free 10 GB tier. Alternatively, you can use your own server with SSH access or select from the providers listed [here](https://www.borgbackup.org/support/commercial.html).

## Step 1 - Install Borg

Expand All @@ -42,66 +42,90 @@ If you use Docker or a similar container tool, you can also use this [Borgmatic

If you already keep some Python tools around, you can just add Borg to it:

```
$ pip install borgbackup
```shell
pip install borgbackup
```

### Install as Pre-Compiled Binary

If you don't manage a set of Python tools, the simplest way to install Borg is to download the latest release from their [Github page](https://github.com/borgbackup/borg/releases). This binary will have everything you need in one file.
If you don't manage a set of Python tools, the simplest way to install Borg is to download the latest release from their [Github page](https://github.com/borgbackup/borg/releases/latest). This binary will have everything you need in one file.

First we move to the **Downloads** folder, where we downloaded Borg to.

```
$ cd ~/Downloads
```shell
cd ~/Downloads
```

Then give execute permissions to Borg.

{% tabs os_commands %}

{% tab os_commands MacOS %}

```shell
chmod +x borg-macos64
```
$ chmod +x borg-macosx64

{% endtab %}

{% tab os_commands Linux %}

```shell
chmod +x borg-linux64
```

Last, move it to a standard folder for applications. Depending on your setup you may need to enter your admin-password to write to this folder.
{% endtab %}

{% tab os_commands *BSD %}

```shell
chmod +x borg-freebsd64
```
$ sudo mv borg-macosx64 /usr/local/bin/borg

{% endtab %}

{% endtabs %}

Finally, move it to a standard folder for applications. Depending on your setup you may need to enter your admin-password to write to this folder.

```shell
sudo mv borg-macosx64 /usr/local/bin/borg
```

That's it. You now have Borg installed on your local machine. You can test it by simply starting it and checking the version

```
$ borg --version
```shell
borg --version
```

This should print something like `borg 1.2.2`.
This should print something like `borg 1.2.4`.

## Step 2 – Install Python 3 and Borgmatic (optional)

After completing step 1, you are ready to use Borg for simple operations. If you want to schedule regular backup jobs and exclusions, it's a good idea to install a helper tool – Borgmatic – as well. This time Python is required and there is no pre-compiled binary.

First make sure, you have a recent version of Python installed. On macOS, use Homebrew:

```
$ brew install python3
```shell
brew install python3
```

On Linux, use the system's package manager, like Apt:

```
$ apt install python3
```shell
apt install python3
```

With Python working, you can install Borgmatic. For system-wide backups, you should install it as global system package.

```
$ sudo pip3 install borgmatic
```shell
sudo pip3 install borgmatic
```

If you plan on only backing up your home folder, installing it as user-package will work too:

```
$ pip3 install --user borgmatic
```shell
pip3 install --user borgmatic
```

There are many other considerations in installing Python packages. If you get an error, make sure you know which Python installation you are using and what permissions are needed.
Expand All @@ -112,16 +136,16 @@ In this step we will securely connect your machine to the remote repository usin

Let's start by generating a public-private key pair based on the (currently) most secure Ed25519 standard.

```
$ ssh-keygen -o -a 100 -t ed25519
```shell
ssh-keygen -o -a 100 -t ed25519
```

This will ask you for a location to save the key and a password. If you want to use automatic backups on your Mac, you should enter an empty passphrase or add the passphrase to the Keychain.

The above command will create a public- and a private key. The private key should remain a secret on your local machine. The public key is passed to your remote/cloud backup repository. You can display the public key using this command:

```
$ cat ~/.ssh/id_ed25519.pub
```shell
cat ~/.ssh/id_ed25519.pub
```

This will display your public key as single line. Now just copy it and provide it to your repository provider. When using [BorgBase.com](https://www.borgbase.com), you can add public keys under **Account > SSH Keys**.
Expand Down Expand Up @@ -152,18 +176,18 @@ To initialize the remote repo, use the commands below. Replace the `ssh://` URL
{% tab init Borg 1.x %}

```shell
$ export BORG_REPO=ssh://...
$ borg init -e repokey-blake2
export BORG_REPO=ssh://...
borg init -e repokey-blake2
```

{% endtab %}

{% tab init Borg 2.x %}

```shell
$ export BORG_REPO=ssh://...
$ borg benchmark cpu # optional: find fastest encryption algorithm
$ borg rcreate --encryption=repokey-blake2-aes-ocb
export BORG_REPO=ssh://...
borg benchmark cpu # optional: find fastest encryption algorithm
borg rcreate --encryption=repokey-blake2-aes-ocb
```

**Note**: Encryption speed can vary depending on CPU and if a virtual machine is used. So it's best to benchmark it and choose the fastest combination. More [here](https://borgbackup.readthedocs.io/en/2.0.0b4/usage/rcreate.html#choosing-an-encryption-mode).
Expand All @@ -180,15 +204,15 @@ After the command is done, you have your remote backup repository set up and rea
{% tab create Borg 1.x %}

```shell
$ borg create ::my-archive-1 ~/Documents
borg create ::my-archive-1 ~/Documents
```

{% endtab %}

{% tab create Borg 2.x %}

```shell
$ borg create my-archive-1 ~/Documents
borg create my-archive-1 ~/Documents
```

{% endtab %}
Expand All @@ -203,14 +227,14 @@ It would be quite cumbersome to manually run Borg every few hours and keep comin

Let's start by creating a default Borgmatic folder:

```
$ mkdir -p ~/.config/borgmatic
```shell
mkdir -p ~/.config/borgmatic
```

Then open the main config file with your favorite text editor. I'll use `nano`, but `vi`, Sublime or Atom will work too.

```
$ nano ~/.config/borgmatic/config.yaml
```shell
nano ~/.config/borgmatic/config.yaml
```

If you keep your repos with BorgBase you can copy a pre-made Borgmatic confirguration from the **Setup** page. Or start with this version and fill in the blanks:
Expand Down Expand Up @@ -238,7 +262,6 @@ encryption_passphrase: CHANGE ME!!
archive_name_format: "{hostname}-{now}"

# Number of times to retry a failing backup
# Needs recent Borgmatic version
retries: 5
retry_wait: 5

Expand Down Expand Up @@ -282,7 +305,7 @@ storage:
archive_name_format: "{hostname}-{now}"

# Number of times to retry a failing backup
# Needs recent Borgmatic version
# Requires Borgmatic version 1.7.x
retries: 5
retry_wait: 5

Expand Down Expand Up @@ -327,15 +350,15 @@ Once you are happy with the options, save the file and initialize your repositor
{% tab init Borg 1.x %}

```shell
$ borgmatic init --encryption repokey-blake2
borgmatic init --encryption repokey-blake2
```

{% endtab %}

{% tab init Borg 2.x %}

```shell
$ borgmatic init --encryption repokey-blake2-chacha20-poly1305
borgmatic init --encryption repokey-blake2-chacha20-poly1305
```

{% endtab %}
Expand All @@ -344,8 +367,8 @@ $ borgmatic init --encryption repokey-blake2-chacha20-poly1305

Simply running `borgmatic` without any options will create a new backup. We will add the `--verbosity` option to see what's going on during the first backup.

```
$ borgmatic --verbosity 2
```shell
borgmatic --verbosity 2
```

Depending on your backup folder size, this may take a while. After Borgmatic is done, you can view the available snapshots and some statistics.
Expand All @@ -355,15 +378,15 @@ Depending on your backup folder size, this may take a while. After Borgmatic is
{% tab borgmatic_rinfo Borgmatic >= 1.7 %}

```shell
$ borgmatic rinfo
borgmatic rinfo
```

{% endtab %}

{% tab borgmatic_rinfo Borgmatic <= 1.6 %}

```shell
$ borgmatic --list --info
borgmatic --list --info
```

{% endtab %}
Expand Down Expand Up @@ -395,8 +418,8 @@ You can see that there are 2 snapshots in this repository with a total of 134 GB

With Borgmatic working, we can set up Cron to do regular backups in the future. Cron is an old Unix tool to run _repeated_ tasks. It's usually included in all Unix-like systems.

```
$ crontab -e
```shell
crontab -e
```

This will open an editor to add new cron jobs. There are many ways to schedule cron-jobs. For hourly backups enter this:
Expand Down