How to backup all data on netbox?

[ssangho8]

Hi,

I'm operating netbox through running docker-compose.

I'm wondering if there's a way to backup all data (e.g. IP addresses, Devices) on netbox.

I want to backup data every day or every week.

Here's the information when I type "docker ps -a" command on my netbox server.

docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b3379a749353 netboxcommunity/netbox:v3.2-1.6.1 "/sbin/tini -- /opt/…" 2 months ago Up 2 months 0.0.0.0:8000->8080/tcp, :::8000->8080/tcp netboxdocker_netbox_1
bcd810b00e02 netboxcommunity/netbox:v3.2-1.6.1 "/sbin/tini -- /opt/…" 2 months ago Up 2 months netboxdocker_netbox-housekeeping_1
fc17e44536a7 netboxcommunity/netbox:v3.2-1.6.1 "/sbin/tini -- /opt/…" 2 months ago Up 2 months netboxdocker_netbox-worker_1
c3eb6d82b84e postgres:14-alpine "docker-entrypoint.s…" 2 months ago Up 2 months 5432/tcp netboxdocker_postgres_1
b16e3fb413e5 redis:6-alpine "docker-entrypoint.s…" 2 months ago Up 2 months 6379/tcp netboxdocker_redis-cache_1
feb69922effa redis:6-alpine "docker-entrypoint.s…" 2 months ago Up 2 months 6379/tcp netboxdocker_redis_1

Please help.

[PieterL75]

Use the postgres pg_dump to backup the database :
https://docs.netbox.dev/en/stable/administration/replicating-netbox/

[Nino-Rey]

I'm also thinking the same way Netbox Devices has Export function which return a CSV file and I'd like to run it weekly to backup all our Devices and put it on S3 bucket. Is there any reference that I could look at in this repo?

[candlerb]

That won't work, because you cannot restore Netbox from CSV files. Firstly, in general CSV files don't include unique ID references for one object to refer to another. And secondly, there are circular references which have to be restored simultaneously (in particular: a device's primary_ip points to an IP address, but the IP address points to an interface which points to the device).

Therefore, you'll will have to use pg_dump to make a usable Netbox backup. Copying it to an S3 bucket can be done in a script using awscli or s3cmd.

[ssangho8]

I installed psql v10 and I get error message below when I type "psql" on CLI.

psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Should I manage psql within the docker container? or on local server?

[candlerb]

Easiest is to enter the postgres container and run psql inside the container (i.e. use docker exec ...)

In principle you can install the postgres client on your server or a remote workstation, and connect to the postgres database over the network. However this may require some setting up, for example adding rules to hba.conf to allow connections from your workstation's IP address.

[ssangho8]

I currently can't find where hba.conf file located at.

I just got in to the 'netbox' database on postgresql container.

[candlerb]

You'll need to switch your userid to the user that postgres runs under, which if the container is based on Ubuntu will probably be user psql:

sudo -u postgres psql netbox

For things like hba.conf, please refer to postgres documentation or a postgres user group.

[ssangho8]

In postgres.env file located at /env,
POSTGRES_DB=netbox
POSTGRES_PASSWORD=~~
POSTGRES_USER=netbox

I got into the postgres container after running below.
#docker exec -it --user root netboxdocker_postgres_1 /bin/bash

Also I could get into the netbox database after running below.
bash-5.1# psql netbox -U netbox

Do you mean I have to use pg_dump so that I could export whole data on netbox database?

[candlerb]

Yes. You'll be able to combine it into a single command, probably some variation of the following (untested as I don't run netbox in docker):

docker exec --user root netboxdocker_postgres_1 /usr/local/bin/pg_dump netbox -U netbox

[mingueladavid]

This is my first time using netbox and postgres. After a lot of test try to find the correct data base and restore procedure I created the following script to generate the data base backup. It only work due to permission problem in /var/lib/postgresql/ and the restore should be do using postgres user. I used the as part of the backup the option --clean, because if you restore over the actual database it give you errors "tables already exist" sorry for my English because my firts languaje is spanish. The owner of this script should be postgres.

#!/bin/sh

**********************************************************************************************************

05-jul-2023 DRM

Script para correr backup de la base de datos de Netbox. Este script debe estar localizado

y debe correrse en el directorio /var/lib/postgresql. El script debe tener como ownership a postgres

-rwxr--r-- 1 postgres postgres db_backup.sh

La restauración de la base de datos debe hacerlo bajo el usuario de postgres

us - postgres

postgres@netboxU20:~$ psql -e -d netbox < netbox-mysql-FECHA_DEL_BACKUP.sql

El backup de la base de datos se guardara los ultimos 5 dias.

**********************************************************************************************************

N="netbox-mysql"
D=date +"%Y%m%d"
Z=date +"%Y%m%d" -d "-5 days"

BKP="/home/oberon"
ARC="/var/backup"

/usr/bin/su -c 'pg_dump -U postgres -F p --clean netbox > /var/lib/postgresql/db_clean.sql' postgres
/usr/bin/mv /var/lib/postgresql/db_clean.sql /var/lib/postgresql/$N-$D.sql
/usr/bin/rm /var/lib/postgresql/$N-$Z.sql

[candlerb]

N="netbox-mysql"

Your colleagues may find that confusing, since this is a dump from postgres not mysql :-)

This is my rough-and-ready backup script:

#!/bin/sh -e
mkdir -p /var/backups/netbox
chmod 750 /var/backups/netbox
su -c 'pg_dump --exclude-table-data extras_cachedvalue --exclude-table-data extras_objectchange netbox' postgres |
  gzip -9 >/var/backups/netbox/netbox.$(date +%Y%m%d%H%M).gz

called from a cronjob:

#!/bin/sh -eu
/usr/local/bin/netbox-backup
find /var/backups/netbox -mtime +90 -type f -delete

[mingueladavid]

Thanks, yes you right N="netbox-mysql" but is just a variable......

[PieterL75]

I have this :

#!/bin/bash

cd /opt/backup/netbox

DATETIME=$(date +%y%m%d_%H%M)
MAXDAYS=45

#database
pg_dump -Fc --username netbox --host localhost netbox > $(hostname)_$DATETIME.sql
find . -name "*.sql" -maxdepth 1 -mtime +$MAXDAYS -delete

#files
tar -czf $(hostname)_$DATETIME.files.tgz --exclude=".git" --exclude="__pycache__" /opt/netbox/netbox/local /opt/netbox/netbox/media /opt/netbox/netbox/reports /opt/netbox/netbox/scripts
find . -name "*.files.tgz" -maxdepth 1 -mtime +$MAXDAYS -delete

#configuration
crontab -l >/opt/netbox/netbox/netbox/netbox.crontab
tar -czf $(hostname)_$DATETIME.configs.tgz --exclude=".git" --exclude="__pycache__" /opt/netbox/local_requirements.txt /opt/netbox/netbox/netbox/configuration.py /etc/nginx/sites-available/netbox /opt/netbox/netbox/netbox/netbox.crontab
find . -name "*.configs.tgz" -maxdepth 1 -mtime +$MAXDAYS -delete

sftp mybackupserver:/opt/backup/netbox <<< $"put $(hostname)_$DATETIME.sql"
sftp mybackupserver:/opt/backup/netbox <<< $"put $(hostname)_$DATETIME.files.tgz"
sftp mybackupserver:/opt/backup/netbox <<< $"put $(hostname)_$DATETIME.configs.tgz"

/usr/bin/find /opt/backup/netbox -maxdepth 1 -mtime +5 -name "netbox*" -print -delete

and in the /home/netbox folder I have
/home/netbox/.pgpass
with

localhost:*:netbox:netbox:mysecretpassword

all ran from the 'netbox' crontab (3 times a day)

10 02 * * * /home/netbox/nbbackup.sh
10 12 * * * /home/netbox/nbbackup.sh
10 18 * * * /home/netbox/nbbackup.sh

[Lord-ZuzurNC]

Hello,

I'm a gravedigger but I'm wondering why you delete 45+ days old sql files in line 10 then all 5+ days old file starting by netbox (so sql as well if hostname start with that word, which may happens for a lot of us) at line 25 (last).

(less important but why using "find" then "/usr/bin/find" at the end?)

[Lord-ZuzurNC]

I rewrote the script to suit me (using @PieterL75 and @candlerb comments). If someone does have similar needs, here you go bellow.
I added a "BAKDIR" variable, the backup of the configuration file "gunicorn.py".
I deleted the sftp commands (as I will manage using another method) and the last deleting 5+ days old files.

#!/bin/bash

BAKDIR=/opt/backup
DATETIME=$(date +%y%m%d_%H%M)
MAXDAYS=31

cd $BAKDIR

# PostgreSQL Database Backup
pg_dump -Fc --exclude-table-data extras_cachedvalue --exclude-table-data extras_objectchange --username netbox --host localhost netbox > ./$(hostname)_$DATETIME.sql
find $BAKDIR -name "*.sql" -maxdepth 1 -mtime +$MAXDAYS -delete

# Data Backup
tar -czf $(hostname)_$DATETIME.files.tgz --exclude=".git" --exclude="__pycache__" /opt/netbox/netbox/media /opt/netbox/netbox/reports /opt/netbox/netbox/scripts
find $BAKDIR -name "*.files.tgz" -maxdepth 1 -mtime +$MAXDAYS -delete

# Configuration Backup
crontab -l > /opt/netbox/netbox/netbox/netbox.crontab
tar -czf $(hostname)_$DATETIME.configs.tgz --exclude=".git" --exclude="__pycache__" /opt/netbox/local_requirements.txt /opt/netbox/netbox/netbox/configuration.py /opt/netbox/gunicorn.py /etc/nginx/sites-available/netbox /opt/netbox/netbox/netbox/netbox.crontab
find $BAKDIR -name "*.configs.tgz" -maxdepth 1 -mtime +$MAXDAYS -delete

I also add, if anyone come here and to the same, that you need to change the permission to 0600 for the .pgpass
chmod 0600 ~/.pgpass