Could really use some guidance upgrading Netbox from v2.4.4 to current

[quantumzedno]

Hi All,

I could really use some assistance, I started a new position and the company i'm working for now is already leveraging netbox for IPAM/DCIM, however their instance hasnt been updated in years... it's currently sitting on version 2.4.4 and i'm having a heck of a time trying to update it.

I cloned the VM so i can test trying to upgrade it without corrupting anything, but after installing updated versions of all the dependencies listed on the upgrade doc, i ran the upgrade script and got the following error dump.

sudo PYTHON=/usr/bin/python3.8 ./upgrade.sh
Running NetBox upgrade as root, press any key to continue or ^C to cancel
Cleaning up stale Python bytecode (find . -name "*.pyc" -delete)...
Removing old Python packages (pip3 uninstall -r old_requirements.txt -y)...
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see pypa/pip#5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
WARNING: Skipping django-rest-swagger as it is not installed.
WARNING: Skipping psycopg2 as it is not installed.
WARNING: Skipping pycrypto as it is not installed.
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
Updating required Python packages (pip3 install -r requirements.txt --upgrade)...
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see pypa/pip#5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting Django<2.1,>=1.11
Downloading Django-2.0.13-py3-none-any.whl (7.1 MB)
|████████████████████████████████| 7.1 MB 9.1 MB/s
Collecting django-cors-headers==2.4.0
Using cached django_cors_headers-2.4.0-py2.py3-none-any.whl (12 kB)
Collecting django-debug-toolbar==1.9.1
Using cached django_debug_toolbar-1.9.1-py2.py3-none-any.whl (206 kB)
Collecting django-filter==1.1.0
Using cached django_filter-1.1.0-py2.py3-none-any.whl (45 kB)
Collecting django-mptt==0.9.1
Using cached django_mptt-0.9.1-py2.py3-none-any.whl (105 kB)
Collecting django-tables2==1.21.2
Using cached django-tables2-1.21.2.tar.gz (2.7 MB)
Collecting django-taggit==0.22.2
Using cached django_taggit-0.22.2-py2.py3-none-any.whl (45 kB)
Collecting django-taggit-serializer==0.1.7
Using cached django-taggit-serializer-0.1.7.tar.gz (5.6 kB)
Collecting django-timezone-field==2.1
Using cached django_timezone_field-2.1-py2.py3-none-any.whl (8.4 kB)
Collecting djangorestframework==3.8.1
Using cached djangorestframework-3.8.1-py2.py3-none-any.whl (923 kB)
Collecting drf-yasg[validation]==1.9.2
Using cached drf_yasg-1.9.2-py2.py3-none-any.whl (841 kB)
Collecting graphviz==0.8.4
Using cached graphviz-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting Markdown==2.6.11
Using cached Markdown-2.6.11-py2.py3-none-any.whl (78 kB)
Collecting natsort==5.3.3
Using cached natsort-5.3.3-py2.py3-none-any.whl (34 kB)
Collecting ncclient==0.6.0
Using cached ncclient-0.6.0.tar.gz (69 kB)
Collecting netaddr==0.7.19
Using cached netaddr-0.7.19-py2.py3-none-any.whl (1.6 MB)
Collecting paramiko==2.4.1
Using cached paramiko-2.4.1-py2.py3-none-any.whl (194 kB)
Collecting Pillow==5.2.0
Downloading Pillow-5.2.0.tar.gz (14.5 MB)
|████████████████████████████████| 14.5 MB 71.5 MB/s
Collecting psycopg2-binary==2.7.5
Downloading psycopg2-binary-2.7.5.tar.gz (427 kB)
|████████████████████████████████| 427 kB 41.2 MB/s
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3guid84m/psycopg2-binary_e4d8eed0bc3b44449db4dcedea8fa08d/setup.py'"'"'; file='"'"'/tmp/pip-install-3guid84m/psycopg2-binary_e4d8eed0bc3b44449db4dcedea8fa08d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-supy5x2o
cwd: /tmp/pip-install-3guid84m/psycopg2-binary_e4d8eed0bc3b44449db4dcedea8fa08d/
Complete output (23 lines):
running egg_info
creating /tmp/pip-pip-egg-info-supy5x2o/psycopg2_binary.egg-info
writing /tmp/pip-pip-egg-info-supy5x2o/psycopg2_binary.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-supy5x2o/psycopg2_binary.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-pip-egg-info-supy5x2o/psycopg2_binary.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-supy5x2o/psycopg2_binary.egg-info/SOURCES.txt'

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).

----------------------------------------

WARNING: Discarding https://files.pythonhosted.org/packages/17/dc/3f6bfae668594013cb9ef0cba4f4863451220944650868ad67332ca9069c/psycopg2-binary-2.7.5.tar.gz#sha256=c2ac7aa1a144d4e0e613ac7286dae85671e99fe7a1353954d4905629c36b811c (from https://pypi.org/simple/psycopg2-binary/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement psycopg2-binary==2.7.5 (from versions: 2.7.4, 2.7.5, 2.7.6, 2.7.6.1, 2.7.7, 2.8, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6, 2.9, 2.9.1, 2.9.2, 2.9.3)
ERROR: No matching distribution found for psycopg2-binary==2.7.5
WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
File "netbox/manage.py", line 8, in
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'
Collecting static files (python3 netbox/manage.py collectstatic --no-input)...
Traceback (most recent call last):
File "netbox/manage.py", line 8, in
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

i'm assuming i need to do some kind of interm upgrade before moving to the latest version but i don't know where to start.

I have spun up an install of the latest release on a separate VM, so ideally id love to just do any Database upgrades and import the old database into the new server if possible, but again, i don't know where to start with something like that either.

Thank you so much in advance

[jeremystretch]

For something this far out of date, your best course of action is probably going to be:

1. Make a copy of the database and media archive (you can skip the media archive if your installation doesn't have any image attachments)
2. Create an entirely new installation of NetBox v2.11.12 and make sure that it works
3. Drop the database from the v2.11.12 install and load your copy from v2.4
4. Run upgrade.sh to apply all database migrations
5. Restart the NetBox services and verify that the data was loaded successfully
6. Upgrade to the latest v3.1 release

[dayonoiki]

Hi,
just found I was in the wrong directory to run python3 manage.py sqlmigrate extras 0054 . does the output show any issues ? with regards the database ownership problem, i'll refer this to our database team to have a look.
sqlmigrate0054.txt

Regards,

[candlerb]

Sorry, the output of sqlmigrate isn't very helpful to me.

Can you just go into a postgres shell (sudo -u postgres psql netbox) and run \d

For me, all the owners are the same:

                              List of relations
 Schema |                      Name                      |   Type   |  Owner
--------+------------------------------------------------+----------+---------
 public | auth_group                                     | table    | netbox
 public | auth_group_id_seq                              | sequence | netbox
 public | auth_group_permissions                         | table    | netbox
 public | auth_group_permissions_id_seq                  | sequence | netbox
 public | auth_permission                                | table    | netbox
 public | auth_permission_id_seq                         | sequence | netbox
...etc

If it's not like that, then I think you need to fix it. You could do this either by editing the 1.9.x psql dump file (which is just plain text) prior to restoring it, or by suitable "ALTER TABLE/SEQUENCE ..." commands on the restored database.

[dayonoiki]

already sent the output dt_ds-log.txt

but here is the output of \d
sql_output.txt

[candlerb]

already sent the output dt_ds-log.txt

Sorry, missed that first time.

It appears you have exactly the problem that was in the postings I linked to before: some tables and/or sequences owned by "postgres" and some by "netbox". This is probably due to how the system was first installed.

You need to change everything which is currently owned by "postgres" to "netbox" (also check that in configuration.py, you are connecting to the database as the "netbox" user)

[dayonoiki]

Hi,
I was able to change all the tables owned by "postgres" to "netbox" successfully. then ran the the upgrade.sh and restarted services. Netbox now successfully running 2.11.12. And then finally upgraded to 3.2.5 successfully.

Thanks for your help