Install uwsgi in venv and point service file to venv binary

[tyler-8]

Currently, the uwsgi installation isn't done into a virtual environment, but the environment of ansible_python_interpreter, which may not be using the same python binary that the NetBox venv is using.

This task installs uwsgi and happens after the NetBox-specific venv has been created.

ansible-role-netbox/tasks/main.yml

Lines 68 to 79 in dbead68

	- name: Install uWSGI via pip
	pip:
	name: uwsgi
	state: "{{ 'latest' if netbox_keep_uwsgi_updated else 'present' }}"
	umask: "0022"
	environment:
	PATH: "/usr/local/bin:{{ _path }}"
	notify:
	- reload netbox.service
	retries: 2
	register: _netbox_uwsgi_install
	until: _netbox_uwsgi_install is succeeded

I propose adding virtualenv: "{{ netbox_virtualenv_path }}" to that task to ensure it's installed using the same venv as the rest of the app.

Additionally, the systemd service template relies on /usr/bin/env, which doesn't have any awareness of the venv as currently written.

ansible-role-netbox/templates/netbox.service.j2

Lines 8 to 9 in dbead68

	[Service]
	ExecStart=/usr/bin/env uwsgi --ini {{ netbox_shared_path }}/uwsgi.ini

If the above task is modified to install uwsgi into the venv, then the template could be adjusted to use

ExecStart={{ netbox_virtualenv_path }}/bin/uwsgi --ini {{ netbox_shared_path }}/uwsgi.ini`

All in all this would give users more flexibility in using Python versions from different sources (altinstall, pyenv, package managers, etc).

Something would probably need to be modified with this task as well but we'd likely need to decouple it from netbox_python_binary (and possibly other tasks).

ansible-role-netbox/tasks/install_packages_yum.yml

Lines 28 to 35 in dbead68

	- name: Install psycopg2/selinux via pip on Red Hat-based distros
	pip:
	name:
	- psycopg2-binary
	- selinux
	state: present
	vars:
	ansible_python_interpreter: "{{ netbox_python_binary }}"

[lae]

So I believe there was a reason why we used uwsgi from package repositories instead of within the virtualenv, but I can't quite remember why. I think it might have had something to do with compilation, possibly? Maybe missing features or maybe just the actual amount of time it takes to install uwsgi (faster to keep a single version externally than build it every time a new netbox release comes out).

which doesn't have any awareness of the venv as currently written.

This isn't exactly true. I'm pretty sure uwsgi itself is aware of the virtualenv and drops into it, per the uwsgi.ini.j2 template. (also /usr/bin/env is not intended to be in anyway related to virtualenvs)

Where exactly are you trying to gain more flexibility? Inside the virtualenv?

[tyler-8]

reason why we used uwsgi from package repositories instead of within the virtualenv

It's still being installed via pip, just not in a venv.

faster to keep a single version externally than build it every time a new netbox release comes out

This seems like a negligible difference compared to the overall install process, uwsgi doesn't take a notably longer time to build than most other non-wheel packages I don't think.

This isn't exactly true. I'm pretty sure uwsgi itself is aware of the virtualenv and drops into it ... Where exactly are you trying to gain more flexibility? Inside the virtualenv?

What kicked off this exploration was I was testing with new Python versions installed via pyenv. I install pyenv and a new Python version before calling this role, then pointed netbox_python_binary to the new binary and was getting errors from uwsgi not being able to find Django installed (even though it was installed in the new venv. I then realized that uwsgi was still pointing to a version installed in /usr/local/bin/ and built with the older version of Python.

[tyler-8]

I should also mention that I was getting errors from this task

ansible-role-netbox/tasks/main.yml

Line 55 in dbead68

- name: Ensure Postgres database exists (via TCP)

because psycopg2 wasn't being re-installed under the new version of Python. Not entirely sure why that was happening even on subsequent runs because it definitely was installed yet but the task completed without doing anything.

[lae]

Just to not let this stagnate, I still haven't remembered the exact reasoning behind using uwsgi outside of the virtualenv (if not for one of the guesses I provided before). I just remember that I did indeed originally try to stick to running everything inside the virtualenv, so I must have ran into some sort of obstacle with uwsgi that made me decide to use it outside.

If you can manage to get a functional changeset to the role to run uwsgi inside the venv, I guess I can consider adopting it, but then I guess consideration also needs to be made for existing NetBox installations (do we uninstall uwsgi? etc).

[tyler-8]

I'll say this, I've done a local fork of this playbook that uses a virtualenv-installed version of uwsgi this past week without any issues. It makes the most sense to me, however I could write an update that leaves it up to the user I suppose.

You'd have two options then:

1. Use the default behavior
2. Use a uwsgi binary installed into the venv

[ironiq]

Just a thougth for the uwsgi in venv: if you plan to support RHEL7/CentOS7 and Netbox3, the uwsgi from the distro will not work. I have a running Netbox 3.4.1 on CentOS7 and had to install uwsgi in venv, because otherwise it did not work correctly (python 3.6 vs. python 3.8 in venv).

[lae]

btw if someone can test the venv uwsgi setup on all technically supported distros and confirm that it works fine, I think I wouldn't mind making it the default for a major release and including update notes to suggest manually removing the non-venv install.

[ironiq]

Actually i tested it on CentOS7 with rh-python38 and Netbox 3.4.1. This setup works, i did the same thing what @tyler-8 mentioned in his opening comment. Unfortunately i cannot test other setup.