Add SSH Key field when making git datasources

[Mailstorm-ctrl]

NetBox version

3.6.3

Feature type

New functionality

Proposed functionality

This is in regards to #14941

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Use case

GitHub enterprise tenants with Managed Users require the use of an external IdP to authenticate to GitHub. Username and password authentication will not work to the API or to login. Allowing the use of SSH keys allows people using an EMU instance (or just preference) to use git over SSH.

Database changes

Most likely need a new field that will allow a user to supply a SSH key path. I can see a use case for requiring multiple different keys

External dependencies

No response

[ross-cello]

Would love to see this implemented

[cpmills1975]

Please do implement this. My local git repo requires SSH keys to authenticate. Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB. The Credential model would be one of either username/password combination or SSH key and optional password. The datasource would have a foreign key relationship to the credential making the credential re-usable across several datasources if desired. If Dulwich needs the key in a file (I'll admit to only cursory glances at the docs), then the key would need to be written to a temporary file when any remote file sync activity takes place.

[jeremystretch]

This is open for volunteers.

[markkuleinio]

Does someone know how to even provide the keyfile to porcelain.clone()? I mean, I've tried to provide key_filename="path_to_keyfile" in the call, but it doesn't seem to do anything.

SyncError('Fetching remote data failed (HangupException): [email protected]: Permission denied (publickey).\r')

I don't seem to find any example anywhere.

(aaand update: key_filename works just fine, provided that I also restart netbox-rq, not just netbox... to be continued)

[markkuleinio]

Workaround to use SSH key:

Add this in /etc/systemd/system/netbox-rq.service, in the [Service] section:

Environment="GIT_SSH_COMMAND=ssh -i /path/to/ssh_private_key"

and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart netbox-rq

Limitation: You can only select one keyfile --> If you have more than one data source, you need to authorize the same private key to access all those data sources.

[Mailstorm-ctrl]

Well this is exciting. I got this to work with no code changes...though this is dirty and I'm almost sure it goes against some kind of practice. I have successfully gotten this to work with multiple repos with multiple keys. Here's how I did it:

We are going to be enabling login on the local netbox user on the host server! If you don't want to do this, then sorry you can't do this

1. Enable local login on the netbox user (or whatever user you used for the system account when initially installing netbox.
   sudo usermod -s /bin/bash netbox
2. Switch to the user to generate your SSH keys
   sudo su - netbox
3. Generate your ssh keys and name them whatever you want.
4. Run eval "$(ssh-agent)" (See here for why)
5. Run the following for however many keys you made:
   ssh-add <path to sshkey>
6. In GitHub, create your deploy keys
7. Back on the netbox host, create or modify the ~/.ssh/config file while still acting as the netbox user. Modify the following to your liking:

Host repo1.github.com
        HostName github.com
        PreferredAuthentications publickey
        IdentityFile ~/.ssh/key1
Host repo2.github.com
        HostName github.com
        PreferredAuthentications publickey
        IdentityFile ~/.ssh/key2

8. In netbox, go ahead and add a git datasource. However, for the url you will do something similar to this: [email protected]:orgname/repo1. Leave the username and password blank. I can't verify if the branch works or not.
9. Enjoy your SSH connected repos

[llamafilm]

You also have to include the server in /home/netbox/.ssh/known_hosts or else the sync will fail.

[jeremystretch]

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file.

Given the complexity of maintaining SSH keys, IMHO it would be worth splitting out the authentication methods from the data sources at this point and creating a 'Credential' model to store the keys in the DB.

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base.

[Mailstorm-ctrl]

Add a field that allows a user to supply a path to an SSH key file to use to authenticate to GitHub.

Reading through this again, I'm not sure this is a suitable solution to the root problem. Asking for a path to a file on disk assumes 1) the user has knowledge of the underlying filesystem and 2) the user has access to install a file.

To me setting something like this up doesn't need to be "user friendly". It's like making scripts. Making scripts assumes you have good knowledge of netbox models and python in general.

Unfortunately I don't have a better proposal at the moment, but I also don't want to burn time and effort delivering an implementation that likely won't suffice for a large portion of the user base.

The work-around I provided seems to be working as intended. I can write better documentation on how to achieve this if you think it's beneficial overall. From what I read, this is probably the most supported option when you need to interact with multiple keys in git.

[jeremystretch]

It's like making scripts.

We discovered we needed to add the ability to upload/sync scripts directly into NetBox specifically because the local filesystem was inaccessible to many users, which precluded them from using scripts.

[Mailstorm-ctrl]

Not exactly sure what changed, but when upgrading from 4.0.3 to 4.1.0, this method no longer works. You get "Permission Denied" as the error. Logging in as netbox on the underlying OS and then using a sample python file that runs the same instructions as netbox does completes without errors.

False alarm. The netbox user didn't have permissions to the scripts directory

[psharp-vs]

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

[Mailstorm-ctrl]

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar.

[psharp-vs]

This gets closer to a solution, but IMO we should stop short of actually storing private keys in the database. That's just not something we can do securely.

A number of popular git providers (GitHub, GitLab, Bitbucket) allow for the creation of PAT tokens which are repo specific and reduce the risk of storing an entire private key, might be a good middle ground.

These are personal access tokens which means they are tied to a user...not the organization itself. Not ideal in enterprise environments in my opinion. Would be fine for sandbox environments or similar.

I had imagined them tied to a service account. That's at least our orgs preferred method to have applications access git resources. In fact, after exploring a bit more we actually disable PAT tokens for individuals but enable them for service accounts for precisely this use case.

[cpmills1975]

I'd be happy to accept the risk of holding SSH keys in the database. I'm stuck in that my corporate git repo requires keys for authentication. These keys can (should be) be entirely separate to any and all other systems and the key can be granted read only access to just one repo if required. The risks in uploading keys that have full access to multiple systems should be somewhat obvious, but documentation can perhaps help with that.

Failing that, I'd accept SSH keys on the file system and a pointer to them in the DB, but appreciate that's because I can put them there and this won't work for everyone in every environment.

[psharp-vs]

I did a bit of exploring on this to see what kind of code changes might be needed.

The user/password/branch data is stored as a JSON blob in the database. Adding an ssh key would be trivial and not require any database changes. Is it secure? Probably not, but we're already storing plain-text passwords so is it already any worse? Adding a little bit of logic to the fetch method to support an ssh key if it exists should be pretty easy as well.

I had mentioned PAT tokens, which is something many git providers can generate. I have only tested on GitHub, but this already works, the PAT token acts like a password.

[mskalecki]

If using GitHub, the new (still in preview) Fine-Grained Personal Access Tokens are an exceptionally good alternative to storing multiple ssh keys. You can lock a PAT down to have read-only access to only the contents of an individual repo (or multiple individual repos). And, as @psharp-vs mentioned, is is already supported by Netbox. Just use the repo's https url, username = oauth2, and password = <personal access token>

Does it make sense to add a note to the documentation that Personal Access Tokens can be used in place of username / password in this way? Seems really likely that a lot of users won't be Git / GitHub experts and are just used to always using SSH keys (like I was).