Vendor keyring & keyring-subprocess

[Darsstar]

I went ahead and made a proper PR for what I first talked about in #8719 (comment)

Q: But keyring has C dependencies, how do you expect this to work?
A: Some of keyring's first party backend's have C dependencies, we won't be using those and instead use my keyring-subprocess library's pure Python backend. Therefore we don't actually need to vendor them. If you would like I can alter the vendoring configuration to make sure those backend's are not vendored.

Fixes #8719
Can be an alternative, or in addition to: #11215. Which fixes the same issue.

Would go very well together with #11029

I still haven't looked at the Pip zipapp implementation and what that means for keyring users. I only know keyring and truststore being exceptions to the vendoring rule came up.

Related: pypa/pipx#829 would no longer be something would want

[judahrand]

@Darsstar I'm not sure this is needed now that #11589 is merged? They achieve similar goals if I understand it correctly?

[Darsstar]

Woow, you got that merged quick!

Time to take a look. My first question to answer by studing the code would be "how does this interact with activating a virtual environment which has keyring as a dependency?".
My keyring-subprocess first tried to look for a global keyring, but Poetry has keyring as a dependency and therefor a keyring executable which was found before the global one.

I suspect I will end up closing this PR.

[judahrand]

Woow, you got that merged quick!

Time to take a look. My first question to answer by studing the code would be "how does this interact with activating a virtual environment which has keyring as a dependency?". My keyring-subprocess first tried to look for a global keyring, but Poetry has keyring as a dependency and therefor a keyring executable which was found before the global one.

I suspect I will end up closing this PR.

Yeah, the code that got merged defaults to using a locally installed keyring via import. This was done to avoid breaking things for people. The fallback of looking on PATH just uses shutil.which so it'll find the first one anyway.

Additionally, with the addition of the --python option to pip one could try having a 'global' pip installed as a zipapp or with pipx and a pipx install of keyring. The global pip and keyring could then talk to each other even when using pip install --python .venv for example.

Super cool possibilities.

[uranusjr]

There’s an issue on not automatically importing keyring #8719. Perhaps we should make the option multi-choice (no keyring, force subprocess, and force import)?

[judahrand]

There’s an issue on not automatically importing keyring #8719. Perhaps we should make the option multi-choice (no keyring, force subprocess, and force import)?

The issue is that if we're just looking on PATH with which then import==subprocess when import is possible, no? (In terms of the results) Unless, we make that logic more sophisticated.

I like the idea of making it up to the user which option is used though! A lot.

[uranusjr]

Not always, say you can do

$ virtualenv foo
$ foo/bin/pip install keyring
$ pipx install keyring
$ foo/bin/pip install whatever  # Likely using the pipx one, not the virtualenv.

But I get your point, the subprocess implementation preferring the same keyring in the same environment is still a problem in many situations.

---

Also for the Poetry issue specifically, keyring in the env Poetry is installed into should not be in PATH (or you probably should seriously visit how you set up your dev env), so at least for this scenario it’s possible to prefer the out-of-env keyring installation.

[Darsstar]

Perhaps we should make the option multi-choice (no keyring, force subprocess, and force import)?

You/it got my vote! For whatever that is worth... (it referring to your idea, not you. Regardless of who might end up implementing it.)

---

The issue is that if we're just looking on PATH with which then import==subprocess when import is possible, no? (In terms of the results) Unless, we make that logic more sophisticated.

I guess shutil.which's path parameter can be used after using the sysconfig module to remove the scripts path from PATH would be a pretty good start?

---

From what I can tell by just studying the code that should work after a tiny change in our setup. (include a username in the URL.)
I'll try to test that next thursday at work when I'm back in the office after some days off.

[judahrand]

Yup it also requires us to set the username. We're using the Google Artifact Registry keyring backend.

[Darsstar]

I'll try to test that next thursday at work when I'm back in the office after some days off.

I forgot, then I remembered but didn't have time, then I forgot again, but I finally remembered again!

It doesn't work on Windows due to hardcoding "\n" as the line seperator instead of os.linesep. I just fixed that as part of my --force-keyring PR: #11029.

Other than that it works great! My fear of it tripping up Poetry turned out to be a nothing burger as you expected, @uranusjr .

Time to create a very quick bug report...