-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Why does connection to localhost
fail on local ?
#7429
Comments
localhost
on local ?localhost
fail on local ?
@manuraj17 I was also able to repro on my local with Will discuss with team and update on what is the best course of action here. Thanks once again for catching this. |
@purnesh42H Thanks for checking on this. Will wait for the update 👍🏽 |
Client logs with 1 second timeout when using
|
@purnesh42H |
@manuraj17 the problem seems to be when using |
this call is blocked for longer when using Line 212 in 2bcbcab
|
@manuraj17 I am not able to repro this anymore and |
Are you still able to reproduce this? Can you run with full debug logging enabled and include the logs? |
@dfawley Will check and update; Though any idea on what is happening in this scenario? |
No idea. If your machine can't resolve |
If it's my machine issue wouldn't it fail for
Not sure what what you are on to here coz it still doesn't clariy how @purnesh42H was able to reproduce it for a while; did he have some configuration issue? |
@manuraj17 can you pull the latest and then try running with To clarify, even for me the resolution was just taking a bit longer with |
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed. |
Are you suggesting the latest from the grpc main branch? |
Yes |
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed. |
We ran into this issue running code using the latest grpc-go release as of 8 August 2024. While I unfortunately cannot provide any logs or code, I can say we only started seeing the issue when switching from using Similar to the reporter, changing from localhost to 127.0.0.1 for the server and client also avoided the error, even with all invalid/unreachable nameservers in (Note: while the nameservers in (Also, localhost was listed in /etc/hosts similar to the reporter.) |
Apologies for the double post, forgot the @purnesh42H and I'm not sure editing mentions into comments actually works lol |
Update: curiously, the problem seems to stop manifesting immediately when the machine is physically disconnected from all networks. Perhaps somewhere there's code that's causing this DNS record lookup to get skipped if all network interfaces are down, allowing the localhost entry in /etc/hosts to take over and the health check to proceed? |
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed. |
In new This type of issue with dns resolution is more likely related to system configuration rather than a problem with the gRPC library itself as it relies on the underlying system for DNS resolution. If the system takes time to resolve localhost, this delay will be reflected in gRPC client behavior as well. |
We are seeing the same issue, and indeed using |
@ClaudioAlbertin can you provide the gRPC logs when the DNS resolver fails? |
@ClaudioAlbertin ping on this |
Unfortunately I can only reliably reproduce the issue on our CI server in some tests with very noisy output, but I'll see what logs I can get out of it. It may be worth noting for @manuraj17 that the grpc-go DNS resolver also performs a lookup for a TXT record (in the hope of finding a Service Config), which does not happen if an IP address or the passthrough resolver is used. This lookup contacts an actual DNS server, whereas resolving |
@manuraj17 as suggested by @ClaudioAlbertin, can you try disabling service config using the |
@ClaudioAlbertin to be sure, are you seeing the issue with Also, do you happen to be using an http proxy? If yes, there is a known issue (#7556) which causes DNS resolution to happen on the client instead of the proxy which could cause timeouts if the client can't reach the DNS itself. |
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed. |
I faced this issue when I was testing out a simple application; Not able to put a finger on what exactly am missing.
My client code is returning with this error
I have my server running as
Client connecting to
localhost
asThis code fails.
But when I connect with either of
It works. I am trying to understand what am missing here. Can anyone help me understand? TIA!
Full application code available here
UPDATE (More Info):
Some more information
Version
I am able to connect if I am using
127.0.0.0
and[::1]
The issue specifically seems to occur when
localhost
is used.etc/hosts
is belowI am running the server at
add = :50051
and initialising likeIt works with Dial
When I use
Dial
it worksI have tried all variations to connect without using Dial, and everything except
localhost
works. The resolution of localhost also seems to be working correctly. I get the following logsNOTE: IPV6 gets resolved first.
Though I am not able to figure what failed here. I will have end up not using localhost for now. Though, curious what is happening here.
The text was updated successfully, but these errors were encountered: