"304 Not Modified" responses from the registry

[mangeshpatiln]

We are using yarn 1.22.22. Facing following issue due to the buffer-xor-1.0.3.tgz is not available on path.
[2/4] Fetching packages...
error Error: https://registry.yarnpkg.com/buffer-xor/-/buffer-xor-1.0.3.tgz: Request failed "304 Not Modified"
at ResponseError.ExtendableBuiltin (/usr/lib/node_modules/yarn/lib/cli.js:696:66)
at new ResponseError (/usr/lib/node_modules/yarn/lib/cli.js:802:124)
at Request. (/usr/lib/node_modules/yarn/lib/cli.js:66750:16)
at Request.emit (node:events:518:28)
at module.exports.Request.onRequestResponse (/usr/lib/node_modules/yarn/lib/cli.js:142287:10)
at ClientRequest.emit (node:events:518:28)
at HTTPParser.parserOnIncomingClient (node:_http_client:698:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
at TLSSocket.socketOnData (node:_http_client:540:22)
at TLSSocket.emit (node:events:518:28)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

When we try to access buffer-xor-1.0.2.tgz and buffer-xor-2.0.2.tgz it is available.

[arcanis]

It seems like something broke on the registry. It's strange that every report so far seem to originate from India - could that somehow come from a proxy you'd all be using? I pinged the npm folks on the OpenJS slack, but they're on US timezone, so that may take some time.

[ShellXploit]

+1 @arcanis would you be able to suggest any workarounds we could hit-and-try as we have been facing this issue as well.

[arcanis]

+1 @arcanis would you be able to suggest any workarounds we could hit-and-try as we have been facing this issue as well.

In the future you probably should consider enabling the offline mirror (see this blog post for 1.x, or this page for 2.x/3.x/4.x) to avoid having your builds rely on the npm registry, which is known to be flaky every once in a while. Unfortunately that's something you need to do ahead of time, so it won't help in the present case.

Right now, unless you have a perfect mirror available (I don't have any to recommend, especially since it has security implications), I don't have a workaround to suggest.

[arcanis]

Seems to affect npm as well btw: npm/cli#7986

[arcanis]

Please note that using a registry you don't control can be dangerous. For instance, being a chinese entity, npmmirror.com is likely subject to the National Intelligence Law. For this reason I'd prefer to avoid suggestions of alternate registries hidden on this repository, to not give the impression the Yarn project is endorsing them.

Remove yarn lock first then

Or migrate to modern releases of Yarn. We haven't stored the registry url in the lockfile for the past six years. If you remove the lockfile, your dependencies will all be upgraded, which may lead to unexpected breakages.

[musadiqpeerzada-refyne]

It seems like something broke on the registry. It's strange that every report so far seem to originate from India - could that somehow come from a proxy you'd all be using? I pinged the npm folks on the OpenJS slack, but they're on US timezone, so that may take some time.

The issue is with Mumbai, India region. working fine at other places

[ShellXploit]

Solution that is working for us:

• Deployed a squid proxy in us-east-1 region
• Configured the proxy to forward all the requests for YARN and NPM official registry
• Updated the yarn proxy using yarn config set proxy <url> and yarn config set https-proxy <url> -- here the URL is of the squid proxy

Since we were using Jenkins with custom runners we have added the yarn config commands to the init script of those nodes to propagate this change across all the pipelines.

Context:
The issue seems to be impacting India region based on the two threads in yarn and npm and by setting up the proxy in us-east-1 all the traffic is being directed to US mirrors.

Disclaimer:

• There will be an increase in the cost from the new resource and the data transfers being done via us-east-1 region
• If you have any private mirrors you will have to whitelist them from the proxy, and even add a routing for them in the squid config
• There will be a significant increase in the latency due to this additional cross-region hop
• While configuring the proxy ensure it's not publicly accessible and you enforce security measures (WAF, inbound rules, etc) to prevent any malicious activity
• Please ensure your compliance doesn't blocks you from routing requests via US servers

[arcanis]

An incident has been opened on the npm side. I'll close & lock this issue as there's nothing the Yarn team can do. Please follow the incident for updates.