npm install fails behind corporate proxy

[masteph]

After upgrading from 0.6.3 to 0.6.17 the install process of npm fails in post install step.
It turned out to be a proxy issue:

set NODE_DEBUG=net,http,https
npm install
[...]
HTTP 30356: SOCKET ERROR: connect ETIMEDOUT 104.16.18.35:443 Error: connect ETIMEDOUT 104.16.18.35:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16)
Trying to download "https://registry.npmjs.org/esbuild-windows-64/-/esbuild-windows-64-0.6.17.tgz"
Failed to download "https://registry.npmjs.org/esbuild-windows-64/-/esbuild-windows-64-0.6.17.tgz": connect ETIMEDOUT 104.16.18.35:443
Install unsuccessful
[...]
npm ERR! Failed at the [email protected] postinstall script.
[...]

The reason is that the fetch routine does direct http(s) connections.
https://github.com/evanw/esbuild/blob/master/lib/install.ts#L123

If there would be some way to use npm to download the dependencies that would be great.
Else you should provide a way to respect http_proxy and https_proxy env variables.

Right now I have to find a Windows machine that can do direct downloads and copy the files afterward.

[evanw]

I just published version 0.6.18, which will fall back to npm install if all else fails.

[danvim]

@evanw I'm getting the same problem on esbuild 0.12.20

It failed to run the following and fell back to try to download the binary from npm directly.

PS> npm install --loglevel=error --prefer-offline --no-audit --progress=false [email protected]
npm ERR! code ETARGET
npm ERR! notarget No matching version found for [email protected]
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

But if I remove the prefer offline flag, it works fine.

When I check the npm cache log, it contains

...
27 http fetch GET 200 http://corporate-proxy/esbuild-windows-64 9ms (from cache)
...
36 verbose stack esbuild-windows-64: No match version found for [email protected].
...

I'm assuming my cache doesn't have the latest version, and so prefer offline doesn't really work.

Then once I've downloaded it once without the prefer offline flag. The package info is now cached and yarn add -D esbuild now works perfectly fine.

I have npm 7.6.0

Would removing --prefer-offline be a good solution?

[evanw]

Just curious: Are you able to install [email protected] with your corporate proxy? Knowing if that package works for you or not will hopefully help me fix the root cause of these issues.

Context: The esbuild-experimental-optdeps package isn't meant to be used for real. Instead it's part of an experiment of mine to use a different package installation strategy that doesn't use an install script. If that strategy works out, I'm planning to remove the install script from the esbuild package to hopefully avoid issues like yours. See also issue #789 for more context.

[danvim]

@evanw Hmm... I'm not sure I can verify quickly. This installation problem only happens if I have cache before a new version of esbuild is released, then it wouldn't be able to find the new version in the local cached manifest and fail. Installation always works for the initial installation, or if I clear my npm cache.

To elaborate, because I have installed 0.12.19 previously, I believe the manifest listing all versions until 0.12.19 is available locally. And when 0.12.20 is released, I can install any version below 0.12.20. But installing [email protected] with prefer offline flag would result in the above error. Installing [email protected] without the flag is fine.

So far, I've only been able to verify this theory with the .19 and .20 release.

But I am able to install esbuild-experimental-optdeps.

[Random90]

Hello, I have the same issue as @danvim, being behind corp proxy, but while generating angular 12 library where esbuild is probably dependency of ng-packagr, since that is the only package installed while generating new library.
I don't really have a way to install it manually, since this is a step of schematics workflow process, which just fails:

UPDATE angular.json (11123 bytes)
UPDATE package.json (2315 bytes)
UPDATE tsconfig.json (960 bytes)
| Installing packages (npm)...
> [email protected] postinstall C:\[...]\node_modules\esbuild
> node install.js

Trying to install "esbuild-windows-64" using npm
Failed to install "esbuild-windows-64" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false [email protected]
npm ERR! code ETARGET
npm ERR! notarget No matching version found for [email protected].
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! A complete log of this run can be found in:
[...]

Trying to download "https://registry.npmjs.org/esbuild-windows-64/-/esbuild-windows-64-0.12.21.tgz"
Failed to download "https://registry.npmjs.org/esbuild-windows-64/-/esbuild-windows-64-0.12.21.tgz": connect ETIMEDOUT 104.16.24.35:443
Install unsuccessful
npm WARN [email protected] requires a peer of ajv@^6.9.1 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\webpack-dev-server\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
[...]
× Package install failed, see above.
The Schematic workflow failed. See above.

I am able to install package separately using npm install, so this is probably not the issue with proxy itself.

[evanw]

I suspect these issues may have been introduced by #551, which landed in version 0.8.12 on November 21, 2020. That PR changed the nested npm invocation from a subdirectory inside the package to a temporary folder outside the package.

This change was done to fix issues on Windows since deleting a directory on Windows is randomly sometimes impossible, and leaving a nested node_modules directory there can sometimes cause problems with tools that scan over node_modules directories and expect them to have a specific structure. However, perhaps we could decide we don't care about that case as much as getting this to work.

If my hypothesis is correct, that would mean that installing 0.8.11 behind a corporate proxy should succeed but installing 0.8.12 behind a corporate proxy should fail (assuming installing the latest version of esbuild behind the same proxy also fails). It would be good for someone to verify whether this is the case or not.

In any case, it sounds like I should re-open this issue.

[danvim]

@evanw I believe in my case, I can install any version as long as they are already in the manifest I've cached locally. If you were to release a newer version tomorrow, I wouldn't be able to install that unless I clear my local cache to remove my local manifest for esbuild-windows-64.

[sfwhite]

I'm running into an issue upgrading, with self signed cert exception, while upgrading from 0.12.1 to any version above 0.12.8. Upgrading to 0.12.8 is successful.

Manually executing the step and removing the --prefer-offline flag succeeds on those higher versions.
To further complicate things, versions above 0.12.20 are flagged as malware by our corporate firewall and will not download despite the workaround.

Not sure if this is the same problem, but I'm also behind a corporate proxy, so seemed to be the best fit I could find.

npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node install.js
npm ERR! Trying to install "esbuild-windows-32" using npm
npm ERR! Failed to install "esbuild-windows-32" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false [email protected]
npm ERR! npm ERR! code ETARGET
npm ERR! npm ERR! notarget No matching version found for [email protected].
npm ERR! npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! npm ERR! notarget a package version that doesn't exist.
npm ERR!
npm ERR! npm ERR! A complete log of this run can be found in:
npm ERR! npm ERR! C:\Users\[userid]\AppData\Local\npm-cache_logs\2021-08-25T19_38_34_299Z-debug.log
npm ERR!
npm ERR! Trying to download "https://registry.npmjs.org/esbuild-windows-32/-/esbuild-windows-32-0.12.22.tgz"
npm ERR! Failed to download "https://registry.npmjs.org/esbuild-windows-32/-/esbuild-windows-32-0.12.22.tgz": self signed certificate in certificate chain
npm ERR! Install unsuccessful

[danvim]

So my computer apparently also fails with create react app with a similar issue.

facebook/create-react-app#8294

It also worked when I removed the --offline flag from the installation command. I think they might be related🤔

[silverwind]

It seems this issue with ETARGET is not completely fixed, but I can't provice reliable reproduction but here's part of a failing log:

npm ERR! command sh -c -- node install.js
npm ERR! [esbuild] Failed to find package "esbuild-linux-64" on the file system
npm ERR! [esbuild] Trying to install package "esbuild-linux-64" using npm
npm ERR! [esbuild] Failed to install package "esbuild-linux-64" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false [email protected]
npm ERR! npm ERR! code ETARGET
npm ERR! npm ERR! notarget No matching version found for [email protected].
npm ERR! npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! npm ERR! notarget a package version that doesn't exist.
npm ERR! [esbuild] Trying to download "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.15.12.tgz"
npm ERR! [esbuild] Failed to download "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.15.12.tgz": connect ETIMEDOUT 104.16.18.35:443

Running the failing npm install command manually works and populates the npm cache so subsequent installs work fine. I have honestly no idea how this ETARGET happens and I suspect a bug in npm itself.