buildNodePackage produces ENOTCACHED when referring to the same package via different hashes (?)

[oddlama]

Describe the bug

When a package-lock.json file contains references to the same package and version but by using a different hashing algorithm (e.g. sha1 and sha512), the subsequent npm build invocation in the sandbox seems to only find one of them, raising an ENOTCACHED error for the other.

Steps To Reproduce

{lib, pkgs, ...}: {
  myPackage = pkgs.buildNpmPackage rec {
    pname = "immich-server";
    version = "1.52.1";
    src = pkgs.fetchgit {
      url = "https://github.com/immich-app/immich.git";
      rev = "34d300d1da296721d647c751169c4441ba33ec7d";
      sha256 = "15jap54jrk8gf79ifdfcvkkmhs3pf53r3skdsz0nylqbqa7gwkbr";
    };
    npmDepsHash = lib.readFile (pkgs.runCommand "immich-npm-deps" {} ''
      ${pkgs.prefetch-npm-deps}/bin/prefetch-npm-deps ${src}/package-lock.json > $out
    '');

    nativeBuildInputs = with pkgs; [
      pkg-config
      python3
    ];

    # Required for sharp
    buildInputs = with pkgs; [
      vips
    ];
  };
}

try to build the contained derivation and the following error will appear:

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/apsv8hkws0i4l77w0gikgp3zrczrv7xx-server
source root is server
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
Executing npmConfigHook
Configuring npm
Validating consistency between /tmp/nix-build-immich-server-1.52.1.drv-0/server/package-lock.json and /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/package-lock.json
Installing dependencies
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: Use your platform's native performance.now() and performance.timeOrigin.
npm WARN deprecated [email protected]: Please use @jridgewell/sourcemap-codec instead
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm ERR! code ENOTCACHED
npm ERR! request to https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz failed: cache mode is 'only-if-cached' but no cached response is available.

npm ERR! Log files were not written due to an error writing to the directory: /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal

ERROR: npm failed to install dependencies

Here are a few things you can try, depending on the error:
1. Set `makeCacheWritable = true`
  Note that this won't help if npm is complaining about not being able to write to the logs directory -- look above that for the actual error.
2. Set `npmFlags = [ "--legacy-peer-deps" ]`

The corresponding multiple definitions that can be found for color-name-1.1.3 in the lockfile are:

14888:        "color-name": {
14889-          "version": "1.1.3",
14890:          "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz",
14891-          "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw=="
14892-        }

11988:        "color-name": {
11989-          "version": "1.1.3",
11990:          "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz",
11991-          "integrity": "sha1-p9BVi9icQveV3UIyj3QIMcpTvCU="
11992-        },

[some other identical ones]

Nonetheless, both files seem to be available:

base64 -d <<< "p9BVi9icQveV3UIyj3QIMcpTvCU=" | xxd -pr
a7d0558bd89c42f795dd42328f740831ca53bc25

base64 -d <<< "dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==" | xxd -pr

74ecbedc0b96ddadb035b64722e319a537208c6b8b53fb812ffb9b71917d3976c3a3c7dfe0ef32569e417f479f4bcb84a18a39ab8171edd63d3a04065e002c40
ls -l /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/_cacache/content-v2/sha1/a7/d0/558bd89c42f795dd42328f740831ca53bc25 /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/_cacache/content-v2/sha512/74/ec/bedc0b96ddadb035b64722e319a537208c6b8b53fb812ffb9b71917d3976c3a3c7dfe0ef32569e417f479f4bcb84a18a39ab8171edd63d3a04065e002c40
-r--r--r-- 1 root root 4.0K Jan  1  1970 /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/_cacache/content-v2/sha1/a7/d0/558bd89c42f795dd42328f740831ca53bc25
-r--r--r-- 1 root root 2.9K Jan  1  1970 /nix/store/b64hvf4lf576vsa2r9c0c406q886227m-immich-server-1.52.1-npm-deps/_cacache/content-v2/sha512/74/ec/bedc0b96ddadb035b64722e319a537208c6b8b53fb812ffb9b71917d3976c3a3c7dfe0ef32569e417f479f4bcb84a18a39ab8171edd63d3a04065e002c40

Regenerating the lockfile (and thus replacing all hashing algorithms with sha512) "fixes" the issue, or more likely works around it.

Expected behavior

The original package-lock file should work without modifications.

Notify maintainers

@winterqt

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

[lilyinstarlight]

This should be fixed by #214454 and another branch that I don't think has been PR'd yet

I'll try to get through those and get the fixes merged in the next few days (the FOD for open-stage-control is actually currently non-reproducible because of this and it's the exact same color-name-1.1.3 dep funnily enough)

[Qyriad]

Is there any workaround in the meantime until #214454 is merged?

[lilyinstarlight]

Is there any workaround in the meantime until #214454 is merged?

Can you test what I've pushed to that PR now?

I just pulled in that commit from that other branch and made some fixups, so it would be helpful to get testers. Thank you!

[pinpox]

@oddlama Do you have a working derivation for running immich on NixOS? I'm looking into setting it up

[oddlama]

@pinpox Yes, look here if you need it right now, but beware that this isn't 100% finished yet (some runtime deps are still missing). Once finished I will upstream this into nixkpgs. (The draft PR #244803)