treewide: support structuredAttrs in setup hooks

[wolfgangwalther]

Description of changes

I tried to build a package with __structuredAttrs which uses meson, realized that meson setup hooks don't support structuredAttrs, yet, and then... this happened.

The first 5 commits make the helpers in stdenv related to structuredAttrs a bit better re-usable. The remaining commits are using the "new" concatTo and prependToVar / appendToVar to fix many setup hooks.

The main goal here is to support structuredAttrs in those setup hooks without having any "if structuredAttrs then ..." conditionals except for a few in stdenv. I think this works pretty well.

There's a lot of changes in here, but many of them should be very easy to review. Most of them were tested by building a package using the related hooks: Once with structuredAttrs on master, which would fail, then with and without structuredAttrs on this branch - both would pass.

In any case, I pushed all those commits mostly to show how those helpers in stdenv would be used across the different hooks. I am happy to split up the PR into smaller pieces later and would welcome feedback especially on the first 5 commits.

The problem in 99% of those hooks is that $xxxFlags is used without [@], which will always just return the first array item when using structuredAttrs. However a simple change to $xxxFlags[@] isn't going to fly, if whitespace should be treated properly.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[wolfgangwalther]

Hm, so @AndersonTorres is listed as maintainer for scons, but was not mentioned by ofborg. I assume that's because I only change the setup-hook.sh file and not a nix file.

Do I need to mention all maintainers manually in this case? I assumed ofborg would also work by commit prefix scons:, but that didn't add the mention either.

[wolfgangwalther]

Rebased to resolve merge conflicts. Anyone up for a review of the first five stdenv: commits for the general base?

[SomeoneSerge]

We'll have to look out to ensure nobody else starts to depend on the stdout-based concatStringsSep in the meantime, but I think that's manageable:)

[emilazy]

Really great to see this land! Looking forward to the follow‐up for the other hooks. Let’s all do our best to get structured attributes on by default as soon as we can.

[trofi]

Bisect says the bfd97a6 stdenv: make _accumFlagsArray independent of structuredAttrs changed quoting rules for some variables. For example sudo is now broken in staging as:

$ nix build --no-link -f. -L sudo
...
sudo> ./configure: interpreter directive changed from "#! /bin/sh" to "/nix/store/k5i08diz6c0bvklbzdp35rpprrnml9bh-bash-5.2p32/bin/sh"                               
sudo> configure flags: --disable-static --prefix=/nix/store/8kqimiqs90p0vlvm43ia2v46wc040gbw-sudo-1.9.15p5 --with-env-editor --with-editor=/run/current-system/sw/bin
/nano --with-rundir=/run/sudo --with-vardir=/var/db/sudo --with-logpath=/var/log/sudo.log --with-iologdir=/var/log/sudo-io --with-sendmail=/run/wrappers/bin/sendmail
 --enable-tmpfiles.d=no password for %p:                                                                                                                             
sudo> configure: WARNING: you should use --build, --host, --target                                                                                                   
sudo> configure: WARNING: you should use --build, --host, --target                                                                                                   
sudo> configure: WARNING: you should use --build, --host, --target                                                                                                   
sudo> configure: WARNING: invalid host type: %p:        
...
sudo> checking for password-ar... ar
sudo> checking for password-ranlib... ranlib
sudo> checking build system type... Invalid configuration 'password': machine 'password-unknown' not recognized
sudo> configure: error: /nix/store/k5i08diz6c0bvklbzdp35rpprrnml9bh-bash-5.2p32/bin/bash ./scripts/config.sub password failed

This happenens because sudo has this snippet:

  configureFlagsArray = [
    "--with-passprompt=[sudo] password for %p: " # intentional trailing space
  ];

It used to be passed as a single string, but now it gets expanded to an argument list.

[emilazy]

I guess we just need to restore the special handling of *Array? I’ll try to take a look in the next day or so if nobody beats me to it.

[SomeoneSerge]

It used to be passed as a single string, but now it gets expanded to an argument list.

Most peculiar, I'm surprised this used to work because this does not seem to be a __structuredAttrs derivation yet it sets *Array as a nix variable...

EDIT: it probably worked because *Array was used with "${...[@]}" unconditionally of it being an actual array, so maybe bash saw it as "a single element-array that wasn't declared as an array" or something - I'm not a bash expert, but I think we should simply enable __structuredAttrs:)

[emilazy]

I guess just toggling on __structuredAttrs would be nice, but this is going to break out‐of‐tree packages. We probably need to fix it.

[SomeoneSerge]

I guess just toggling on __structuredAttrs would be nice, but this is going to break out‐of‐tree packages. We probably need to fix it.

👍🏻 Two separate matters: make sudo structuredAttrs, add a warning/error out-of-tree users when [[ name = *"Array" ]], possibly recovering the old behavior

[SomeoneSerge]

FWIW I think the way *Array variables are used in the manual is they are set on the bash side, e.g. in preConfigure

EDIT:

I’ll try to take a look in the next day or so if nobody beats me to it.

I'm probably free in a couple of hours

[philiptaron]

With #334973 merged, I think we're safe for staging-next. Anyone disagree?