Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

treewide: replace substituteAll with replaceVars (#1) #365440

Merged
merged 1 commit into from
Dec 16, 2024
Merged

treewide: replace substituteAll with replaceVars (#1) #365440

merged 1 commit into from
Dec 16, 2024

Conversation

wolfgangwalther
Copy link
Contributor

Some more gifts from the refactor tractor.

Driving towards #237216

Those are a sub-set of really simple-to-replace cases. I ran the following script:

  find pkgs -type f -iname '*.nix' -print0 | \
    xargs -0 -n100 perl -0777 -i -pe \
      's/substituteAll {([^}]*?)\n[ ]*src =([^;]+);/replaceVars\2 \{\1/gs'

  git ls-files -m | xargs sed -i -E \
    -e 's/\{ substituteAll\b/{ replaceVars/g' \
    -e 's/, substituteAll\b/, replaceVars/g' \
    -e 's/substituteAll,/replaceVars,/g' \
    -e 's/substituteAll;/replaceVars;/g'

but then only selected a smaller part of the changes with the goal of avoiding too many rebuilds, so that this can target master and easily be tested with nixpkgs-review.

Everything that fails to build is left out and will come at the end with more specific fixes.

@philiptaron

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@wolfgangwalther
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review

x86_64-linux

⏩ 4 packages marked as broken and skipped:
  • postgresql13JitPackages.pgvecto-rs
  • postgresql13Packages.pgvecto-rs
  • postgresql17JitPackages.pgvecto-rs
  • postgresql17Packages.pgvecto-rs
❌ 5 packages failed to build:
  • obs-studio-plugins.obs-transition-table
  • python311Packages.heudiconv
  • python311Packages.heudiconv.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
✅ 189 packages built:
  • activitywatch
  • ansible-cmdb
  • ansible-cmdb.dist
  • apptainer
  • apptainer-overriden-nixos
  • authenticator
  • auto-cpufreq
  • auto-cpufreq.dist
  • aw-server-rust
  • bashblog
  • bcache-tools
  • bonnmotion
  • bottles
  • bottles-unwrapped
  • budgie-control-center
  • budgie-control-center.debug
  • budgie-session
  • budgie-session.debug
  • budgie-session.man
  • calyx-vpn
  • cargo-benchcmp
  • chawan
  • cheese
  • cheese.devdoc
  • cheese.man
  • cjdns
  • cosmocc
  • cosmopolitan
  • cosmopolitan.dist
  • coz
  • curseradio
  • curseradio.dist
  • cytoscape
  • davfs2
  • dcm2niix
  • deja-dup
  • dmensamenu
  • dmensamenu.dist
  • ebpf-verifier
  • edgedb
  • emacsPackages.lsp-bridge
  • evolution-ews
  • evolutionWithPlugins
  • febio
  • febio-studio
  • flottbot
  • fprintd
  • fprintd-tod
  • fprintd-tod.devdoc
  • fprintd.devdoc
  • fractal
  • fxlinuxprintutil
  • gammu
  • glycin-loaders
  • gnome-decoder
  • gnome-music
  • govulncheck
  • grilo-plugins
  • heroic
  • home-assistant-component-tests.sms
  • hplipWithPlugin
  • hypnotix
  • ibus-engines.hangul
  • inochi-creator
  • kazam
  • kazam.dist
  • keybase
  • leocad
  • libelfin
  • libpam-wrapper
  • librum
  • libsForQt5.kdenlive
  • libvpl
  • likwid
  • loupe
  • luaformatter
  • mangohud
  • mangohud.doc
  • mangohud.man
  • mangojuice
  • mysql-workbench
  • nekoray
  • networkmanager-fortisslvpn
  • networkmanager-iodine
  • networkmanager-l2tp
  • networkmanager-openconnect
  • networkmanager-openvpn
  • networkmanager-vpnc
  • nifskope
  • nitter
  • novnc
  • obs-studio
  • obs-studio-plugins.advanced-scene-switcher
  • obs-studio-plugins.droidcam-obs
  • obs-studio-plugins.input-overlay
  • obs-studio-plugins.looking-glass-obs
  • obs-studio-plugins.obs-3d-effect
  • obs-studio-plugins.obs-backgroundremoval
  • obs-studio-plugins.obs-command-source
  • obs-studio-plugins.obs-composite-blur
  • obs-studio-plugins.obs-freeze-filter
  • obs-studio-plugins.obs-gradient-source
  • obs-studio-plugins.obs-gstreamer
  • obs-studio-plugins.obs-hyperion
  • obs-studio-plugins.obs-livesplit-one
  • obs-studio-plugins.obs-move-transition
  • obs-studio-plugins.obs-multi-rtmp
  • obs-studio-plugins.obs-mute-filter
  • obs-studio-plugins.obs-ndi
  • obs-studio-plugins.obs-nvfbc
  • obs-studio-plugins.obs-pipewire-audio-capture
  • obs-studio-plugins.obs-replay-source
  • obs-studio-plugins.obs-rgb-levels-filter
  • obs-studio-plugins.obs-scale-to-sound
  • obs-studio-plugins.obs-shaderfilter
  • obs-studio-plugins.obs-source-clone
  • obs-studio-plugins.obs-source-record
  • obs-studio-plugins.obs-source-switcher
  • obs-studio-plugins.obs-teleport
  • obs-studio-plugins.obs-text-pthread
  • obs-studio-plugins.obs-tuna
  • obs-studio-plugins.obs-vaapi
  • obs-studio-plugins.obs-vertical-canvas
  • obs-studio-plugins.obs-vintage-filter
  • obs-studio-plugins.obs-vkcapture
  • obs-studio-plugins.obs-webkitgtk
  • obs-studio-plugins.obs-websocket
  • obs-studio-plugins.waveform
  • obs-studio-plugins.wlrobs
  • obs-studio.debug
  • onionshare
  • onionshare-gui
  • onionshare-gui.dist
  • onionshare.dist
  • org-stats
  • passage
  • pika-backup
  • postgresql14JitPackages.pgvecto-rs
  • postgresql14Packages.pgvecto-rs
  • postgresql15JitPackages.pgvecto-rs
  • postgresql15Packages.pgvecto-rs
  • postgresql16JitPackages.pgvecto-rs
  • postgresql16Packages.pgvecto-rs
  • protontricks
  • protontricks.dist
  • python-cosmopolitan
  • python311Packages.caffe
  • python311Packages.caffe.bin
  • python311Packages.pypamtest
  • python311Packages.python-gammu
  • python311Packages.python-gammu.dist
  • python312Packages.pypamtest
  • python312Packages.python-gammu
  • python312Packages.python-gammu.dist
  • qgis
  • qgis-ltr
  • qvge
  • r128gain
  • r128gain.dist
  • riseup-vpn
  • rtfm
  • scanservjs
  • shotcut
  • sing-geosite
  • siyuan
  • snapshot
  • static-server
  • stumpish
  • teavpn2
  • tmuxPlugins.fingers
  • tmuxPlugins.tmux-thumbs
  • totem
  • typescript-language-server
  • vdhcoapp
  • veracrypt
  • vym
  • vym.man
  • warsow
  • warsow-engine
  • webcord-vencord
  • weechatScripts.multiline
  • weechatScripts.wee-slack
  • wingpanel-indicator-ayatana
  • xborders
  • xborders.dist
  • yabridge
  • you-get
  • you-get.dist
  • zsh-powerlevel10k

The failing packages are failing on master, too.

philiptaron
philiptaron approved these changes Dec 15, 2024
View reviewed changes
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit message appears to GitHub to be referencing this PR:

And when merged will likely refer to this one:

pkgs/applications/graphics/leocad/default.nix Show resolved Hide resolved
pkgs/applications/graphics/qvge/default.nix Show resolved Hide resolved
@philiptaron
Copy link
Contributor

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365440

x86_64-linux

⏩ 4 packages marked as broken and skipped:
  • postgresql13JitPackages.pgvecto-rs
  • postgresql13Packages.pgvecto-rs
  • postgresql17JitPackages.pgvecto-rs
  • postgresql17Packages.pgvecto-rs
❌ 10 packages failed to build:
  • activitywatch
  • cosmocc
  • cosmopolitan
  • cosmopolitan.dist
  • obs-studio-plugins.obs-transition-table
  • python-cosmopolitan
  • python311Packages.heudiconv
  • python311Packages.heudiconv.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
✅ 184 packages built:
  • ansible-cmdb
  • ansible-cmdb.dist
  • apptainer
  • apptainer-overriden-nixos
  • authenticator
  • auto-cpufreq
  • auto-cpufreq.dist
  • aw-server-rust
  • bashblog
  • bcache-tools
  • bonnmotion
  • bottles
  • bottles-unwrapped
  • budgie-control-center
  • budgie-control-center.debug
  • budgie-session
  • budgie-session.debug
  • budgie-session.man
  • calyx-vpn
  • cargo-benchcmp
  • chawan
  • cheese
  • cheese.devdoc
  • cheese.man
  • cjdns
  • coz
  • curseradio
  • curseradio.dist
  • cytoscape
  • davfs2
  • dcm2niix
  • deja-dup
  • dmensamenu
  • dmensamenu.dist
  • ebpf-verifier
  • edgedb
  • emacsPackages.lsp-bridge
  • evolution-ews
  • evolutionWithPlugins
  • febio
  • febio-studio
  • flottbot
  • fprintd
  • fprintd-tod
  • fprintd-tod.devdoc
  • fprintd.devdoc
  • fractal
  • fxlinuxprintutil
  • gammu
  • glycin-loaders
  • gnome-decoder
  • gnome-music
  • govulncheck
  • grilo-plugins
  • heroic
  • home-assistant-component-tests.sms
  • hplipWithPlugin
  • hypnotix
  • ibus-engines.hangul
  • inochi-creator
  • kazam
  • kazam.dist
  • keybase
  • leocad
  • libelfin
  • libpam-wrapper
  • librum
  • libsForQt5.kdenlive
  • libvpl
  • likwid
  • loupe
  • luaformatter
  • mangohud
  • mangohud.doc
  • mangohud.man
  • mangojuice
  • mysql-workbench
  • nekoray
  • networkmanager-fortisslvpn
  • networkmanager-iodine
  • networkmanager-l2tp
  • networkmanager-openconnect
  • networkmanager-openvpn
  • networkmanager-vpnc
  • nifskope
  • nitter
  • novnc
  • obs-studio
  • obs-studio-plugins.advanced-scene-switcher
  • obs-studio-plugins.droidcam-obs
  • obs-studio-plugins.input-overlay
  • obs-studio-plugins.looking-glass-obs
  • obs-studio-plugins.obs-3d-effect
  • obs-studio-plugins.obs-backgroundremoval
  • obs-studio-plugins.obs-command-source
  • obs-studio-plugins.obs-composite-blur
  • obs-studio-plugins.obs-freeze-filter
  • obs-studio-plugins.obs-gradient-source
  • obs-studio-plugins.obs-gstreamer
  • obs-studio-plugins.obs-hyperion
  • obs-studio-plugins.obs-livesplit-one
  • obs-studio-plugins.obs-move-transition
  • obs-studio-plugins.obs-multi-rtmp
  • obs-studio-plugins.obs-mute-filter
  • obs-studio-plugins.obs-ndi
  • obs-studio-plugins.obs-nvfbc
  • obs-studio-plugins.obs-pipewire-audio-capture
  • obs-studio-plugins.obs-replay-source
  • obs-studio-plugins.obs-rgb-levels-filter
  • obs-studio-plugins.obs-scale-to-sound
  • obs-studio-plugins.obs-shaderfilter
  • obs-studio-plugins.obs-source-clone
  • obs-studio-plugins.obs-source-record
  • obs-studio-plugins.obs-source-switcher
  • obs-studio-plugins.obs-teleport
  • obs-studio-plugins.obs-text-pthread
  • obs-studio-plugins.obs-tuna
  • obs-studio-plugins.obs-vaapi
  • obs-studio-plugins.obs-vertical-canvas
  • obs-studio-plugins.obs-vintage-filter
  • obs-studio-plugins.obs-vkcapture
  • obs-studio-plugins.obs-webkitgtk
  • obs-studio-plugins.obs-websocket
  • obs-studio-plugins.waveform
  • obs-studio-plugins.wlrobs
  • obs-studio.debug
  • onionshare
  • onionshare-gui
  • onionshare-gui.dist
  • onionshare.dist
  • org-stats
  • passage
  • pika-backup
  • postgresql14JitPackages.pgvecto-rs
  • postgresql14Packages.pgvecto-rs
  • postgresql15JitPackages.pgvecto-rs
  • postgresql15Packages.pgvecto-rs
  • postgresql16JitPackages.pgvecto-rs
  • postgresql16Packages.pgvecto-rs
  • protontricks
  • protontricks.dist
  • python311Packages.caffe
  • python311Packages.caffe.bin
  • python311Packages.pypamtest
  • python311Packages.python-gammu
  • python311Packages.python-gammu.dist
  • python312Packages.pypamtest
  • python312Packages.python-gammu
  • python312Packages.python-gammu.dist
  • qgis
  • qgis-ltr
  • qvge
  • r128gain
  • r128gain.dist
  • riseup-vpn
  • rtfm
  • scanservjs
  • shotcut
  • sing-geosite
  • siyuan
  • snapshot
  • static-server
  • stumpish
  • teavpn2
  • tmuxPlugins.fingers
  • tmuxPlugins.tmux-thumbs
  • totem
  • typescript-language-server
  • vdhcoapp
  • veracrypt
  • vym
  • vym.man
  • warsow
  • warsow-engine
  • webcord-vencord
  • weechatScripts.multiline
  • weechatScripts.wee-slack
  • wingpanel-indicator-ayatana
  • xborders
  • xborders.dist
  • yabridge
  • you-get
  • you-get.dist
  • zsh-powerlevel10k

@jian-lin jian-lin removed their request for review December 16, 2024 02:53
@philiptaron philiptaron merged commit 66cdf59 into NixOS:master Dec 16, 2024
23 of 24 checks passed
@wolfgangwalther wolfgangwalther deleted the refactor-tractor-1 branch December 16, 2024 07:41
@wolfgangwalther
Copy link
Contributor Author

wolfgangwalther commented Dec 19, 2024 •
edited
Loading

@jwiegley posted a comment directly on the merged commit. For visibility, I reproduce it here:

Hi, I'm not sure why this didn't show up in the automated testing, but this change cannot work because there is no gcc variable to be replaced in setup.sh. It seems that replaceVars is more strict than substituteAll in this regard.

This relates to pkgs/misc/my-env/default.nix.

Looking at the nixpkgs-review list, this package was indeed not built. Fix in #366611.

@wolfgangwalther
Copy link
Contributor Author

I went through the builds that failed for other reasons during nixpkgs-review and those that didn't trigger a build at all, e.g. because of conditionals. Confirmed manually that all replacements are available - no other problems found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@philiptaron philiptaron philiptaron approved these changes

@adisbladis adisbladis Awaiting requested review from adisbladis

@AndersonTorres AndersonTorres Awaiting requested review from AndersonTorres

@K900 K900 Awaiting requested review from K900

@Ma27 Ma27 Awaiting requested review from Ma27

@matthewbauer matthewbauer Awaiting requested review from matthewbauer

@NickCao NickCao Awaiting requested review from NickCao

@SuperSandro2000 SuperSandro2000 Awaiting requested review from SuperSandro2000

@thoughtpolice thoughtpolice Awaiting requested review from thoughtpolice

@ttuegel ttuegel Awaiting requested review from ttuegel

Assignees
No one assigned
Labels
6.topic: emacs Text editor 6.topic: games 6.topic: qt/kde 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@wolfgangwalther @philiptaron @jwiegley