Wrap Qt applications

[ttuegel]

Introduces wrapQtAppsHook similar to wrapGAppsHook to solve long-standing problems with Qt applications, particularly plugin search paths. Wrapped Qt applications can run directly from the Nix store without being installed and using nix-shell --pure. (Actually, applications that use OpenGL on NixOS cannot run from pure Nix shells because of Mesa; this is solved by setting LD_LIBRARY_PATH for the OpenGL drivers. I did not address this here because it is not a Qt problem.)

This does not apply the hook to every application. My intent is that package maintainers can make those changes can be merged to master. Ultimately, the burden of interactive testing will fall on them anyway. This is just the changes to Qt necessary to make that possible. Ark and Okular are wrapped, as proof-of-concept.

This also does not remove the patches from Qt that infers plugin and QML paths from PATH; that is still necessary for desktop environments (particularly themes) to work as expected. I worked on this pull request for more than a year, but I am still no closer to getting rid of that patch. Finally I decided to rip out everything but the simplest version of the wrapper. For the time being, we can use both the wrapper and the patch together; the latter is inelegant, but everyone's applications will work.

This would close #44047, as it addresses the same issue, but more completely: it solves problems with QT_PLUGIN_PATH, QML2_IMPORT_PATH, XDG_DATA_DIRS and others.

Closes: #53750, #44047, #51597, #42864 (probably others)

TODO: closure size analysis and documentation

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Assured whether relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

[matthewbauer]

I kind of thought that was an intentional design decision in qt infra to allow different qt runtimes…

This goes back to the debate on wrapping vs propagatedUserEnvPkgs. Wrapping seems like a simpler way to do things to me, but I’m not sure if everyone agrees. See discussion on this issue by @edolstra:

#43049 (comment)

[ttuegel]

I kind of thought that was an intentional design decision in qt infra to allow different qt runtimes

Dynamically loading Qt plugins at runtime is intentional, and this pull request does not alter that. This pull request addresses the fact that it is much, much too easy to mix Qt versions--which always ends badly.

[danbst]

@ttuegel would this help also for proprietary binaries, where Qt is patched into binary with autoPatchelfHook? (a recent example #54969 (comment))

[ttuegel]

@danbst This alone would not help the situation with binary distributions, but it would help not to put everything into propagatedUserEnvPkgs and this would let us do that.

[peterhoeg]

Is there any reason we don't make wrapQtAppsHook part of the KDE specific mkDerivation and apply it unconditionally to avoid having to touch everything in kdeApplications?

[flokli]

@peterhoeg this still won't fix packages outside KDE world, and there's a lot of QT applications out there - so I really prefer the approach in #44047 not requiring any wrapper.

[samueldr]

Sorry if I sound grumpy or mad, but it's not my intention in the comment. Read the following in a neutral voice.

I may be out of line here (and definitely biased), but "similar to wrapGAppsHook" will mean that we're going to have to deal with the current assortment of failures in a less predictable fashion, just like #54278, where currently a bunch of mismatched gdk_pixbuf work, and a bunch fail.

Fixing the issues in the libraries directly would ensure no one has to remember to use that one hook on their software, and fixes the issue closer to the source, instead of papering over. The main issue being that most of the time not adding the hook works fine. There wouldn't be much of a developer experience (DX?) issue if it failed noisily in all circumstances when the hook is missing.

I have one concession to make to those wrappers: they require much less effort to maintain. Solving this particular issue at the root isn't really intrusive, and relatively self-contained, but I guess it's half part to good software and half part to luck.

[ttuegel]

Is there any reason we don't make wrapQtAppsHook part of the KDE specific mkDerivation and apply it unconditionally to avoid having to touch everything in kdeApplications?

@peterhoeg I haven't tried that, but it should be OK.

EDIT: For that matter, I don't see any reason not add wrapQtAppsHook to qt5.mkDerivation.

[ttuegel]

The main issue being that most of the time not adding the hook works fine. There wouldn't be much of a developer experience (DX?) issue if it failed noisily in all circumstances when the hook is missing.

@samueldr
I agree that a solution which requires no developer intervention--or is at least noisy about needing developer intervention--provides a better experience. Despite that, a number of Qt application maintainers are already rolling their own wrappers. I realized--thanks to @peterhoeg's remark above--that we could simply include the hook in mkDerivation; would that satisfy?

[peterhoeg]

#44047 is a lot more invasive and I for one simply don't know enough about Qt internals to be able to say anything remotely intelligent about the viability of that approach.

This PR (with the qt5.mkDerivation variant) will solve the first 3 out of the following 4.

• KDE applications in nixpkgs
• Qt applications in nixpkgs
• Qt applications out of tree
• Binary Qt applications

Why don't we move ahead with this for now while working on #44047?

[timor]

Thanks for the effort! I just recently thought of trying the exact same thing before I found this issue :).

[matthewbauer]

@ttuegel pinging on this. Hoping we can get this in soon as it closes quite a few other issues around this.

[edolstra]

This caused significant closure size increases, see #69272.

[edolstra]

To be honest the wrapQtApp approach (generally turning most build-time dependencies into runtime dependencies) seems fundamentally wrong to me. So we end up with cmake, pkgconfig, patchelf, flex, bison and dozens of -dev outputs in the runtime closure just because they happen to have a /share subdirectory.

Possibly better approach: only add those packages to XDG_DATA_DIRS that are already in the runtime closure.

[ttuegel]

To be honest the wrapQtApp approach (generally turning most build-time dependencies into runtime dependencies) seems fundamentally wrong to me.

The wrapper hook is designed only to pick up host dependencies, so I really don't understand why nativeBuildInputs are being captured.

As far as Qt is concerned, the correct solution is to comply with FHS so that resources have well-known canonical names, but that's not an option here.

Possibly better approach: only add those packages to XDG_DATA_DIRS that are already in the runtime closure.

I think it is unlikely that this would work because the Qt/KDE folks strongly prefer dynamic dependencies, but for the sake of argument, how can we detect during the build what packages are in the runtime closure?

[edolstra]

The wrapper hook is designed only to pick up host dependencies

That doesn't prevent capturing -dev outputs, right?

Anyway, a short-term fix might be:

• Filter out *-dev.
• Be more selective than including every store path that has a /share subdirectory, but instead check for the actual paths that the app requires (e.g. /share/applications or /share/mime).

how can we detect during the build what packages are in the runtime closure?

Probably looking at the RPATH (patchelf --print-rpath) is a decent approximation.

[ttuegel]

That doesn't prevent capturing -dev outputs, right?

It prevents capturing build tools like cmake, and this:

Be more selective than including every store path that has a /share subdirectory, but instead check for the actual paths that the app requires (e.g. /share/applications or /share/mime).

We already select specific required paths in the setup hook, which should also eliminate -dev outputs.

Probably looking at the RPATH (patchelf --print-rpath) is a decent approximation.

This will break at least packages that: use plugins, use QML, or have kservice (D-Bus) dependencies.

Filter out *-dev.

This seems like a reasonable course of action.