Dependency resolution errors due to non-upgradeable packages should refer to compiler version

[sjakobi]

This issue is inspired by two stackoverflow questions where the user wasn't sure how to resolve dependency problems involving base and template-haskell:

https://stackoverflow.com/questions/71173561/how-to-resolve-dependency-issues-in-cabal-project:

$ cabal v2-build
Resolving dependencies...
cabal.exe: Could not resolve dependencies:
[__0] trying: TMS-0.1.0.0 (user goal)
[__1] trying: persistent-2.13.3.0 (dependency of TMS)
[__2] next goal: template-haskell (dependency of persistent)
[__2] rejecting: template-haskell-2.18.0.0/installed-2.18.0.0 (conflict:
persistent => template-haskell>=2.13 && <2.18)
[__2] skipping: template-haskell-2.18.0.0 (has the same characteristics that
caused the previous version to fail: excluded by constraint '>=2.13 && <2.18'
from 'persistent')
[__2] rejecting: template-haskell-2.17.0.0, template-haskell-2.16.0.0,
template-haskell-2.15.0.0, template-haskell-2.14.0.0,
template-haskell-2.13.0.0, template-haskell-2.12.0.0,
template-haskell-2.11.1.0, template-haskell-2.11.0.0,
template-haskell-2.10.0.0, template-haskell-2.9.0.0, template-haskell-2.8.0.0,
template-haskell-2.7.0.0, template-haskell-2.6.0.0, template-haskell-2.5.0.0,
template-haskell-2.4.0.1, template-haskell-2.4.0.0, template-haskell-2.3.0.1,
template-haskell-2.3.0.0, template-haskell-2.2.0.0 (constraint from
non-upgradeable package requires installed instance)
[__2] fail (backjumping, conflict set: persistent, template-haskell)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: persistent, TMS, template-haskell

https://stackoverflow.com/questions/71092123/installing-a-haskell-program-like-hackage-cli:

$ cabal v2-build
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: hackage-cli-0.1.0.0 (user goal)
[__1] next goal: base (dependency of hackage-cli)
[__1] rejecting: base-4.15.1.0/installed-4.15.1.0 (conflict: hackage-cli =>
base>=4.10.0.0 && <4.13)
[__1] skipping: base-4.16.0.0, base-4.15.1.0, base-4.15.0.0, base-4.14.3.0,
base-4.14.2.0, base-4.14.1.0, base-4.14.0.0, base-4.13.0.0 (has the same
characteristics that caused the previous version to fail: excluded by
constraint '>=4.10.0.0 && <4.13' from 'hackage-cli')
[__1] rejecting: base-4.12.0.0, base-4.11.1.0, base-4.11.0.0, base-4.10.1.0,
base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0,
base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1,
base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0,
base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0,
base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from
non-upgradeable package requires installed instance)
[__1] fail (backjumping, conflict set: base, hackage-cli)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, hackage-cli

It would helpful if such messages would make the connection to the compiler version being used and indicate that the problem can be resolved by picking a different version.

[ulysses4ever]

Yes, please! A great hindrance to newcomers.

[gbaz]

Cabal has no builtin database which would let it connect specific base versions to specific ghc versions. We could change the error message to more explicitly state that base is a non-upgradable package and in particular explain what it means to be non-upgradable.

Also, I guess the message could say "you are compiling with ghc-xxx. this version of the compiler comes with package base-4.15.1.0. this package is not able to be reinstalled, a build plan is required that allows this package is required, or you may attempt to build with a compiler that provides base >= 4.10.0.0 && < 4.13"

[noughtmare]

Cabal has no builtin database which would let it connect specific base versions to specific ghc versions.

Perhaps this is a good direction to move towards? Just a quick idea: what if "non-upgradable" would become an explicit property of packages. Then a .cabal package file could contain information about which version of GHC it is fixed to. Cabal would then be able to use that information to figure out which GHC version is capable of building that package.

The main problem I see is that this is very GHC specific. But I guess it shouldn't be very hard to make a similar mechanism work for other compilers. Although it seems that the base hosted on hackage is already GHC specific, so maybe this is not a big problem?

[Mikolaj]

Yes, being GHC specific is not a problem. However, isn't your proposal going in the direction of cabal choosing (or even installing; shudder) a Haskell compiler?That would be problematic. OTOH, suggesting to the user "you may want -w ghc-9.2.2 or relax dep of foo or --allow-newer baz or -v2 to see full solver output" sounds fine, compared to just two screenfuls of solver output as of now.

[noughtmare]

I think it should indeed not be cabal's job to install the compiler. However, I think it would be great if cabal could give accurate error messages and suggestions. Also, I would like to see a hook that allows ghcup/nix to automatically install/switch compiler versions when running cabal, but I don't think that is really related to this particular issue.

[gbaz]

what if "non-upgradable" would become an explicit property of packages?

Cabal already knows which constraints are forced to "builtin". In this case I think we could just change the error message on builtin constraints being unsatisfiable to be clearer.

[noughtmare]

I mean also adding an explicit mention of the version of GHC that a non-upgradable package requires, e.g. base 4.16.0.0 would explicitly list in its .cabal file that is only compiles with GHC 9.2.1. Then the message could be like your suggestion, but end with: "... or you may attempt to build with GHC >= 8.2.1 && < 8.8.1" (assuming that the base to GHC version mapping is monotonic).

[gbaz]

That wouldn't play well with our current solver. We'd have to solve relative to all base versions (no constraint) and then pick out the base versions that would work, if only we were able to install them. Its a lot of work compared to just pointing out more clearly that the current base (version xxx) is pinned to the current compiler version (which is yyy).

[noughtmare]

If you already have the information "base >= 4.10.0.0 && < 4.13" then it should be easy to look in the metadata of those two base versions to find the corresponding GHC versions (if we had that metadata which is what I'm suggesting), right?

[gbaz]

We don't have that information. The problem could occur in any particular choice of any particular transitive dep, and only running a solver in a way that the solver doesn't currently work could collect all possible working choices of base.

[noughtmare]

My suggestion was entirely based on your first comment. To me that comment suggest that knowing which base versions correspond to GHC versions would help and that we do have that "base >= 4.10.0.0 && < 4.13" because you include it in your suggested error message (and it is also in the original error message). Anyway, I think your suggestion is a big improvement already, so maybe it is good to start with that.

[sjakobi]

Cabal has no builtin database which would let it connect specific base versions to specific ghc versions.

For reference: There are machine-readable sources for this data, although they probably are not complete enough to be directly usable by Cabal:

• The GHCup metadata contains this information. See e.g. https://github.com/haskell/ghcup-metadata/blob/ee446fb9f97db95f731a5e4b7b0c76c0d462860a/ghcup-0.0.6.yaml#L157-L159.

• The source file for the boot library version history wiki page even contains the other boot libraries, but it hasn't been updated in a while: https://gitlab.haskell.org/bgamari/ghc-utils/-/blob/7ac783811a261c6c5e09d48022e898fc9d9c217e/library-versions/pkg_versions.txt There are unmerged MRs with updates, e.g. https://gitlab.haskell.org/bgamari/ghc-utils/-/merge_requests/32

[grayjay]

It sounds like there are two challenges here. One is knowing which version of GHC corresponds to each version of each non-upgradable package. The other is suggesting a version of base or GHC that would work.

Finding a compiler version that would work would be very difficult, because the solver currently stops after it finds a single set of conflicts that mean that there is no solution. There could be more. For example, one target package could require an older version of base while another target requires a newer version of template-haskell.

I think we should focus on improving the error message about the current compiler being incompatible, because it would already help avoid a lot of confusion. The error would probably be most useful when the conflict involves only a target package and a non-upgradable package, as in the example with hackage-cli. When more packages are involved, there are more possible ways to resolve the conflict. For example, TMS could loosen its bounds on persistent above, if an older version of persistent worked with the installed version of template-haskell.

[grayjay]

#8542 is related and has a suggestion for the error message.

[ulysses4ever]

I think we should focus on improving the error message about the current compiler being incompatible, because it would already help avoid a lot of confusion.

Please, let's do it: it's 20% of complexity of discussed above for 80% of profit, so to speak.

[VannTen]

Hello. Just stumbled upon that on my first project.

Two questions:

• To identify this case, it is enough to check if the conflict set has packages from the nonUpgradeable list ?
  Or will it be better to check if there is a GlobalConstraintInstalled ?
• Can we just change the "requires installed instance" message (which btw does not mean much when you don't have the context) ? Or will it be better to has a more comprehensive message at the end (with "I've had most trouble..." etc) ?