Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0123] Flake names #123

Draft
wants to merge 15 commits into
base: master
Choose a base branch
from
Draft
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions rfcs/0123-flake-names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
feature: flake-names
start-date: 2022-03-12
author: Anselm Schüler
co-authors: None
shepherd-team: None
shepherd-leader: None
related-issues: None
---

# Summary
[summary]: #summary

Flakes can declare the field `name`.
It represents the name of the flake.
The derivations for a flake are no longer called `source`, but use the flake name.
schuelermine marked this conversation as resolved.
Show resolved Hide resolved

# Motivation
[motivation]: #motivation

Flake-centric workflows often end up with a lot of derivations named “source”, and it’s difficult to navigate this.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s probably worth a bit more explanations: for most intents and purposes, the store is supposed to be an opaque thing, and not meant to be directly “navigated”.

Now it is a very leaky abstraction, so my remark probably doesn’t really hold in practice, but it would be good to make this explicit (maybe show some concrete examples of where having all the sources be called “source” is an issue)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an issue if you want to know what a nix command is doing at the moment. It often just displays “building XXXX-source”

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong, but that exact message shouldn’t happen (not because of flakes at least). This message happens when Nix builds a derivation, but flake sources aren’t derivations, they are directly instantiated during the evaluation (with builtins.fetchTree) using a different code path with amongst other things a more informative display.

What you might get though is something like “fetching source from https://cache.nixos.org” (because although they aren’t derivations, they can be substituted from the binary cache). Probably we could improve the message in that case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify: That message can happen for reasons unrelated to flakes, because a number of fetchers from nixpkgs hardcode or default name to “source” (and these are regular derivations, so you can get that message)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that’s good to know, sorry for not knowing that.

Copy link
Author

@schuelermine schuelermine May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the store path being meant to be opaque: In my opinion that’s just practically not true at all, and the fact that derivation names exist at all proves it. If anything, at least this is useful for people that want to spelunk in the depths of what Nix does.

I also regularly use store paths to figure out what package or other source provides some binary in my profile. I have a shell function goto-bin for going into the directory of the real path of an executable, and I sometimes skim realpath output to see a derivation name.

This can also make flake-centric commands friendlier and easier to approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Unique names are useful for the purpose of finding duplicate things caused by diamond dependencies. (E.g. `a -> b -> d1`, `a -> c -> d2`. Without a unique name (`d`), it is not possible to correlate `d1` and `d2` reliably.)

# Detailed design
[design]: #detailed-design

A new supported property for flakes is introduced, `name`.
The derivation that contains the flake’s content is called `flake-source-${name}`
If a source control tag (e.g. git tag such as V3) is available, it is `flake-source-${name}-${tag}`.
schuelermine marked this conversation as resolved.
Show resolved Hide resolved

# Examples and Interactions
[examples-and-interactions]: #examples-and-interactions

Running `nix flake metadata` on a flake that declares this field displays it at the top.
Running `nix flake show` on a flake that declares this field shows the name instead of the URL, followed by the URL in parentheses.

Examples:

File `/example/flake.nix`
```nix
{
name = "example";
outputs = { ... }: {
lib.example = "example";
};
Comment on lines +42 to +44
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
outputs = { ... }: {
lib.example = "example";
};
outputs = { self, ... }: {
lib.example = self.name;
};

Making the name accessible will allow flake definition libraries (flake-utils, flake-parts, etc) to automatically use it in some places. For instance when defining an anonymous (non-path-value) module, it's a good idea to use the flake name and module attribute to form the module key. Otherwise, the module can't be disabled with disabledModules.
It will also allow to detect when multiple versions of the same flake are combined, and report a good error message instead of "suchandsuch option has already been defined".

}
```

Shell
```console
$ nix eval git+file:///example
[…] copying flake-source-example
"example"
```

Example of interactions:

Shell (using the previous file)
```
$ nix flake metadata /example
Name: example
Resolved URL: git+file:///example
Locked URL: …
$ nix flake show /example
example (git+file:///home/anselmschueler/Code/example?rev=b0&rev=c714c8624f5d49a9d88e6e24550dd88515923c18)
└───lib: unknown
```

# Drawbacks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The big drawback here (where “big” would probably need to be refined, I’ve no idea how much of an issue it is in practice) is that since the name is part of the store path, two flakes with the same content but a different name will end-up in a different place in the store − which itself means that everything that depends on them would have to be rebuild.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But flakes with a different name will necessarily have different content? Or are the flake.* files not copied to the store?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point (I had a convoluted scenario in mind, but that’s probably way out of the scope of this RFC now that I think of it).

That’s actually highlighting an interesting design challenge: There’s currently a neat separation between the fetching code and the code that knows about what the flake is. inputs.foo = something gets translated to a call to builtins.fetchTree something, and its result is then fed to the outputs function. And the fetchTree part doesn’t know (nor has to know) that it’s fetching a flake.
But if the name gets defined in the flake (and used in the output path), then the fetchTree logic must be a bit more convoluted:

  • It has to know whether it is fetching a flake
  • When fetching a flake, it must get its flake.nix, evaluate it to find its name, and only then put it in the store with the correct name.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn’t it be better to have this conversation outside the review widget? I feel like this aspect of the discussion is somewhat hidden & non-chronological.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • When fetching a flake, it must get its flake.nix, evaluate it to find its name, and only then put it in the store with the correct name.

This can be solved by adding name to the lock file.
We can use the same trick for other fetch-related parameters, like whether to fetch submodules or not, whether to git-crypt decrypt, etc, all of which would also be nice to configure in flake.nix.
fetchTree remains unaware of flake-ness.

[drawbacks]: #drawbacks

This may cause clutter and additional maintenance.
Since this changes the output of nix flake metadata and nix flake show, it might cause scripts that read this output to break.

# Alternatives
[alternatives]: #alternatives

Flake names could be handled entirely through outside means, with things like the global registry merely pointing to flakes under names.

# Unresolved questions
[unresolved]: #unresolved-questions

The name scheme could be changed. `flake-source-${name}` could be too long. Alternatives include `source-${name}`.
The interactions with nix flake metadata and nix flake show are not critical to the design, which is mostly aimed at clarifying derivation names.

# Future work
[future]: #future-work

Flake usability can be improved.