[RFC 0146] Meta.Categories, not Filesystem Directory Trees #146
Conversation
AndersonTorres
changed the title
|
The idea of this RFC was triggered by the RFC #140, by the way. Ref.: #140 (comment) |
|
One of the advantages of |
| Nonetheless, other good software collections do this just fine, and we can | ||
| easily imitate them. Indeed, we can follow/take a peek at how Repology keeps | ||
| the categorizations defined by those software collections. | ||
|
|
There was a problem hiding this comment.
Another alternative might be an open taxonomy.
There was a problem hiding this comment.
+1. It would allow us to import the existing categories as they are, and thanks to nix we can map them to freedesktop categories in the category definition, like the current license-spdx mapping
There was a problem hiding this comment.
Well, I believe I can produce some preliminar code on this regard.
Basically I can verbatim copy-paste the categories of Freedesktop, just for start.
There was a problem hiding this comment.
Here's a largely uncontrolled categorization, although skewed by a built in scaffolding tool that only suggests some 20 categories: Haskell packages by category
My takeaways:
- listing an uncurated set of categories does not appear to be useful
- curating and restricting are different ideas!
- some categories highlight library ecosystems (e.g. the Scotty category) - this could alternatively be captured by a related packages attribute (presumably out of scope, but worth considering)
- we might want to embed these categories into ours - if we go with some open categories
There was a problem hiding this comment.
I have proposed something like "related categories" on the preliminary code:
AndersonTorres
changed the title
AndersonTorres
force-pushed
the
meta-categories
branch
from
921c726 to
610c110
Compare
| 2. Bikeshedding | ||
|
|
||
| How many and which categories we should create? Can we expand them later? |
There was a problem hiding this comment.
Allowing to add, remove, or split categories would mean that any bikeshedding wouldn't block other processes, which is a good thing.
| ``` | ||
|
|
||
| # Drawbacks | ||
| [drawbacks]: #drawbacks |
There was a problem hiding this comment.
What about ease/speed of finding all packages of a specific category or a set of categories ?
I mean: by using a list of categories it means that to find all compiler packages we would have to go through every packages categories list, and compare every elements of that list.
There is no fast hash lookup like meta.categories.compiler.
(not 100% sure about it) One way to make single category lookup fast could be to generate a lookup attrset like:
{
meta.categories_lookup = { compiler = lib.categories.compiler; . . . };
}Another way could be to NOT use a list of categories in the first place and directly use an attrset like that one above. And get the added benerit of inherit to easily write it (avoiding one more usage of with x; y):
{
meta.categories = { inherit (lib.categories) compiler; . . . };
}The only downside I see is that we loose the ordering of categories (not mentioned yet, but the first one could mark the pkg's primary category) on the other hand it removes a bikeshed waiting to happen (how to order categories).
I don't think ordering is that important though, and if we want to mark the primary category of a pkg there are other ways, like using a special category name meta.categories.primary = lib.categories.foobar.
To check if a package has a set of categories I don't think there's a better way than to check if the pkg is in the first category, then the second, etc..
What do you think?
There was a problem hiding this comment.
Hum, the problem I can see is that it would be not orthogonal. Look:
{
. . .
meta = {
maintainers = [ maintainers.MeMyselfAndI ];
categories = { inherit (lib.categories) compiler; }
};
. . .
}Why maintainers is a list while categories is a strange dictionary?
There was a problem hiding this comment.
Why maintainers is a list while categories is a strange dictionary?
That can be solved by migrating maintainers list to a dictionary.
There was a problem hiding this comment.
And meta.platfoms too?
nix-repl> :l <nixpkgs>
Added 16546 variables.
nix-repl> pkgs.bochs.meta.platforms
[ "i686-cygwin" "x86_64-cygwin" "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" "i686-freebsd" "x86_64-freebsd" "x86_64-solaris" "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux" "armv7l-linux" "i686-linux" "m68k-linux" "mipsel-linux" "mips64el-linux" "powerpc64-linux" "powerpc64le-linux" "riscv32-linux" "riscv64-linux" "s390-linux" "s390x-linux" "x86_64-linux" "aarch64-netbsd" "armv6l-netbsd" "armv7a-netbsd" "armv7l-netbsd" "i686-netbsd" "m68k-netbsd" "mipsel-netbsd" "powerpc-netbsd" "riscv32-netbsd" "riscv64-netbsd" "x86_64-netbsd" "i686-openbsd" "x86_64-openbsd" "x86_64-redox" ]
nix-repl>
There was a problem hiding this comment.
One could switch meta to attrubute sets, but currently it's all lists, yeah, so makes sense to follow this tradition
I'd like to have a section in this RFC discussing its relationship and synergies with RFC 140. Does one of them depend on the other? Or are they completely orthogonal to each other? |
|
Completely orthogonal. Certainly if this RFC solves a corner problem of that other #140 (losing the classification), but they have no dependencies. |
|
For initial categorization one could extract categories from current package paths. So for example applications/editors/emacs get |
| backward references. | ||
| - Especially in some auxiliary tools like editor plugins, testing suites, | ||
| autoupdate scripts and so on. | ||
| - Rewriting `all-packages.nix` can be error-prone (even using Metapad) and it |
There was a problem hiding this comment.
Also, all-packages.nix is HUGE. 40306 lines on the master at the time of me writing this comment. No person can ever go through that manually, which means that one would have to use file searching functionality (which is very inconvenient for stuff like python3, which is often also an argument to a lot of other packages, and very error-prone because sometimes stuff has weird names). It also takes a while to load and parse.
For now I have made a preliminary code‚ copying the Freedesktop.org menu entries. It is already in the PR queue for Nixpkgs acceptance. |
|
@fgaz @natsukium friendly ping |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfc-0146-fcp-meta-categories/47818/1 |
|
So we are starting the Final Comment Period with disposition to accept, running until 2024-07-08 Discourse announcement: https://discourse.nixos.org/t/rfc-0146-fcp-meta-categories/47818 We are also now looking for volunteers for the category list maintenance team! |
|
I volunteer to be part of the Categorization Team. I have a strong background in ontology from my work on Wikidata, so I feel that I could meaningfully contribute here. (This is the correct spot, right? 🙂) |
|
Would be intressted in joing the catagory maintenance team. |
Me too. |
|
Re: team — @GetPsyched over at Discourse https://discourse.nixos.org/t/rfc-0146-fcp-meta-categories/47818/2 (as the majority of team joining interest comments are here, I make a reference not to forget) |
|
Apologies, I didn't want to create more noise on the RFC itself; however since we're nearing a merge soon, I guess this is fine. |
|
What you did is perfectly good, but my duty as a shepherd leader is to keep the situation easier to summarise! (And yeah, we don't have and don't expect much else going here right now) |
|
RFCSC: @7c6f434c, the FCP period has passed, can you confirm if this RFC is good to be merged? |
GetPsyched
added
status: FCP
|
Yes, I think there have been no blocking objections (and some local criticisms have been addressed acceptably) |
GetPsyched
added
status: accepted
and removed
status: FCP
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfcsc-meeting-2024-07-08/48678/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/tagging-categorising-nixpkgs/55396/3 |
This RFC proposes a new
meta.categoriesattribute in order to categorize Nix packages, instead of using the filesystem dirtree.Rendered!