From 3a6b694b9beb9b2d80eb75c8df5b6342c6b45774 Mon Sep 17 00:00:00 2001 From: Hofer-Julian <30049909+Hofer-Julian@users.noreply.github.com> Date: Mon, 12 Aug 2024 14:53:28 +0200 Subject: [PATCH] docs(rfc): pixi global proposal (#1757) [Rendered version](https://github.com/Hofer-Julian/pixi/blob/docs/pixi-global-proposal/docs/features/pixi_global_manifest.md) --------- Co-authored-by: Ruben Arts Co-authored-by: Bas Zalmstra --- docs/design_proposals/pixi_global_manifest.md | 252 ++++++++++++++++++ mkdocs.yml | 2 + 2 files changed, 254 insertions(+) create mode 100644 docs/design_proposals/pixi_global_manifest.md diff --git a/docs/design_proposals/pixi_global_manifest.md b/docs/design_proposals/pixi_global_manifest.md new file mode 100644 index 0000000000..eae9efe3f6 --- /dev/null +++ b/docs/design_proposals/pixi_global_manifest.md @@ -0,0 +1,252 @@ +# Pixi Global Manifest + +## Motivation + +`pixi global` is currently limited to imperatively managing CLI packages. +The next iteration of this feature should fulfill the following needs: + +- Shareable global environments. +- Managing complex environments with multiple packages as dependencies +- Flexible exposure of binaries + +## Design Considerations + +There are a few things we wanted to keep in mind in the design: + +1. **User-friendliness**: Pixi is a user focused tool that goes beyond developers. The feature should have good error reporting and helpful documentation from the start. +2. **Keep it simple**: The CLI should be all you strictly need to interact with global environments. +3. **Unsurprising**: Simple commands should behave similar to traditional package managers. +4. **Human Readable**: Any file created by this feature should be human-readable and modifiable. + +## Manifest + +The global environments and exposed will be managed by a human-readable manifest. +This manifest will stick to conventions set by `pixi.toml` where possible. +Among other things it will be written in the TOML format, be named `pixi-global.toml` and be placed at `~/.pixi/manifests/pixi-global.toml`. +The motivation for the location is discussed [further below](#multiple-manifests) + +```toml title="pixi-global.toml" +# The name of the environment is `python` +# It will expose python, python3 and python3.11, but not pip +[envs.python.dependencies] +python = "3.11.*" +pip = "*" + +[envs.python.exposed] +python = "python" +python3 = "python3" +"python3.11" = "python3.11" + +# The name of the environment is `python_3_10` +# It will expose python3.10 +[envs.python_3_10.dependencies] +python = "3.10.*" + +[envs.python_3_10.exposed] +"python3.10" = "python" + +``` + +## CLI + +Install one or more packages `PACKAGE` and expose their binaries. +If `--environment` has been given, all packages will be installed in the same environment. +If the environment already exists, the command will return with an error. +`--expose` can be given if `--environment` is given as well or if only a single `PACKAGE` will be installed. +The syntax for `MAPPING` is `exposed_name=binary_name`, so for example `python3.10=python`. + +``` +pixi global install [--expose MAPPING] [--environment ENV] ... +``` + +Remove environments `ENV`. +``` +pixi global uninstall ... +``` + +Update `PACKAGE` if `--package` is given. If not, all packages in environments `ENV` will be updated. +If the update leads to binaries being removed, it will offer to remove the mappings. +If the user declines the update process will stop. +If the update leads to binaries being added, it will offer for each binary individually to expose it. +`--assume-yes` will assume yes as answer for every question that would otherwise be asked interactively. +``` +pixi global update [--package PACKAGE] [--assume-yes] ... +``` + +Add one or more packages `PACKAGE` into an existing environment `ENV`. +If environment `ENV` does not exist, it will return with an error. +Without `--expose` no binary will be exposed. +If you don't mention a spec like `python=3.8.*`, the spec will be unconstrained with `*`. +The syntax for `MAPPING` is `exposed_name=binary_name`, so for example `python3.10=python`. + +``` +pixi global add --environment ENV [--expose MAPPING] ... +``` + +Remove package `PACKAGE` from environment `ENV`. +If that was the last package remove the whole environment and print that information in the console. +If this leads to binaries being removed, it will offer to remove the mappings. +If the user declines the remove process will stop. +``` +pixi global remove --environment ENV PACKAGE +``` + +Add one or more `MAPPING` for environment `ENV` which describe which binaries are exposed. +The syntax for `MAPPING` is `exposed_name=binary_name`, so for example `python3.10=python`. +``` +pixi global expose add --environment ENV ... +``` + +Remove one or more exposed `BINARY` from environment `ENV` +``` +pixi global expose remove --environment ENV ... +``` + +Ensure that the environments on the machine reflect the state in the manifest. +The manifest is the single source of truth. +Only if there's no manifest, will the data from existing environments be used to create a manifest. +`pixi global sync` is implied by most other `pixi global` commands. + +``` +pixi global sync +``` + +List all environments, their specs and exposed binaries +``` +pixi global list +``` + + +### Simple workflow + +Create environment `python`, install package `python=3.10.*` and expose all binaries of that package +``` +pixi global install python=3.10.* +``` + +Update all packages in environment `python` +``` +pixi global update python +``` + +Remove environment `python` +``` +pixi global uninstall python +``` + +Create environment `python` and `pip`, install corresponding packages and expose all binaries of that packages +``` +pixi global install python pip +``` + +Remove environments `python` and `pip` +``` +pixi global uninstall python pip +``` + +Create environment `python-pip`, install `python` and `pip` in the same environment and expose all binaries of these packages +``` +pixi global install --environment python-pip python pip +``` + + +### Adding dependencies + +Create environment `python`, install package `python` and expose all binaries of that package. +Then add package `hypercorn` to environment `python` but doesn't expose its binaries. + +``` +pixi global install python +pixi global add --environment python hypercorn +``` + +Update package `cryptography` (a dependency of `hypercorn`) to `43.0.0` in environment `python` + +``` +pixi update --environment python cryptography=43.0.0 +``` + +Then remove `hypercorn` again. +``` +pixi global remove --environment python hypercorn +``` + + +### Specifying which binaries to expose + +Make a new environment `python_3_10` with package `python=3.10` and expose the `python` executable as `python3.10`. +``` +pixi global install --environment python_3_10 --expose "python3.10=python" python=3.10 +``` + +Now `python3.10` is available. + + +Run the following in order to expose `python` from environment `python_3_10` as `python310` instead. + +``` +pixi global expose remove --environment python_3_10 python3.10 +pixi global expose add --environment python_3_10 "python310=python" +``` + +Now `python310` is available, but `python3.10` isn't anymore. + + +### Syncing + +Most `pixi global` sub commands imply a `pixi global sync`. + +- Users should be able to change the manifest by hand (creating or modifying (adding or removing)) +- Users should be able to "export" their existing environments into the manifest, if non-existing. +- The manifest is always "in sync" after `install`/`remove`/`inject`/`other global command`. + + +First time, clean computer. +Running the following creates manifest and `~/.pixi/envs/python`. +``` +pixi global install python +``` + +Delete `~/.pixi` and syncing, should add environment `python` again as described in the manifest +``` +rm `~/.pixi/envs` +pixi global sync +``` + +If there's no manifest, but existing environments, pixi will create a manifest that matches your current environments. +It is to be decided whether the user should be asked if they want an empty manifest instead, or if it should always import the data from the environments. +``` +rm +pixi global sync +``` + +If we remove the python environment from the manifest, running `pixi global sync` will also remove the `~/.pixi/envs/python` environment from the file system. +``` +vim +pixi global sync +``` + +## Open Questions + +### Should we version the manifest? + +Something like: + +``` +[manifest] +version = 1 +``` + +We still have to figure out which existing programs do something similar and how they benefit from it. + +### Multiple manifests + +We could go for one default manifest, but also parse other manifests in the same directory. +In order to modify those with the `CLI` one would have to add an option `--manifest` to select the correct one. + +- pixi-global.toml: Default +- pixi-global-company-tools.toml +- pixi-global-from-my-dotfiles.toml + +It is unclear whether the first implementation already needs to support this. +At the very least we should put the manifest into its own folder like `~/.pixi/global/manifests/pixi-global.toml` diff --git a/mkdocs.yml b/mkdocs.yml index b1f0566e73..6e14f58389 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -120,6 +120,8 @@ nav: - Multi Platform: features/multi_platform_configuration.md - Multi Environment: features/multi_environment.md - Lockfile: features/lockfile.md + - Design Proposals: + - Pixi Global Manifest: design_proposals/pixi_global_manifest.md - Advanced: - Authentication: advanced/authentication.md - Info Command: advanced/explain_info_command.md