diff --git a/source/specifications/direct-url.rst b/source/specifications/direct-url.rst index fc00fa132..7a1560e87 100644 --- a/source/specifications/direct-url.rst +++ b/source/specifications/direct-url.rst @@ -6,8 +6,9 @@ Recording the Direct URL Origin of installed distributions ========================================================== This document specifies a :file:`direct_url.json` file in the -:file:`*.dist-info` directory of an installed distribution, to record the -Direct URL Origin of the distribution. +``*.dist-info`` directory of an installed distribution, to record the +Direct URL Origin of the distribution. The general structure and usage of +``*.dist-info`` directories is described in :ref:`recording-installed-packages`. .. contents:: Contents :local: diff --git a/source/specifications/entry-points.rst b/source/specifications/entry-points.rst index 6c94a25fb..a11f3ca93 100644 --- a/source/specifications/entry-points.rst +++ b/source/specifications/entry-points.rst @@ -81,8 +81,9 @@ File format Entry points are defined in a file called :file:`entry_points.txt` in the :file:`*.dist-info` directory of the distribution. This is the directory -described in :pep:`376` for installed distributions, and in :pep:`427` for -wheels. The file uses the UTF-8 character encoding. +described in :ref:`recording-installed-packages` for installed distributions, +and in :ref:`binary-distribution-format` for wheels. +The file uses the UTF-8 character encoding. The file contents are in INI format, as read by Python's :mod:`configparser` module. However, configparser treats names as case-insensitive by default, diff --git a/source/specifications/externally-managed-environments.rst b/source/specifications/externally-managed-environments.rst new file mode 100644 index 000000000..3876d8e34 --- /dev/null +++ b/source/specifications/externally-managed-environments.rst @@ -0,0 +1,22 @@ + +.. _externally-managed-environments: + +=============================== +Externally Managed Environments +=============================== + +While some Python installations are entirely managed by the user that installed +Python, others may be provided and managed by another means (such as the +operating system package manager in a Linux distribution, or as a bundled +Python environment in an application with a dedicated installer). + +Attempting to use conventional Python packaging tools to manipulate such +environments can be confusing at best and outright break the entire underlying +operating system at worst. Documentation and interoperability guides only go +so far in resolving such problems. + +:pep:`668` defined an ``EXTERNALLY-MANAGED`` marker file that allows a Python +installation to indicate to Python-specific tools such as ``pip`` that they +neither install nor remove packages into the interpreter’s default installation +environment, and should instead guide the end user towards using +:ref:`virtual-environments`. diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 3265ce56f..f4a290e29 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -22,10 +22,18 @@ Package Distribution Metadata declaring-build-dependencies declaring-project-metadata platform-compatibility-tags + +Package Installation Environment Metadata +----------------------------------------- + +.. toctree:: + :maxdepth: 1 + recording-installed-packages entry-points direct-url - + virtual-environments + externally-managed-environments Package Distribution File Formats --------------------------------- diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index 6a01544d5..bac887b2b 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -21,12 +21,14 @@ History and change workflow =========================== The metadata described here was first specified in :pep:`376`, and later -amended in :pep:`627`. +amended in :pep:`627` (and other PEPs). It was formerly known as *Database of Installed Python Distributions*. -Further amendments (except trivial language or typography fixes) must be made -through the PEP process (see :pep:`1`). +As with other PyPA specifications, editorial amendments with no functional +impact may be made through the GitHub pull request workflow. Proposals for +functional changes that would require amendments to package building and/or +installation tools must be made through the PEP process (see :pep:`1`). -While this document is the normative specification, these PEPs that introduce +While this document is the normative specification, the PEPs that introduce changes to it may include additional information such as rationales and backwards compatibility considerations. @@ -66,12 +68,14 @@ encouraged to start normalizing those fields. provide API for such tools to consume, so tools can have access to the unnormalized name when displaying distrubution information. -This ``.dist-info`` directory can contain these files, described in detail -below: +This ``.dist-info`` directory may contain the following files, described in +detail below: * ``METADATA``: contains project metadata * ``RECORD``: records the list of installed files. * ``INSTALLER``: records the name of the tool used to install the project. +* ``entry_points.txt``: see :ref:`entry-points` for details +* ``direct_url.json``: see :ref:`direct-url` for details The ``METADATA`` file is mandatory. All other files may be omitted at the installing tool's discretion. @@ -169,7 +173,7 @@ Here is an example snippet of a possible ``RECORD`` file:: If the ``RECORD`` file is missing, tools that rely on ``.dist-info`` must not attempt to uninstall or upgrade the package. -(This does not apply to tools that rely on other sources of information, +(This restriction does not apply to tools that rely on other sources of information, such as system package managers in Linux distros.) @@ -197,6 +201,18 @@ For example, if a tool is asked to uninstall a project but finds no ``RECORD`` file, it may suggest that the tool named in ``INSTALLER`` may be able to do the uninstallation. + +The entry_points.txt file +========================= + +This file MAY be created by installers to indicate when packages contain +components intended for discovery and use by other code, including console +scripts and other applications that the installer has made available for +execution. + +Its detailed specification is at :ref:`entry-points`. + + The direct_url.json file ======================== @@ -207,3 +223,32 @@ This file MUST NOT be created when installing a distribution from an other type of requirement (i.e. name plus version specifier). Its detailed specification is at :ref:`direct-url`. + + +Intentionally preventing changes to installed packages +====================================================== + +In some cases (such as when needing to manage external dependencies in addition +to Python ecosystem dependencies), it is desirable for a tool that installs +packages into a Python environment to ensure that other tools are not used to +uninstall or otherwise modify that installed package, as doing so may cause +compatibility problems with the wider environment. + +To achieve this, affected tools should take the folllowing steps: + +* Rename or remove the ``RECORD`` file to prevent changes via other tools (e.g. + appending a suffix to create a non-standard ``RECORD.tool`` file if the tool + itself needs the information, or omitting the file entirely if the package + contents are tracked and managed via other means) +* Write an ``INSTALLER`` file indicating the name of the tool that should be used + to manage the package (this allows ``RECORD``-aware tools to provide better + error notices when asked to modify affected packages) + +Python runtime providers may also prevent inadvertent modification of platform +provided packages by modifying the default Python package installation scheme +to use a location other than that used by platform provided packages (while also +ensuring both locations appear on the default Python import path). + +In some circumstances, it may be desirable to block even installation of +additional packages via Python-specific tools. For these cases refer to +:ref:`externally-managed-environments` \ No newline at end of file diff --git a/source/specifications/virtual-environments.rst b/source/specifications/virtual-environments.rst new file mode 100644 index 000000000..c38c2faeb --- /dev/null +++ b/source/specifications/virtual-environments.rst @@ -0,0 +1,50 @@ + +.. _virtual-environments: + +=========================== +Python Virtual Environments +=========================== + +For Python 3.3 and later versions, :pep:`405` introduced interpreter level support +for the concept of "Python Virtual Environments". Each virtual environment has +its own Python binary (allowing creation of environments with various Python +versions) and can have its own independent set of installed Python packages in +its site directories, but shares the standard library with the base installed +Python. While the concept of virtual environments existed prior to this update, +there was no previously standardised mechanism for declaring or discovering them. + + +Runtime detection of virtual environments +========================================= + +At runtime, virtual environments can be identified by virtue of ``sys.prefix`` +(the filesystem location of the running interpreter) having a different value +from ``sys.base_prefix`` (the default filesytem location of the standard library +directories). + +:ref:`venv-explanation` in the Python standard library documentation for the +:py:mod:`venv` module covers this along with the concept of "activating" a +virtual environment in an interactive operating system shell (this activation +step is optional and hence the changes it makes can't be reliably used to +detect whether a Python program is running in a virtual environment or not). + + +Declaring installation environments as Python virtual environments +================================================================== + +As described in :pep:`405`, a Python virtual environment in its simplest form +consists of nothing more than a copy or symlink of the Python binary accompanied +by a ``site-packages`` directory and a ``pyvenv.cfg`` file with a ``home`` key +that indicates where to find the Python standard library modules. + +While designed to meet the needs of the standard :py:mod:`venv` module, this +split installation and ``pyvenv.cfg`` file approach can be used by *any* +Python installation provider that desires Python-specific tools to be aware that +they are already operating in a virtual environment and no further environment +nesting is required or desired. + +Even in the absence of a ``pyvenv.cfg`` file, any approach (e.g. +``sitecustomize.py``, patching the installed Python runtime) that results in +``sys.prefix`` and ``sys.base_prefix`` having different values, while still +providing a matching default package installation scheme in ``sysconfig``, will +be detected and behave as a Python virtual environment.