Skip to content

Commit

Permalink
Allow direct execution (#41)
Browse files Browse the repository at this point in the history
Also updates the documentation and installation suggestions

Closes #26
  • Loading branch information
ncoghlan authored Oct 23, 2024
1 parent bb1c7e9 commit c43733b
Show file tree
Hide file tree
Showing 7 changed files with 78 additions and 19 deletions.
4 changes: 2 additions & 2 deletions docs/development/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ venvstacks can then be executed via the ``-m`` switch:

.. code-block:: console
$ .venv/bin/python -m venvstacks --help
$ .venv/bin/venvstacks --help
Usage: python -m venvstacks [OPTIONS] COMMAND [ARGS]...
Usage: venvstacks [OPTIONS] COMMAND [ARGS]...
Lock, build, and publish Python virtual environment stacks.
Expand Down
21 changes: 14 additions & 7 deletions docs/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,14 @@ Installing
----------

``venvstacks`` is available from the :pypi:`Python Package Index <venvstacks>`,
and can be installed with :pypi:`pip`:
and can be installed with :pypi:`pipx` (or similar tools):

.. code-block:: console
$ pipx install venvstacks
Alternatively, it can be installed as a user level package (although this may
make future Python version upgrades more irritating):

.. code-block:: console
Expand All @@ -27,9 +34,9 @@ The command line help also provides additional usage information:

.. code-block:: console
$ .venv/bin/python -m venvstacks --help
$ venvstacks --help
Usage: python -m venvstacks [OPTIONS] COMMAND [ARGS]...
Usage: venvstacks [OPTIONS] COMMAND [ARGS]...
Lock, build, and publish Python virtual environment stacks.
Expand Down Expand Up @@ -100,7 +107,7 @@ Locking environment stacks

.. code-block:: console
$ python -m venvstacks lock sklearn_demo/venvstacks.toml
$ venvstacks lock sklearn_demo/venvstacks.toml
The ``lock`` subcommand takes the defined layer requirements from the specification,
and uses them to perform a complete combined resolution of all of the environment stacks
Expand All @@ -116,7 +123,7 @@ Building environment stacks

.. code-block:: console
$ python -m venvstacks build sklearn_demo/venvstacks.toml
$ venvstacks build sklearn_demo/venvstacks.toml
The ``build`` subcommand performs the step of converting the layer specifications
and their locked requirements into a working Python environment
Expand All @@ -133,7 +140,7 @@ Publishing environment layer archives

.. code-block:: console
$ python -m venvstacks publish --tag-outputs --output-dir demo_artifacts sklearn_demo/venvstacks.toml
$ venvstacks publish --tag-outputs --output-dir demo_artifacts sklearn_demo/venvstacks.toml
Once the environments have been successfully built,
the ``publish`` command allows each layer to be converted to a separate
Expand All @@ -154,7 +161,7 @@ Locally exporting environment stacks

.. code-block:: console
$ python -m venvstacks local-export --output-dir demo_export sklearn_demo/venvstacks.toml
$ venvstacks local-export --output-dir demo_export sklearn_demo/venvstacks.toml
Given that even considering the use of ``venvstacks`` implies that some layer archives may be of
significant size (a fully built `pytorch` archive weighs in at multiple gigabytes, for example),
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ rich-cli = [
"typer>=0.12.4",
]

[project.scripts]
venvstacks = "venvstacks.cli:main"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
Expand Down
15 changes: 8 additions & 7 deletions src/venvstacks/_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

from contextlib import contextmanager
from pathlib import Path
from typing import Any, Generator
from typing import Any, Generator, Mapping

WINDOWS_BUILD = hasattr(os, "add_dll_directory")

Expand Down Expand Up @@ -78,23 +78,24 @@ def get_env_python(env_path: Path) -> Path:


def run_python_command_unchecked(
# Narrow list/dict type specs here due to the way `subprocess.run` params are typed
# Narrow list type spec here due to the way `subprocess.run` params are typed
command: list[str],
*,
env: dict[str, str] | None = None,
env: Mapping[str, str] | None = None,
**kwds: Any,
) -> subprocess.CompletedProcess[str]:
if env is None:
env = os.environ.copy()
env.update(_SUBPROCESS_PYTHON_CONFIG)
run_env = os.environ.copy()
if env is not None:
run_env.update(env)
run_env.update(_SUBPROCESS_PYTHON_CONFIG)
result: subprocess.CompletedProcess[str] = subprocess.run(
command, env=env, text=True, **kwds
)
return result


def run_python_command(
# Narrow list/dict type specs here due to the way `subprocess.run` params are typed
# Narrow list type spec here due to the way `subprocess.run` params are typed
command: list[str],
**kwds: Any,
) -> subprocess.CompletedProcess[str]:
Expand Down
23 changes: 22 additions & 1 deletion src/venvstacks/cli.py
Original file line number Diff line number Diff line change
@@ -1,21 +1,42 @@
"""Command line interface implementation"""

import os.path
import sys

from typing import Annotated

import typer

from .stacks import StackSpec, BuildEnvironment, _format_json, IndexConfig

# Inspired by the Python 3.13+ `argparse` feature,
# but reports `python -m venvstacks` whenever `__main__`
# refers to something other than the entry point script,
# rather than trying to infer anything from the main
# module's `__spec__` attributes.
_THIS_PACKAGE = __spec__.parent


def _get_usage_name() -> str:
exec_name = os.path.basename(sys.argv[0]).removesuffix(".exe")
if exec_name == _THIS_PACKAGE:
# Entry point wrapper, suggest direct execution
return exec_name
# Could be `python -m`, could be the test suite,
# could be something else calling `venvstacks.cli.main`,
# but treat it as `python -m venvstacks` regardless
py_name = os.path.basename(sys.executable).removesuffix(".exe")
return f"{py_name} -m {_THIS_PACKAGE}"


_cli = typer.Typer(
add_completion=False,
pretty_exceptions_show_locals=False,
no_args_is_help=True,
)


@_cli.callback(name="python -m venvstacks")
@_cli.callback(name=_get_usage_name())
def handle_app_options() -> None:
"""Lock, build, and publish Python virtual environment stacks."""
# TODO: Handle app logging config via main command level options
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Updating metadata and examining built artifacts
To generate a full local sample project build to help debug failures:

$ cd /path/to/repo/
$ pdm run python -m venvstacks build --publish \
$ pdm run venvstacks build --publish \
tests/sample_project/venvstacks.toml ~/path/to/output/folder

This assumes `pdm sync --dev` has been used to set up a local development venv.
Expand Down
29 changes: 28 additions & 1 deletion tests/test_cli_invocation.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
"""Test cases for CLI invocation"""

import subprocess
import sys

from contextlib import contextmanager
from dataclasses import dataclass, field
from pathlib import Path
Expand All @@ -17,6 +20,7 @@

from venvstacks import cli
from venvstacks.stacks import BuildEnvironment, EnvironmentLock, IndexConfig
from venvstacks._util import run_python_command_unchecked


def report_traceback(exc: BaseException | None) -> str:
Expand Down Expand Up @@ -128,6 +132,8 @@ def mocked_runner() -> Generator[MockedRunner, None, None]:
class TestTopLevelCommand:
def test_implicit_help(self, mocked_runner: MockedRunner) -> None:
result = mocked_runner.invoke([])
# Usage message should suggest indirect execution
assert "Usage: python -m venvstacks [" in result.stdout
# Top-level callback docstring is used as the overall CLI help text
cli_help = cli.handle_app_options.__doc__
assert cli_help is not None
Expand All @@ -142,8 +148,29 @@ def test_implicit_help(self, mocked_runner: MockedRunner) -> None:
assert result.exception is None, report_traceback(result.exception)
assert result.exit_code == 0

# See https://github.com/lmstudio-ai/venvstacks/issues/42
@pytest.mark.xfail(
sys.platform == "win32", reason="UnicodeDecodeError parsing output"
)
def test_entry_point_help(self) -> None:
if sys.prefix == sys.base_prefix:
pytest.skip("Entry point test requires test execution in venv")
expected_entry_point = Path(sys.executable).parent / "venvstacks"
command = [str(expected_entry_point), "--help"]
result = run_python_command_unchecked(
command, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
# Usage message should suggest direct execution
assert "Usage: venvstacks [" in result.stdout
# Top-level callback docstring is used as the overall CLI help text
cli_help = cli.handle_app_options.__doc__
assert cli_help is not None
assert cli_help.strip() in result.stdout
# Check operation result last to ensure test results are as informative as possible
assert result.returncode == 0


EXPECTED_USAGE_PREFIX = "Usage: python -m venvstacks"
EXPECTED_USAGE_PREFIX = "Usage: python -m venvstacks "
EXPECTED_SUBCOMMANDS = ["lock", "build", "local-export", "publish"]
NO_SPEC_PATH: list[str] = []
NEEDS_SPEC_PATH = sorted(set(EXPECTED_SUBCOMMANDS) - set(NO_SPEC_PATH))
Expand Down

0 comments on commit c43733b

Please sign in to comment.