RuntimeError

[remram44]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu Linux 20.04.1

• Poetry version: 1.1.0

• Link of a Gist with the contents of your pyproject.toml file: https://gitlab.com/ViDA-NYU/datamart/datamart/-/blob/1f15e3db009837cb51ee22274ce9bf3b96e205d7/pyproject.toml

Issue

Since the update, running poetry install crashes with a RuntimeError.

poetry install -vvv

Virtualenv datamart-TD2l9kg0-py3.8 already exists.
Using virtualenv: /home/remram/.cache/pypoetry/virtualenvs/datamart-TD2l9kg0-py3.8
Installing dependencies from lock file

Finding the necessary packages for the current system

  Stack trace:

  17  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  16  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  15  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│ 
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│ 
       173│     def __repr__(self):  # type: () -> str

  14  ~/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│ 
     →  92│         return self.handle()
        93│ 
        94│     def handle(self):  # type: () -> Optional[int]

  13  ~/.poetry/lib/poetry/console/commands/install.py:71 in handle
        69│         self._installer.verbose(self._io.is_verbose())
        70│ 
     →  71│         return_code = self._installer.run()
        72│ 
        73│         if return_code != 0:

  12  ~/.poetry/lib/poetry/installation/installer.py:99 in run
        97│         local_repo = Repository()
        98│ 
     →  99│         return self._do_install(local_repo)
       100│ 
       101│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  11  ~/.poetry/lib/poetry/installation/installer.py:278 in _do_install
       276│ 
       277│         with solver.use_environment(self._env):
     → 278│             ops = solver.solve(use_latest=whitelist)
       279│ 
       280│         # We need to filter operations so that packages

  10  ~/.poetry/lib/poetry/puzzle/solver.py:65 in solve
        63│         with self._provider.progress():
        64│             start = time.time()
     →  65│             packages, depths = self._solve(use_latest=use_latest)
        66│             end = time.time()
        67│ 

   9  ~/.poetry/lib/poetry/puzzle/solver.py:233 in _solve
       231│ 
       232│         try:
     → 233│             result = resolve_version(
       234│                 self._package, self._provider, locked=locked, use_latest=use_latest
       235│             )

   8  ~/.poetry/lib/poetry/mixology/__init__.py:7 in resolve_version
       5│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       6│ 
     → 7│     return solver.solve()
       8│ 

   7  ~/.poetry/lib/poetry/mixology/version_solver.py:84 in solve
        82│             while next is not None:
        83│                 self._propagate(next)
     →  84│                 next = self._choose_package_version()
        85│ 
        86│             return self._result()

   6  ~/.poetry/lib/poetry/mixology/version_solver.py:397 in _choose_package_version
       395│             version = locked
       396│ 
     → 397│         version = self._provider.complete_package(version)
       398│ 
       399│         conflict = False

   5  ~/.poetry/lib/poetry/puzzle/provider.py:447 in complete_package
       445│             for r in requires:
       446│                 if r.is_directory():
     → 447│                     self.search_for_directory(r)
       448│                 elif r.is_file():
       449│                     self.search_for_file(r)

   4  ~/.poetry/lib/poetry/puzzle/provider.py:264 in search_for_directory
       262│             package = _package.clone()
       263│         else:
     → 264│             package = self.get_package_from_directory(
       265│                 dependency.full_path, name=dependency.name
       266│             )

   3  ~/.poetry/lib/poetry/puzzle/provider.py:284 in get_package_from_directory
       282│         cls, directory, name=None
       283│     ):  # type: (Path, Optional[str]) -> Package
     → 284│         package = PackageInfo.from_directory(path=directory).to_package(
       285│             root_dir=directory
       286│         )

   2  ~/.poetry/lib/poetry/inspection/info.py:527 in from_directory
       525│                         info = cls.from_setup_files(path)
       526│                     else:
     → 527│                         info = cls._pep517_metadata(path)
       528│                 except PackageInfoError:
       529│                     if not info:

   1  ~/.poetry/lib/poetry/inspection/info.py:492 in _pep517_metadata
       490│                     )
       491│                 finally:
     → 492│                     os.chdir(cwd.as_posix())
       493│ 
       494│         if info:

  RuntimeError

  generator didn't stop after throw()

  at /usr/lib/python3.8/contextlib.py:162 in __exit__
      158│                 # by 'except BaseException'.
      159│                 if sys.exc_info()[1] is value:
      160│                     return False
      161│                 raise
    → 162│             raise RuntimeError("generator didn't stop after throw()")
      163│ 
      164│ 
      165│ class _AsyncGeneratorContextManager(_GeneratorContextManagerBase,
      166│                                     AbstractAsyncContextManager):

[abn]

Unable to reproduce this issue.

$ docker run --rm -i --entrypoint bash python:3.8 <<EOF
> python -m pip install -q poetry==1.1.0
> git clone --recurse-submodules https://gitlab.com/ViDA-NYU/datamart/datamart.git
> pushd datamart
> python -m poetry install
> EOF
Cloning into 'datamart'...
Submodule 'lib_geo' (https://gitlab.com/ViDA-NYU/datamart/datamart-geo.git) registered for path 'lib_geo'
Cloning into '/datamart/lib_geo'...
Submodule path 'lib_geo': checked out '30bd3db65dd9749261bbfffe04040e2632d6446b'
/datamart /
Creating virtualenv datamart-3NPA7JLM-py3.8 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 76 installs, 0 updates, 0 removals

  • Installing six (1.15.0)
  • Installing idna (2.10)
  • Installing multidict (4.7.6)
  • Installing numpy (1.19.0)
  • Installing python-dateutil (2.8.1)
  • Installing pytz (2020.1)
  • Installing certifi (2020.6.20)
  • Installing chardet (3.0.4)
  • Installing grpcio (1.30.0)
  • Installing joblib (0.16.0)
  • Installing pamqp (2.3.0)
  • Installing pandas (1.0.5)
  • Installing scipy (1.5.1)
  • Installing protobuf (3.12.2)
  • Installing urllib3 (1.25.9)
  • Installing threadpoolctl (2.1.0)
  • Installing yarl (1.4.2)
Connection pool is full, discarding connection: pypi.org
  • Installing aiormq (3.2.2)
  • Installing datamart-geo (0.1 /datamart/lib_geo)
  • Installing lazo-index-service (0.5.1)
  • Installing markupsafe (1.1.1)
  • Installing prometheus-client (0.8.0)
  • Installing pyparsing (2.4.7)
  • Installing pyreadstat (1.0.0)
  • Installing regex (2020.7.14)
  • Installing requests (2.24.0)
  • Installing scikit-learn (0.23.1)
  • Installing wrapt (1.12.1)
  • Installing xlrd (1.2.0)
Connection pool is full, discarding connection: pypi.org
  • Installing aio-pika (6.6.1)
  • Installing alabaster (0.7.12)
  • Installing babel (2.8.0)
  • Installing datamart-materialize (0.7.1 /datamart/lib_materialize)
  • Installing datamart-profiler (0.7.1 /datamart/lib_profiler)
  • Installing docutils (0.16)
  • Installing elasticsearch (7.8.1)
  • Installing imagesize (1.2.0)
  • Installing jinja2 (2.11.2)
  • Installing packaging (20.4)
  • Installing pygments (2.6.1)
  • Installing prometheus-async (19.2.0)
  • Installing sentry-sdk (0.17.4)
  • Installing snowballstemmer (2.0.0)
  • Installing sphinxcontrib-applehelp (1.0.2)
  • Installing sphinxcontrib-devhelp (1.0.2)
  • Installing sphinxcontrib-htmlhelp (1.0.3)
  • Installing sphinxcontrib-jsmath (1.0.1)
  • Installing sphinxcontrib-qthelp (1.0.3)
  • Installing sphinxcontrib-serializinghtml (1.1.4)
  • Installing webencodings (0.5.1)
  • Installing attrs (19.3.0)
  • Installing bleach (3.1.5)
  • Installing datamart-augmentation (0.7.1 /datamart/lib_augmentation)
  • Installing datamart-core (0.0 /datamart/lib_core)
  • Installing defusedxml (0.6.0)
  • Installing mccabe (0.6.1)
  • Installing pycodestyle (2.6.0)
  • Installing pyflakes (2.2.0)
  • Installing pyrsistent (0.16.0)
  • Installing pyyaml (5.3.1)
  • Installing redis (3.5.3)
  • Installing sodapy (2.1.0)
  • Installing sphinx (3.1.2)
  • Installing stringdist (1.0.9)
  • Installing tornado (6.0.4)
  • Installing coverage (5.2)
  • Installing datamart-api-service (0.0 /datamart/apiserver)
  • Installing datamart-coordinator-service (0.0 /datamart/coordinator)
  • Installing datamart-noaa-discovery-service (0.0 /datamart/discovery/noaa)
  • Installing datamart-profiler-service (0.0 /datamart/profiler)
  • Installing datamart-zenodo-discovery-service (0.0 /datamart/discovery/zenodo)
  • Installing flake8 (3.8.3)
  • Installing datamart-socrata-discovery-service (0.0 /datamart/discovery/socrata)
  • Installing jsonschema (3.2.0)
  • Installing sphinx-rtd-theme (0.5.0)
  • Installing readme-renderer (26.0)

[remram44]

Yeah I am unable to reproduce it in Docker either, even on Python 3.8. However it happens consistently on my laptop. I even reinstalled Poetry.

What kind of information would help troubleshooting this? This is not blocking for me since I have to use 1.0.10 anyway, but I thought I'd report it.

[abn]

Ubuntu distros typically do have issues with virtualenv. You can try maybe installing poetry using pipx. Or try pip install --upgrade virtualenv pip if you installed poetry using pip.

[remram44]

I tried to install on a fresh Ubuntu 20.04 but poetry install fails with ModuleNotFoundError: No module named 'distutils.util' (#721? #1837?)

[abn]

IIRC, ubuntu image comes with a non-standard installation of distutils. That is more a distro isssue that anything to do with poetry.

As for information, from what I can tell the issue reported has to do with an exception happening within a context manager, we could harden the handling but the root issue will still persist.

[remram44]

Re-did the whole thing on fresh Ubuntu 20.04, making sure to not install python3 but python2.7. Then installing python3 (which my project needs) once Poetry is set up.

I get warnings when running Poetry, and still can't install:

$ poetry install -vvv
/home/user/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
  "program uses threads.", RuntimeWarning)

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

The currently activated Python version 2.7.18 is not supported by the project (^3.7).
Trying to find and use a compatible version. 
Trying python3
Using python3 (3.8.2)
Creating virtualenv datamart-sRH03EIY-py3.8 in /home/user/.cache/pypoetry/virtualenvs

RuntimeError

failed to find interpreter for Builtin discover of python_spec='python3'

Traceback (most recent call last):
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/clikit/api/command/command.py", line 163, in _do_handle
    self._dispatcher.dispatch(PRE_HANDLE, event)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/clikit/api/event/event_dispatcher.py", line 22, in dispatch
    self._do_dispatch(listeners, event_name, event)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/clikit/api/event/event_dispatcher.py", line 89, in _do_dispatch
    listener(event, event_name, self)
  File "/home/user/.poetry/lib/poetry/console/config/application_config.py", line 119, in set_env
    env = env_manager.create_venv(io)
  File "/home/user/.poetry/lib/poetry/utils/env.py", line 645, in create_venv
    self.build_venv(venv, executable=executable)
  File "/home/user/.poetry/lib/poetry/utils/env.py", line 692, in build_venv
    str(path),
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/virtualenv/run/__init__.py", line 26, in cli_run
    of_session = session_via_cli(args, options, setup_logging)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/virtualenv/run/__init__.py", line 42, in session_via_cli
    parser, elements = build_parser(args, options, setup_logging)
  File "/home/user/.poetry/lib/poetry/_vendor/py2.7/virtualenv/run/__init__.py", line 66, in build_parser
    raise RuntimeError('failed to find interpreter for {}'.format(discover))

[abn]

1. How did you install poetry?
2. Looks like virtualenv cannot discover python3 binary.

[remram44]

Not finding python3 seems to be the same distutils issue, using poetry env with a full path to the Python binary still fails.

[remram44]

Seems those issues were related! I found a problem in the implementation of a context manager and filed #3031. It will fix the RuntimeError, but not the distutils issues.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.