Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poetry can't run scripts from dependencies (which themselves use poetry) installed as editable on Windows #3265

Closed
3 tasks done
kevincon opened this issue Oct 22, 2020 · 5 comments · Fixed by #3339
Closed
3 tasks done
Assignees
Labels
kind/bug Something isn't working as expected

Comments

@kevincon
Copy link
Contributor

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

Issue

My team is experiencing that poetry run is unable to run scripts registered with poetry by editable-installed dependencies of our main poetry project on Windows.

I've created this repo to provide a minimal project to demonstrate this issue: https://github.com/kevincon/poetry_editable_dep_script_bug_repro

Also I'm using Python 3.8.3 32-bit.

In that repo, the main project, parent , has a local, editable path dependency of child which is itself a Poetry project.

Both the parent project and the child project register a script using the tool.poetry.scripts section in their respective pyproject.toml files. All these scripts do is print out Hello from parent CLI! or Hello from child CLI!, respectively.

Installing the parent project is successful:

C:\Users\kevincon\repos\poetry_editable_dep_script_bug_repro 
λ poetry install
Creating virtualenv parent-MPYm-h1y-py3.8 in C:\Users\kevincon\AppData\Local\pypoetry\Cache\virtualenvs
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing child (0.1.0 C:/Users/kevincon/repos/poetry_editable_dep_script_bug_repro/child)

Installing the current project: parent (0.1.0)

As is running the parent project's script:

C:\Users\kevincon\repos\poetry_editable_dep_script_bug_repro 
λ poetry run parent-cli
Hello from parent CLI!

But trying to run the child project's script fails with this error (with 100% reproducibility):

C:\Users\kevincon\repos\poetry_editable_dep_script_bug_repro 
λ poetry run child-cli

  FileNotFoundError

  [WinError 2] The system cannot find the file specified

  at ~\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py:1307 in _execute_child
      1303│             sys.audit("subprocess.Popen", executable, args, cwd, env)
      1304│
      1305│             # Start the process
      1306│             try:
    → 1307│                 hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
      1308│                                          # no special security
      1309│                                          None, None,
      1310│                                          int(not close_fds),
      1311│                                          creationflags,

(we would expect it to not throw an error and instead print out Hello from child CLI!)

However, we can successfully run both the parent and child project scripts if we use poetry version 1.0.10.

I think what changed to break this behavior between 1.0.10 and 1.1.x is #2360, which seems to have changed how scripts of editable projects are installed. Previously a .exe file was created for scripts registered with poetry in the Scripts/ directory of the poetry-managed virtual environment (I think by pip?), but now a .cmd file is created by poetry for these scripts instead.

I think I understand that the parent project's script is run successfully in poetry 1.1.3 because poetry finds it in the parent project's pyproject.toml file and takes this code path:

if scripts and script in scripts:
return self.run_script(scripts[script], args)

But since the child project's script won't be found in the parent project's pyproject.toml file, it takes a different code path for the child script which ultimately tries to use subprocess.Popen() to call the child script:

exe = subprocess.Popen([bin] + list(args), **kwargs)

Unfortunately it looks like the subprocess.Popen call which poetry uses to try to run the child script ends up calling _winapi.CreateProcess() (as shown in the error output above) which I learned does not auto-resolve any other executable extensions besides .exe, so it doesn't find the .cmd executable for the child script. See https://stackoverflow.com/a/10555130 and https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa?redirectedfrom=MSDN#parameters.

This seems to be supported by the fact that we are able to run the child project's script using poetry 1.1.3 if we add the .cmd extension ourselves:

C:\Users\kevincon\repos\poetry_editable_dep_script_bug_repro 
λ poetry run child-cli.cmd
Hello from child CLI!
@kevincon kevincon added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Oct 22, 2020
@finswimmer
Copy link
Member

Hello @kevincon,

thanks a lot for reporting and your detailed investigation 👍 Do you think, you can provide a PR to fix it? :)

fin swimmer

@avirlrma
Copy link

avirlrma commented Nov 1, 2020

Hey,

I would like to work on this. Can I get assigned?

Thanks.

@kevincon
Copy link
Contributor Author

kevincon commented Nov 1, 2020

Ah sorry @avirlrma I forgot to call it out, but I have fixed this in a branch and I think I can put up a PR for it tomorrow. Do you mind if I finish it up?

@avirlrma
Copy link

avirlrma commented Nov 1, 2020

Yes sure, please go ahead.

Copy link

github-actions bot commented Mar 1, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected
Projects
None yet
4 participants