Unable to run tests against local build of Python

[jaraco]

Due to pypa/setuptools#836, I'm trying to make a local build of Python and test against that. By following the directions for compiling, I've build a local copy in ~/p/cpython/python.exe. In the setuptools checkout, I've updated the config to indicate basepython=/Users/jaraco/p/cpython/python.exe (I couldn't use ~ as it wouldn't expand that).

But now when I invoke tox, I get this error:

$ python -m tox 
GLOB sdist-make: /Users/jaraco/Dropbox/code/main/setuptools/setup.py
python create: /Users/jaraco/Dropbox/code/main/setuptools/.tox/python
ERROR: invocation failed (exit code 1), logfile: /Users/jaraco/Dropbox/code/main/setuptools/.tox/python/log/python-0.log
ERROR: actionid: python
msg: getenv
cmdargs: ['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', '-m', 'virtualenv', '--python', '/Users/jaraco/p/cpython/python.exe', 'python']
env: {'TMPDIR': '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/', 'TERM_PROGRAM_VERSION': '387', 'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.3Rh8N2IUMV/Render', 'LANG': 'en_US.UTF-8', 'TERM_PROGRAM': 'Apple_Terminal', 'XPC_SERVICE_NAME': '0', 'XPC_FLAGS': '0x0', 'TERM_SESSION_ID': '5EC107A7-3300-45D9-9CFC-829D5871A8A3', 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.Xg0hq8aXGE/Listeners', 'TERM': 'xterm-256color', '__CF_USER_TEXT_ENCODING': '0x1F5:0x0:0x0', 'SHELL': '/usr/local/bin/zsh', 'HOME': '/Users/jaraco', 'LOGNAME': 'jaraco', 'USER': 'jaraco', 'PATH': '/Users/jaraco/Dropbox/code/main/setuptools/.tox/python/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin', 'SHLVL': '1', 'PWD': '/Users/jaraco/m/setuptools', 'OLDPWD': '/Users/jaraco', 'VELOCIRAPTOR_AUTH_DOMAIN': 'YOUGOV.LOCAL', 'VELOCIRAPTOR_URL': 'https://deploy.yougov.net', 'CRUNCH_EMAIL': '[email protected]', 'SETUPTOOLS_SYS_PATH_TECHNIQUE': 'raw', 'FBURL': 'https://yougov.fogbugz.com', 'FBUSER': '[email protected]', 'YGAPP_YG_LDAP_PASSWORD': 'z8d8jxjs5CiCH5e', 'PYTHONHASHSEED': '997822322', 'NETSUITE_EMAIL': '[email protected]', 'WOLFRAMALPHA_API_KEY': 'Q59EW4-7K8AHE858R', 'LDFLAGS': '-L/usr/local/opt/openssl/lib', 'CFLAGS': '-I/usr/local/opt/openssl/include', '_': '/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', '__PYVENV_LAUNCHER__': '/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', 'VIRTUAL_ENV': '/Users/jaraco/Dropbox/code/main/setuptools/.tox/python'}

Using base prefix '/usr/local'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 2327, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 711, in main
    symlink=options.symlink)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 924, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1131, in install_python
    copy_required_modules(home_dir, symlink)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1061, in copy_required_modules
    dst_filename = change_prefix(filename, dst_prefix)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1035, in change_prefix
    (filename, prefixes)
AssertionError: Filename /Users/jaraco/p/cpython/Lib/os.py does not start with any of these prefixes: ['/Users/jaraco/Library/Python/3.6/lib/python/site-packages', '/Users/jaraco/.local/lib/python/3.6/site-packages', '/Users/jaraco/Library/Python/3.6/site-packages', '/Library/Python/3.6/site-packages', '/usr/local/Extras/lib/python', '/usr/local', '/usr/local']
Running virtualenv with interpreter /Users/jaraco/p/cpython/python.exe

ERROR: InvocationError: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 -m virtualenv --python /Users/jaraco/p/cpython/python.exe python (see /Users/jaraco/Dropbox/code/main/setuptools/.tox/python/log/python-0.log)
____________________________________________ summary _____________________________________________
ERROR:   python: InvocationError: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 -m virtualenv --python /Users/jaraco/p/cpython/python.exe python (see /Users/jaraco/Dropbox/code/main/setuptools/.tox/python/log/python-0.log)

I can replicate this error by simply trying to invoke virtualenv:

$ python -m virtualenv --python ~/p/cpython/python.exe foo
Running virtualenv with interpreter /Users/jaraco/p/cpython/python.exe
Using base prefix '/usr/local'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 2327, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 711, in main
    symlink=options.symlink)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 924, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1131, in install_python
    copy_required_modules(home_dir, symlink)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1061, in copy_required_modules
    dst_filename = change_prefix(filename, dst_prefix)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/virtualenv.py", line 1035, in change_prefix
    (filename, prefixes)
AssertionError: Filename /Users/jaraco/p/cpython/Lib/os.py does not start with any of these prefixes: ['/Users/jaraco/Library/Python/3.6/lib/python/site-packages', '/Users/jaraco/.local/lib/python/3.6/site-packages', '/Users/jaraco/Library/Python/3.6/site-packages', '/Library/Python/3.6/site-packages', '/usr/local/Extras/lib/python', '/usr/local', '/usr/local']

So it seems the issue is with virtualenv. The same issue doesn't exist with venv:

$ ~/p/cpython/python.exe -m venv foo

When setuptools used pytest-runner, this issue would have been avoided by directly using the local python environment to invoke pytest, but with tox, virtualenv is a requirement.

I guess I need to file a bug upstream, but I welcome any suggestions on how I might be able to work around the issue in tox.

[jaraco]

The issue's already been reported as pypa/virtualenv#456.

[obestwalter]

@jaraco Thanks for the report. I am triaging bugs atm and if I understand your problem correctly it's not an actual bug in tox, but a question how to work around a problem, until upstream is fixed? If I got it wrong I'll relabel.

[jaraco]

One way to think about it is that the dependence on virtualenv is a cause of the issue. If one were to use venv or rwt as a mechanism for loading dependencies, the issue would not be present. For example, I was able to work around this issue by not using tox:

$ ~/p/cpython/python.exe -m venv .tox/venv                                                 
$ .tox/venv/bin/python -m pip install -r tests/requirements.txt -q
$ .tox/venv/bin/python -m pytest -k test_bdist_egg
======================================= test session starts ========================================
platform darwin -- Python 3.6.0b2+, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/main/setuptools, inifile: pytest.ini
plugins: flake8-0.8.1, backports.unittest-mock-1.2
collected 196 items / 2 skipped 

setuptools/tests/test_bdist_egg.py .
setuptools/tests/test_easy_install.py .

======================================= 194 tests deselected =======================================
======================= 2 passed, 2 skipped, 194 deselected in 0.86 seconds ========================

or

$ ~/p/cpython/python.exe -m rwt -q -r tests/requirements.txt -- -m pytest -k test_bdist_egg
======================================= test session starts ========================================
platform darwin -- Python 3.6.0b2+, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/main/setuptools, inifile: pytest.ini
plugins: flake8-0.8.1, backports.unittest-mock-1.2
collected 196 items / 2 skipped 

setuptools/tests/test_bdist_egg.py .
setuptools/tests/test_easy_install.py .

======================================= 194 tests deselected =======================================
======================= 2 passed, 2 skipped, 194 deselected in 0.81 seconds ========================

So in that sense, it's more than just a question, but an issue that lies with tox, but for which the solution is almost certainly to be solved upstream. It's conceivable that tox will add an option to use venv or even prefer it on supported Python versions, which would bypass the issue where available.

If this were my project, I would want this ticket to remain an open ticket or to be closed but with a label indicating that it's pending an upstream change, with a slight preference for the former given that there are potentially some things tox could do to address the issue.

[obestwalter]

@jaraco Thank you for the clarification. We'll definitely leave this open until the issue is solved one way or the other.

[obestwalter]

@jaraco adding possibilities to use venv or rwt instead of virtualenv will likely be happening as part of #338.

[gaborbernat]

I think this now has been fixed in virtualenv, if not please raise an issue there.