-
Notifications
You must be signed in to change notification settings - Fork 664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ImportError: No module named six #683
Comments
Looks like you're installing from source and don't use PEP517. Best guess: upgrade to the latest Pip and try again. |
Or maybe s/pip/virtualenv/ as it's where that pip is coming from. |
I was able to avoid it but forcing pre-commit to use python3. With python2 there is no chance to make it work, mainly because there is no way to tell pre-commit to upgrade packaging tools before installing it. In newer versions it also dropped support for py2. These being said, maybe we should enforce python version in hook definition, something similar with what I did at https://review.rdoproject.org/r/#/c/25278/1/.pre-commit-config.yaml |
OTOH, I'm seeing an isolated venv being created in the part of log you didn't copy-paste here: 2020-02-24 09:13:28.946269 | rdo-centos-7 | ERROR: Command errored out with exit status 2:
2020-02-24 09:13:28.946968 | rdo-centos-7 | command: /home/zuul/.cache/pre-commit/reporf1mbW/py_env-python2.7/bin/python /home/zuul/.cache/pre-commit/reporf1mbW/py_env-python2.7/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-neprNd/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i http://mirror.regionone.rdo-cloud.rdoproject.org/pypi/simple --extra-index-url http://mirror.regionone.rdo-cloud.rdoproject.org/wheel/centos-7.7-x86_64 --trusted-host mirror.regionone.rdo-cloud.rdoproject.org -- 'setuptools >= 41.4.0' 'setuptools_scm >= 1.15.0' 'setuptools_scm_git_archive >= 1.0' wheel ansible 'pathlib2; python_version < '"'"'3.2'"'"'' ruamel.yaml six
[...]
2020-02-24 09:13:28.949487 | rdo-centos-7 | Collecting six
2020-02-24 09:13:28.949724 | rdo-centos-7 | Downloading http://mirror.regionone.rdo-cloud.rdoproject.org/pypifiles/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl (10 kB)
[...]
2020-02-24 09:13:28.954259 | rdo-centos-7 | Installing collected packages: setuptools, setuptools-scm, setuptools-scm-git-archive, wheel, MarkupSafe, jinja2, PyYAML, six, ipaddress, pycparser, cffi, enum34, cryptography, ansible, scandir, pathlib2, ruamel.ordereddict, ruamel.yaml.clib, ruamel.yaml
So it installs things from the build-system requires coming from https://github.com/ansible/ansible-lint/blob/9b1c30b/pyproject.toml#L14 and this part looks fine. From the traceback, I'd guess that setuptools is hitting this https://github.com/ansible/ansible-lint/blob/9b1c30b/setup.cfg#L24. Then, it tries to import it. But to import The question is why it seems to evaluate
|
I think, that you should upgrade packaging tools in the same env that |
I tried upgrading during install but apparently it had no effect on pre-commit but my impression is that somehow pre-commit manages to not use these. Anyway, bump to py3 is ok for me, fixing more than one issue with it. |
This now affects all our CIs. |
I believe this may be linked to the “innovative” packaging approach taken by ansible-lint, because I did not see any similar failure to install a package from source. Pre-commit already droppped support for py2, so I suppose that patching it will be another hard sell. Better to fix ansible-lint. |
I'm pretty sure it's related to a pip resolver's bug in connection w/ the new virtualenv. |
I released a patched version at https://pypi.org/project/ansible-lint-fixed/ which is supposed to workaround this issue. This was done out of https://github.com/ssbarnea/ansible-lint fork. |
@ssbarnea that fork should probably link your repo and have an explanation in the description on PyPI. |
@ssbarnea what is the version of python/pip/virtualenv/setuptools where you hit this error? |
I was unable to reproduce it under a clean CentOS 7 container with Python 2.7, Pip 20.0.2, setuptools 44.0.0 and virtualenv 20.0.7 (w/o tox). |
So it's reproducible w/ tox. |
|
Seems to be a bug of tox's isolated_build. |
This can be reproduced by running: $ /ansible-lint/.tox/.package/bin/python -m pip install 'setuptools >= 41.4.0' 'setuptools_scm >= 1.15.0' 'setuptools_scm_git_archive >= 1.0' wheel ansible 'pathlib2; python_version < '"'"'3.2'"'"'' ruamel.yaml six |
Interestingly, this is only reproducible within virtualenv only if cwd is ansible-lint repo: [root@8f1f6759dd38 ansible-lint]# /ansible-lint/.tox/.package/bin/python -m pip install setuptools
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: setuptools in ./.tox/.package/lib/python2.7/site-packages (44.0.0)
ERROR: Exception:
Traceback (most recent call last):
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 412, in run
isolated=options.isolated_mode,
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 543, in get_lib_location_guesses
scheme = distutils_scheme('', *args, **kwargs)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/pip/_internal/locations.py", line 109, in distutils_scheme
d.parse_config_files()
File "/ansible-lint/.tox/.package/lib64/python2.7/site.py", line 153, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/ansible-lint/.tox/.package/lib64/python2.7/site.py", line 153, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/_distutils_patch_virtualenv.py", line 19, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/dist.py", line 702, in parse_config_files
ignore_option_errors=ignore_option_errors)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 121, in parse_configuration
meta.parse()
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 426, in parse
section_parser_method(section_options)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 399, in parse_section
self[name] = value
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 184, in __setitem__
value = parser(value)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 515, in _parse_version
version = self._parse_attr(value, self.package_dir)
File "/ansible-lint/.tox/.package/lib/python2.7/site-packages/setuptools/config.py", line 349, in _parse_attr
module = import_module(module_name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/ansible-lint/lib/ansiblelint/__init__.py", line 27, in <module>
import six
ImportError: No module named six
[root@8f1f6759dd38 ansible-lint]# python -m pip install setuptools
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (44.0.0)
[root@8f1f6759dd38 ansible-lint]# python -m virtualenv /test-venv
created virtual environment CPython2.7.5.final.0-64 in 275ms
creator CPython2Posix(dest=/test-venv, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/tmp/tmpMnHy4o/seed-app-data/v1)
activators PythonActivator,CShellActivator,FishActivator,PowerShellActivator,BashActivator
[root@8f1f6759dd38 ansible-lint]# /test-venv/bin/python -m pip install setuptools
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: setuptools in /test-venv/lib/python2.7/site-packages (44.0.0)
ERROR: Exception:
Traceback (most recent call last):
File "/test-venv/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/test-venv/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 412, in run
isolated=options.isolated_mode,
File "/test-venv/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 543, in get_lib_location_guesses
scheme = distutils_scheme('', *args, **kwargs)
File "/test-venv/lib/python2.7/site-packages/pip/_internal/locations.py", line 109, in distutils_scheme
d.parse_config_files()
File "/test-venv/lib64/python2.7/site.py", line 153, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/test-venv/lib64/python2.7/site.py", line 153, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/test-venv/lib/python2.7/site-packages/_distutils_patch_virtualenv.py", line 19, in parse_config_files
result = old_parse_config_files(self, *args, **kwargs)
File "/test-venv/lib/python2.7/site-packages/setuptools/dist.py", line 702, in parse_config_files
ignore_option_errors=ignore_option_errors)
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 121, in parse_configuration
meta.parse()
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 426, in parse
section_parser_method(section_options)
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 399, in parse_section
self[name] = value
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 184, in __setitem__
value = parser(value)
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 515, in _parse_version
version = self._parse_attr(value, self.package_dir)
File "/test-venv/lib/python2.7/site-packages/setuptools/config.py", line 349, in _parse_attr
module = import_module(module_name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/ansible-lint/lib/ansiblelint/__init__.py", line 27, in <module>
import six
ImportError: No module named six
[root@8f1f6759dd38 ansible-lint]# cd ..
[root@8f1f6759dd38 /]# /test-venv/bin/python -m pip install setuptools
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: setuptools in /test-venv/lib/python2.7/site-packages (44.0.0) |
Reproducer:
$ cat setup.cfg
[metadata]
name = boom
version = attr: boom.versiontools.get_self_version
|
I've found that it's reproducible with virtualenv 20 and works fine with virtualenv 16. I guess I should ping @gaborbernat about this. |
Confirmed that having older virtualenv next to tox fixes things. So |
Filed a bug: pypa/virtualenv#1685 |
This has been fixed upstream by pypa/virtualenv#1688 and released as |
* Update tripleo-ansible from branch 'master' - Merge "Roll role-addition jobs to python3 (centos-8)" - Roll role-addition jobs to python3 (centos-8) This change moves our role-addition jobs to using centos-8 because pre-commit has dropped support for python2 as noted here [0]. Due to this dropped support, the role-addition job needs to be executed on a system that has access to py3. [0] ansible/ansible-lint#683 (comment) Signed-off-by: Kevin Carter <[email protected]> Change-Id: Iec83c9a3510ef000272dae446346996242cadc8c
This change moves our role-addition jobs to using centos-8 because pre-commit has dropped support for python2 as noted here [0]. Due to this dropped support, the role-addition job needs to be executed on a system that has access to py3. [0] ansible/ansible-lint#683 (comment) Signed-off-by: Kevin Carter <[email protected]> Change-Id: Iec83c9a3510ef000272dae446346996242cadc8c
Issue Type
Ansible and Ansible Lint details
Desired Behaviour
Ansible-lint must assure it installs six all the time.
Actual Behaviour (Bug report only)
Full log https://logserver.rdoproject.org/46/25246/13/check/tox-linters/2a3ccec/job-output.txt
My impression is that this issue is specific to centos-7 where we have old versions of the packaging toolset.
Apparently it affects only installation as a pre-commit hook.
The text was updated successfully, but these errors were encountered: