mock setup.py fails to detect incompatible setuptools fork 'distribute' and after install fails with "Versioning for this project requires..." #383
Comments
|
Thanks, this is a great bug report. I suspect the heart of the problem lies in the use of distribution: mock detects too-old setuptools but distribute isn't triggering that errror during installation. You can see here - https://github.com/testing-cabal/mock/blob/master/setup.py#L5 that we require setuptools 17.1 or newer to install: and some bug in distribute is failing to recognise that it is an older version. (Note that this has broken and been fixed multiple times in setuptools from memory :(.). So to confirm this, upgrading setuptools in the venv should fix it: which you note you have done and does fix it. Therefore - I'm going to retitle the bug to indicate the failure is that we don't detect the incompatibility. I have some sympathy for your situation but at the heart of the problem, running unsupported code and seeking community support is a difficult mix. http://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html is a good blog post in this space. One thing you could do to deal with Python 2.6 and such old tools is to install older versions of mock. I'm sure that <2.0.0 versions will work 'correctly' (I say 'correctly' because they suffer some fairly significant bugs that have lead to errors in dependent code due to tests just silently passing. |
rbtcollins
changed the title
|
The underlying issue here is that distribute isn't installing the egg info in a way that pbr can actually read back - and pbr depends on the pkg_resources component of distribute, so the bug is down in that stack somewhere. |
|
Also, I think this is in the category of, patches welcome: I have no interest in devoting time to supporting installs on 2.6 better: I won't deliberately break it, but thats the limit of my interest. |
|
Please note, that this bug is not specific to Python 2.6. I have deliberately provided a reproducer for Python 2.7. Well, I cannot express how happy I would be if I could stop supporting Py2.6. But the system I want to support is my Nokia N9 phone. And since Microsoft shut down the repositories (I mean stopped supporting N9), I am stuck with Py2.6... So, if you are not interested in supporting Python 2.6 anymore, may I ask you to improve the wording of the README accordingly? Until now I was under impression that you actually support Python 2.6... And if you mention there that e.g. setuptools >= 17.1 is required, I think that you can easily close this bug as resolved. But it's up to you of course. Just a suggestion... Actually, I can easily live with some workarounds in my Vagrantfile. |
|
There's a related error when using (It seems to have been introduced here: e795a4a) |
|
@tadeu thats a totally separate discussion: to install properly mock needs a setuptools that handles conditional dependencies correctly, and we're not detecting that the setuptools in use here fails to do so - due to bugs in setup_requires handling in that (ancient) version. (and I do mean ancient). We'd still fail to install properly if pbr was not being used, and mock would still not work. I'll retitle it to be even more expliclt |
rbtcollins
changed the title
|
Thankfully, distribute and the need for it are now long gone :-) |
Summary: We’ve heretofore used the `mock` library without telling Bazel about it. This isn’t generally a problem when `tensorflow` is installed in the same virtualenv, because `tensorflow` pulls in `mock`. But tests that don’t depend on TensorFlow have no build edge to `mock` when imported into google3’s build system, causing build failures after syncing. This commit makes the `mock` dependency explict. We use `mock==1.0.0` rather than the current version (2.0.0) because the current version depends on `pbr` at runtime in a way that does not play well with Bazel: <testing-cabal/mock#383> (It appears to be trying to find some globally registered `setup.cfg` file from which to read a version identifier.) Rather than vendoring and patching `mock`, declaring a fake `:expect_mock_installed` target, or hacking `PBR_VERSION=2.0.0` into the action env, we simply depend on a saner version of the library, which is also the version used in google3. Existing `mock` users have been detected and fixed up: ``` $ bazel query " > kind( > py_.*rule, > rdeps(//..., set($(git grep -l 'import mock\($\| \)')), 1) > ) > " \ > | awk '{ print "add deps @org_pythonhosted_mock|" $0 }' \ > | buildozer -f - \ > ; ``` Test Plan: All notf tests now pass in a virtualenv that does not include `mock`: ``` $ virtualenv -q -p python2 ./ve $ . ./ve/bin/activate (ve) $ pip install 'absl-py>=0.7.0' 'numpy<2.0,>=1.14.5' boto3==1.9.86 flake8==3.5.0 futures==3.1.1 grpcio==1.6.3 yamllint==1.5.0 >/dev/null 2>&1 (ve) $ python -c 'import mock' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/buildtools/current/sitecustomize/sitecustomize.py", line 152, in SetupPathsAndImport return real_import(name, globals, locals, fromlist, level) ImportError: No module named mock (ve) $ cd ~/git/tensorboard (ve) $ bazel test //tensorboard/... --test_tag_filters=support_notf ... INFO: Build completed successfully, 12 total actions //tensorboard:lazy_test PASSED in 0.2s //tensorboard:lib_test PASSED in 0.9s //tensorboard:manager_test PASSED in 0.3s //tensorboard:plugin_util_test PASSED in 0.5s //tensorboard:program_test PASSED in 0.8s //tensorboard/backend:application_test PASSED in 1.0s //tensorboard/backend:http_util_test PASSED in 0.8s //tensorboard/backend:json_util_test PASSED in 0.8s //tensorboard/compat/tensorflow_stub:gfile_test PASSED in 0.8s //tensorboard/summary:summary_test PASSED in 0.8s //tensorboard/util:platform_util_test PASSED in 0.3s ``` Previously, `manager_test` and `application_test` failed in such an environment. wchargin-branch: mock-dep
Summary: We’ve heretofore used the `mock` library without telling Bazel about it. This isn’t generally a problem when `tensorflow` is installed in the same virtualenv, because `tensorflow` pulls in `mock`. But tests that don’t depend on TensorFlow have no build edge to `mock` when imported into google3’s build system, causing build failures after syncing. This commit makes the `mock` dependency explict. We use `mock==1.0.0` rather than the current version (2.0.0) because the current version depends on `pbr` at runtime in a way that does not play well with Bazel: <testing-cabal/mock#383> (It appears to be trying to find some globally registered `setup.cfg` file from which to read a version identifier.) Rather than vendoring and patching `mock`, declaring a fake `:expect_mock_installed` target, or hacking `PBR_VERSION=2.0.0` into the action env, we simply depend on a saner version of the library, which is also the version used in google3. Existing `mock` users have been detected and fixed up: ``` $ bazel query " > kind( > py_.*rule, > rdeps(//..., set($(git grep -l 'import mock\($\| \)')), 1) > ) > " \ > | awk '{ print "add deps @org_pythonhosted_mock|" $0 }' \ > | buildozer -f - \ > ; ``` Test Plan: All notf tests now pass in a virtualenv that does not include `mock`: ``` $ virtualenv -q -p python2 ./ve $ . ./ve/bin/activate (ve) $ pip install 'absl-py>=0.7.0' 'numpy<2.0,>=1.14.5' boto3==1.9.86 flake8==3.5.0 futures==3.1.1 grpcio==1.6.3 yamllint==1.5.0 >/dev/null 2>&1 (ve) $ python -c 'import mock' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/buildtools/current/sitecustomize/sitecustomize.py", line 152, in SetupPathsAndImport return real_import(name, globals, locals, fromlist, level) ImportError: No module named mock (ve) $ cd ~/git/tensorboard (ve) $ bazel test //tensorboard/... --test_tag_filters=support_notf ... INFO: Build completed successfully, 12 total actions //tensorboard:lazy_test PASSED in 0.2s //tensorboard:lib_test PASSED in 0.9s //tensorboard:manager_test PASSED in 0.3s //tensorboard:plugin_util_test PASSED in 0.5s //tensorboard:program_test PASSED in 0.8s //tensorboard/backend:application_test PASSED in 1.0s //tensorboard/backend:http_util_test PASSED in 0.8s //tensorboard/backend:json_util_test PASSED in 0.8s //tensorboard/compat/tensorflow_stub:gfile_test PASSED in 0.8s //tensorboard/summary:summary_test PASSED in 0.8s //tensorboard/util:platform_util_test PASSED in 0.3s ``` Previously, `manager_test` and `application_test` failed in such an environment. wchargin-branch: mock-dep
Reproducer:
virtualenv venv)source venv/bin/activate)pip install --upgrade pip)pip install mock)python -c "import mock")Actual behavior:
An
Exceptionwith the following traceback:Expected behavior:
mock can be imported
Affected versions:
Additional information
An upgrade of setuptools (
pip install --upgrade pip) (to 28.6.1) after the installation of mock does not fix the issue. An upgrade before the installation helps.Normally, I don't use these old libraries but I am forced to be compatible with Python 2.6 and Debian Wheezy seems to be the last Debian release that has Python 2.6 in its official repositories. Since I use that system only for testing and not for development, the provisioning of the system is automated (via Vagrant). Thus the steps above. Normally, I wouldn't even upgrade pip but I could not install mock without that.
What I want to say is that I understand that you expect users to use newer versions. But in such automated cases, it might not be the case. If you (or pbr - I don't know which package caused that) could enforce newer setuptools, it would really help. If not, at least a note in installation instructions would be nice.
Maybe this is a duplicate of #314. Maybe this is even a duplicate of https://bugs.launchpad.net/pbr/+bug/1505007. However, since the pbr bug is open for 1 year already without any response, it might be reasonable to implement a workaround in mock in the meantime.
The text was updated successfully, but these errors were encountered: