mock 1.1+ dependencies not installed with pip < 1.5 (Travis containers, Ubuntu 12.04.5 LTS, CentOS/RHEL 7)

[kuba]

I just tried to install mock in virtualenv on Travis container based infrastructure (Python 2.7.3). By default, I get mock requires setuptools>=17.1. Aborting installation, so I went ahead with pip install -U setuptools which upgraded it to 18.2. However, mock doesn't pull in the funcsigs dependency...

$ python --version
Python 2.7.3
$ virtualenv --no-site-packages -p python2 ./venv
Running virtualenv with interpreter /usr/bin/python2
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in ./venv/bin/python2
Also creating executable in ./venv/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
$ travis_retry ./venv/bin/pip install -U setuptools
Downloading/unpacking distribute from https://pypi.python.org/packages/source/d/distribute/distribute-0.7.3.zip#md5=c6c59594a7b180af57af8a0cc0cf5b4a
  Downloading distribute-0.7.3.zip (145Kb): 145Kb downloaded
  Running setup.py egg_info for package distribute

Downloading/unpacking setuptools>=0.7 (from distribute)
  Downloading setuptools-18.2.tar.gz (630Kb): 630Kb downloaded
  Running setup.py egg_info for package setuptools

Installing collected packages: distribute, setuptools
  Found existing installation: distribute 0.6.24
    Uninstalling distribute:
      Successfully uninstalled distribute
  Running setup.py install for distribute

  Running setup.py install for setuptools

    Installing easy_install script to /tmp/letsencrypt/venv/bin
    Installing easy_install-2.7 script to /tmp/letsencrypt/venv/bin
Successfully installed distribute setuptools
Cleaning up...
$ ./venv/bin/pip install mock
Downloading/unpacking mock
  Downloading mock-1.3.0.tar.gz (70Kb): 70Kb downloaded
  Running setup.py egg_info for package mock

    Installed /tmp/letsencrypt/venv/build/mock/.eggs/pbr-1.6.0-py2.7.egg
    [pbr] Processing SOURCES.txt
    warning: LocalManifestMaker: standard file '-c' not found

    [pbr] In git context, generating filelist from git
    warning: no previously-included files found matching '.gitignore'
    warning: no previously-included files found matching '.gitreview'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking pbr>=0.11 (from mock)
  Downloading pbr-1.6.0.tar.gz (99Kb): 99Kb downloaded
  Running setup.py egg_info for package pbr
    [pbr] Processing SOURCES.txt
    warning: LocalManifestMaker: standard file '-c' not found

    [pbr] In git context, generating filelist from git
    warning: no previously-included files found matching '.gitignore'
    warning: no previously-included files found matching '.gitreview'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no files found matching 'requirements.txt'
    warning: no previously-included files found matching '.gitignore'
    warning: no previously-included files found matching '.gitreview'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking six>=1.7 (from mock)
  Downloading six-1.9.0.tar.gz
  Running setup.py egg_info for package six

    no previously-included directories found matching 'documentation/_build'
Installing collected packages: mock, pbr, six
  Running setup.py install for mock
    [pbr] Writing ChangeLog
    [pbr] Generating ChangeLog
    [pbr] ChangeLog complete (0.1s)
    [pbr] Generating AUTHORS
    [pbr] AUTHORS complete (0.2s)
    [pbr] Reusing existing SOURCES.txt
  Running setup.py install for pbr
    [pbr] Writing ChangeLog
    [pbr] Generating ChangeLog
    [pbr] ChangeLog complete (0.2s)
    [pbr] Generating AUTHORS
    [pbr] AUTHORS complete (0.1s)
    [pbr] Reusing existing SOURCES.txt
    Installing pbr script to /tmp/letsencrypt/venv/bin
  Running setup.py install for six

    no previously-included directories found matching 'documentation/_build'
Successfully installed mock pbr six
Cleaning up...
$ ./venv/bin/pip freeze
argparse==1.2.1
distribute==0.7.3
mock==1.3.0
pbr==1.6.0
six==1.9.0
wsgiref==0.1.2
$ ./venv/bin/python -c "import mock"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/letsencrypt/venv/local/lib/python2.7/site-packages/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/tmp/letsencrypt/venv/local/lib/python2.7/site-packages/mock/mock.py", line 83, in <module>
    import funcsigs
ImportError: No module named funcsigs

The command "./venv/bin/python -c "import mock"" failed and exited with 1 during .

Your build has been stopped.

This is a blocker for certbot/certbot#722, I would appreciate your help!

Related: #262

[snordhausen]

Same problem here with Python 2.6. The fix should be really simple: Add funcsigs as dependency to the setup.py.

[djmitche]

It looks like funcsigs was added to requirements.txt, but not setup.py.

[rbtcollins]

I think you have an old version of pip. Try running 'pip list' and paste the output here.

[mknecht]

@rbtcollins confirmed!

Had this error, too. funcsigs was not installed. After upgrading pip from 1.4.1 to 7.1.2 and reinstalling mock, funcsigs is installed and the error is gone.

As a cookbook:

pip install --upgrade pip
pip install --force-reinstall mock

[snordhausen]

This combination works fine:

$ pip list
argparse (1.2.1)
pip (1.5.4)
setuptools (2.2)
wsgiref (0.1.2)

I also tested successfully with pip 1.5.0, funcsigs was installed and mock was working. So 1.5.0 is the first pip version that can properly install mock.

That might become a long-standing issue, because even CentOS/RHEL 7 seem to ship with pip 1.4.1 and mock does not install without manual intervention.

[kuba]

At first I thought pip was coming from python-pip package found in Ubuntu 12.04.5 LTS, i.e. 1.0 (standard system on Travis CI container infrastructure). pip install -U pip revealed that pip==1.1 was installed instead.

pip install -U pip fixed missing funcsigs issue.

Feel free to close this bug report, but I find it a bit concerning that such a standard tool as mock is not installable in some standard environments.

[djmitche]

It also seems fairly easy to fix (add funcsigs to setup.py).

[rbtcollins]

So if make funcsigs an unconditional dependency (it is already a dependency - putting that declaration in setup.py changes nothing), then funcsigs will be installed everywhere rather than just on the old releases of Python that need it.

But more fundamentally, PyPI is a moving target - pip 1.1 is missing 4 years of pretty significant improvements. The bigger concern for me is that folk are using pip from 4 years ago and expecting it to work!

My thought pattern is this for folk using an LTS Linux distribution or similar - you have a choice:

1. Use the packages provided on the system, and for anything not provided build your own (linux) packages.
2. Use pip (or npm or whatever) but the first thing you do is upgrade to the latest stable release of your packaging tools.

If we as a community support '3. Use pip or whatever and only install your thing' the consequence is that the improvement cycle time for pip and services like PyPI will be decades. Thats pretty much intolerable.

Now there is a balance - things that degrade badly are poor, and this seems to be degrading badly on older pips - we need to find a way to error on pip < 1.5.x - for both wheels and sdists. Since we're going to be in an ecosystem for another 15-20 years where thats the case.

[rbtcollins]

@dstufft has suggested a workaround approach - https://bugs.launchpad.net/pbr/+bug/1502692

[radekholy24]

There is a number of people complaining about pbr in your issue tracker. It seems that pbr does not take seriously bugs that mock users are hitting (1379579, 1505007, 1574554). Is pbr really worth the mock users' pain?

[rbtcollins]

@PyDeQ this bug has -nothing- to do with pbr : it has everything to do with pip < 1.5 not supporting environment markers. See https://www.python.org/dev/peps/pep-0508/ for the current spec if you're interested. The pbr bug is suggesting how pbr can improve the situation by encoding a complex workaround for all users of pbr.

The three bugs that you mention are all to do with interop with old - in some cases glacially ancient - versions of setuptools.

I'm not aware of any bugs in mock that are pbr's cause today. If you are please file a separate ticket about that, with reproduction instructions.

[radekholy24]

Huh, you are right. Maybe I assumed that funcsigs is a dependency of pbr, maybe I got confused by all the bugs I read that day... Don't know... My apologies :-(

That means that there is currently only one mock issue caused by pbr that I am aware of: #314 (comment)

Sorry for hijacking the thread :-(

[cjw296]

My guess is that things have moved on sine this issue was last touched, so closing. If there are still problems, please comment and we'll re-open.