Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert to dh-virtualenv.deb to python 3 #300

Merged
merged 5 commits into from
Jun 2, 2020

Conversation

richvdh
Copy link
Contributor

@richvdh richvdh commented May 18, 2020

As we all know, python2.7 is out of support, and Debian are removing support for it. In particular, python-mock and python-sphinx have both been removed from the Debian repositories for testing and unstable, meaning that it is impossible to satisfy the Build-Depends for dh-virtualenv on those distributions.

Likewise, Debian are removing support for debpython - dh-python superceded it a long time ago.

The long and the short of it is that we need to switch dh-virtualenv to run under python3 rather than python2. (Note that dh-virtualenv will still be able to build other packages to run under python2.)

To get there, we need to do a few things:

  1. Fix a tiny python3-incompatibility in dh-virtualenv

  2. Import debhelper.py into the dh-virtualenv tree, since debpython has been removed. (There's a different version in dh-python, but it's a bit specific to dh-python, and it's not a large module).

  3. change the debian rules for dh-virtualenv.deb to use the pybuild buildsystem. The default buildsystem for python projects (python_distutils) is deprecated and relies on stuff in the python2 packages which we'd rather avoid.

  4. Finally, replace the dependencies with python3 equivalents, and build the package with dh-python3 instead of dh-python2.

Hey presto, dh-virtualenv will run under python 3 instead of python 2.

I've tested this on a range of distributions including Ubuntu Xenial and Focal, and Debian Sid and Stretch.

Fixes #284, #296, #297 and #298.

richvdh added 5 commits May 18, 2020 22:57
iteritems doesn't exist in python 3.
This seems to have been removed from debian in recent versions, and it's not
that big, so let's just bring it in here.
pybuild is a replacement for `python_distutils`, which requires
python2. pybuild is part of `dh-python`, so it will be available wherever our
build deps are installed.

See also:
 * https://wiki.debian.org/Python/LibraryStyleGuide#Overview
 * https://wiki.debian.org/Python/Pybuild
Switch to using python 3 for building the dh-virtualenv deb.

Python2 is dead, and the existing build fails for Ubuntu 20.04 and debian
testing (specifically, python-mock and python-sphinx no longer exist).
Copy link
Collaborator

@nailor nailor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks a lot for the work but here, and sorry for ths review dragging.

Quite symbolic btw that the PR that makes this Python 3 compatible is #300 😁

from dh_virtualenv import Deployment
from dh_virtualenv.cmdline import get_default_parser
from dh_virtualenv.debhelper import DebHelper
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, maybe I should have vendored this in earlier, the sys path trickery has always been... not so nice, and this is so much cleaner 👍

@@ -38,7 +35,7 @@ log = logging.getLogger(__name__)
def _shell_vars(**kwargs):
"""Convert the given values into the equivalent shell snippet defining them."""
return '\n'.join("dh_venv_{0}='{1}'".format(k, v.replace("'", r"'\''"))
for k, v in sorted(kwargs.iteritems()))
for k, v in sorted(kwargs.items()))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm honestly baffled that there isn't more python2-isms in this code 😁

@nailor
Copy link
Collaborator

nailor commented Jun 2, 2020

Once more, thanks a lot, and really happy to merge this and make dh-virtualenv work on python3 only environments 😊

@nailor nailor merged commit ac6e1b1 into spotify:master Jun 2, 2020
richvdh added a commit to matrix-org/synapse that referenced this pull request Jun 3, 2020
Upstream have merged our changes
(spotify/dh-virtualenv#300), so let's switch back to it
instead of using our fork.
richvdh added a commit to matrix-org/synapse that referenced this pull request Jun 3, 2020
Upstream have merged our changes
(spotify/dh-virtualenv#300), so let's switch back to it
instead of using our fork.
phil-flex pushed a commit to phil-flex/synapse that referenced this pull request Jun 16, 2020
Upstream have merged our changes
(spotify/dh-virtualenv#300), so let's switch back to it
instead of using our fork.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python 2 end of life in Sid
2 participants