Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop installing setuptools in python 3.12+ #954

Merged

Conversation

yosifkit
Copy link
Member

@yosifkit yosifkit commented Aug 23, 2024

This fixes the intent that was meant to happen in #833 of no longer including setuptools (or wheel).

Fixes #952.

If you are using Python 3.12+ and you have build or runtime dependencies on setuptools or wheel, a simple addition to your Dockerfile is all that is needed. (I think longer term, they might make sense as additions to your requirements.txt)

FROM python:3.12
WORKDIR /usr/src/app

### ⬇️ this line ⬇️ ##########################
RUN pip install --no-cache-dir setuptools wheel
###############################################

# continue your regular installation
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

# ...

@@ -307,7 +307,10 @@ RUN set -eux; \
"pip==$PYTHON_PIP_VERSION" \
{{ if .setuptools then ( -}}
"setuptools==$PYTHON_SETUPTOOLS_VERSION" \
{{ ) else "" end -}}
{{ ) else ( -}}
--no-setup-tools \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yosifkit yosifkit force-pushed the really-no-setuptools-or-wheel branch from 3889aba to f7aece1 Compare August 24, 2024 00:06
python -c 'import wheel'
{{ ) else ( -}}
# smoke test to ensure that setuptools/wheel are not included
if python -c 'import setuptools' || python -c 'import wheel'; then \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+ python -c import setuptools
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
+ python -c import wheel
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'wheel'

🤔 Maybe these could be 2>/dev/null

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess relevant to #954 (comment), if we made these a dedicated script we could probably do a single script that's more complex, like catching the import exception directly in Python code instead?

Copy link
Contributor

@edmorley edmorley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for opening this! :-)

pip --version
pip --version; \
{{ if .setuptools then ( -}}
# smoke test to ensure that setuptools/wheel are included
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes sense. I removed them so they can just be in the test scripts.

@@ -307,10 +307,23 @@ RUN set -eux; \
"pip==$PYTHON_PIP_VERSION" \
{{ if .setuptools then ( -}}
"setuptools==$PYTHON_SETUPTOOLS_VERSION" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For completeness, and in case get-pip were to ever stop installing wheel by default for older Python versions, should we add wheel as an explicit dependency here (without a version specfier, to match the get-pip behaviour)?

ie:

Suggested change
"setuptools==$PYTHON_SETUPTOOLS_VERSION" \
"setuptools==$PYTHON_SETUPTOOLS_VERSION" \
wheel \

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah, that's probably a good idea, even though I don't love it 😂 😭 ❤️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Windows template will need similar changes too:

{{ if .setuptools then ( -}}
('setuptools=={0}' -f $env:PYTHON_SETUPTOOLS_VERSION) \
{{ ) else "" end -}}
; \
Remove-Item get-pip.py -Force; \
\
Write-Host 'Verifying pip install ...'; \
pip --version; \

@edmorley
Copy link
Contributor

edmorley commented Aug 24, 2024

I think it might also be worth updating the PR description to give an example of how to restore the previous behaviour, for the small subset of users who are are running pip with build isolation disabled, or have broken dependencies (that have run-time dependency on setuptools but that don't correctly list an explicit install_requires dependency on it).

For example:

FROM python:3

WORKDIR /usr/src/app

RUN pip install --no-cache-dir setuptools wheel
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

# ...

(this is an adapted version of the Dockerfile usage example listed at https://hub.docker.com/_/python/)

@tianon
Copy link
Member

tianon commented Aug 28, 2024

docker-library/official-images#17446 👀 (I've verified that works with this PR's images, and fails on the current images)

@tianon tianon closed this Aug 29, 2024
@tianon tianon reopened this Aug 29, 2024
@tianon
Copy link
Member

tianon commented Aug 29, 2024

(Close and reopen to force trigger new CI so it includes our updated test. 🙇 I've also got https://github.com/docker-library/python/actions/runs/10623880671 running fresh on master so we can see the failing baseline.)

@tianon
Copy link
Member

tianon commented Aug 29, 2024

Hilariously, thanks to pypa/get-pip#218, my "baseline" test on master is succeeding because now get-pip.py has the same behavior as this PR. 😂

@tianon tianon merged commit 811625e into docker-library:master Aug 29, 2024
88 checks passed
@tianon tianon deleted the really-no-setuptools-or-wheel branch August 29, 2024 23:26
@tianon
Copy link
Member

tianon commented Aug 29, 2024

Now we're being really explicit about it both directions, which feels right. 😄

docker-library-bot added a commit to docker-library-bot/official-images that referenced this pull request Aug 29, 2024
Changes:

- docker-library/python@811625e: Merge pull request docker-library/python#954 from infosiftr/really-no-setuptools-or-wheel
- docker-library/python@a8ec33a: Stop installing setuptools in python 3.12+
- docker-library/python@5ee49c2: Update 3.9 to get-pip pypa/get-pip@def4aec
- docker-library/python@0dabaf3: Update 3.8 to get-pip pypa/get-pip@def4aec
- docker-library/python@14e03d8: Update 3.13-rc to get-pip pypa/get-pip@def4aec
- docker-library/python@109e83a: Update 3.12 to get-pip pypa/get-pip@def4aec
- docker-library/python@625a0a3: Update 3.11 to get-pip pypa/get-pip@def4aec
- docker-library/python@e84c3f7: Update 3.10 to get-pip pypa/get-pip@def4aec
Nastaliss added a commit to CyberCRI/projects-backend that referenced this pull request Sep 4, 2024
samonaisi pushed a commit to CyberCRI/projects-backend that referenced this pull request Sep 4, 2024
Nastaliss added a commit to CyberCRI/projects-backend that referenced this pull request Sep 5, 2024
* feat: rework stats

* feat: decrease docker version for now

* fix: lint and format

* feat: remove import

* feat: unpin docker version and add wheel + setuptools
docker-library/python#954

* fix test

* use better permissions checks

---------

Co-authored-by: Sam Onaisi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stop including setuptools and wheel in Python 3.12+ images
3 participants