Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Set PIP_USER=no #461

Merged
merged 2 commits into from
Aug 25, 2021
Merged

[python] Set PIP_USER=no #461

merged 2 commits into from
Aug 25, 2021

Conversation

csweichel
Copy link
Collaborator

@gitpod-io
Copy link

gitpod-io bot commented Aug 1, 2021

@csweichel csweichel requested a review from jankeromnes August 1, 2021 19:24
@aledbf aledbf self-requested a review August 2, 2021 01:00
Copy link
Member

@aledbf aledbf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aledbf
Copy link
Member

aledbf commented Aug 2, 2021

@csweichel let's merge #458 before this one, cancel the circle CI build, and merge this PR to release the new images?

cc @jankeromnes

@jankeromnes jankeromnes requested a review from AlexTugarev August 2, 2021 06:33
@jankeromnes
Copy link
Contributor

@csweichel @aledbf Many thanks! 💯

@AlexTugarev Since you previously worked on the Python setup, please have a look as well if you have time. 🙂

Also, on a more general note, I believe we should soon do two things with our Python setup:

  • Drop support for (deprecated) Python 2, and in the same run maybe remove Pyenv entirely (and keep only Ubuntu's stock Python) -- relatedly, now that we have Full Workspace Backup generally enabled, I think we can remove a ton of /workspace hacks

  • Compare our Python stack with https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/python-debian.sh and see if maybe there are a few things we could do better (e.g. some additional tools to install, or if there exists a more common / saner Python dev setup)

@@ -154,6 +154,7 @@ RUN curl -fsSL https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-ins
mypy autopep8 pep8 pylama pydocstyle bandit notebook \
twine \
&& sudo rm -rf /tmp/*
ENV PIP_USER=no
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why unsetting it here whereas its set in gitpod-layer on top? my first impression is that this won't be effective.

also, just a reminder, without forcibly enabling PYTHONUSERBASE under /workspace the backup strategy for python is gone, and prebuilds as well. using a virutal env setup seems to be the best way. it would be great to provide a replacement that works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexTugarev the gitpod-layer was removed gitpod-io/gitpod#4923. I will add the .bashrc-append changes here.

@AlexTugarev
Copy link
Member

... now that we have Full Workspace Backup generally enabled ...

@jankeromnes, we do?

@jankeromnes
Copy link
Contributor

... now that we have Full Workspace Backup generally enabled ...

@jankeromnes, we do?

We don't? 😳 @csweichel @aledbf do you know?

@aledbf
Copy link
Member

aledbf commented Aug 2, 2021

We don't? flushed @csweichel @aledbf do you know?

It is not enabled yet (there is an issue with the user permissions when we create the workspace backup)

Copy link
Contributor

@jankeromnes jankeromnes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for improving Gitpod's Python setup! 💯

Blocking this PR until #461 (comment) is resolved, so that we don't accidentally merge it as is.

@gitpod-io gitpod-io deleted a comment from pingalex bot Aug 9, 2021
@gitpod-staging
Copy link

@aledbf
Copy link
Member

aledbf commented Aug 11, 2021

@csweichel @jankeromnes @AlexTugarev can we merge this PR? Without it, most of the examples are broken when the new image builder mk3 is used (due to the removal of the gitpod layer)
(needs a rebase first)

Copy link
Contributor

@jankeromnes jankeromnes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock this PR, but I would very very much prefer it if we remove all of these silly variables.

  • For Python, we're unwittingly breaking many people's setups by forcing some PIP_USER or VENV things

  • For Ruby, our setup is so broken that every major Ruby project just replaces it entirely in a Dockerfile (also, we still use RVM instead of today's standard Rbenv)

  • For Rust, the /workspace hack still looks okay, but here too I'd very much prefer removing the hack altogether, and instead enable Full Workspace Backup everywhere

Comment on lines +179 to +181
ENV GEM_HOME=/workspace/.rvm
ENV GEM_PATH=$GEM_HOME:$GEM_PATH
ENV PATH=/workspace/.rvm/bin:$PATH
Copy link
Contributor

@jankeromnes jankeromnes Aug 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this does more harm than good. Maybe let's remove it?

Suggested change
ENV GEM_HOME=/workspace/.rvm
ENV GEM_PATH=$GEM_HOME:$GEM_PATH
ENV PATH=/workspace/.rvm/bin:$PATH

Comment on lines +157 to +160
ENV PIP_USER=no
ENV PIPENV_VENV_IN_PROJECT=true
ENV PYTHONUSERBASE=/workspace/.pip-modules
ENV PATH=$PYTHONUSERBASE/bin:$PATH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no Python experience to tell whether setting any of these variables is okay.

However, I don't think you'll find any reviewers here with Python experience, so continuing to wait for a review here is unhelpful.

Also, if possible, I would very much prefer not setting any of these variables. I'm aware that this may break package persistence (they'll land outside of /workspace) but for that I'd rather focus on enabling Full Workspace Backup instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but for that I'd rather focus on enabling Full Workspace Backup instead.

this is one of those issues where we have a circular dependency; we need this PR to switch to the new image builder mk3. With that, we can focus on the dazzle refactor that will reduce the image size and simplify the dependencies tangle, and then we can start using stargz. After that, we should focus on Full Workspace Backup. Without all those previous steps we are not improving the size, usability, or modularity of the workspace container images.

full/Dockerfile Outdated Show resolved Hide resolved
csweichel and others added 2 commits August 24, 2021 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Gitpod break the behavior of pre-commit of Python
4 participants