Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Officialy support python 3.12 #17350

Closed
FloChehab opened this issue Oct 23, 2023 · 9 comments · Fixed by #19588
Closed

Officialy support python 3.12 #17350

FloChehab opened this issue Oct 23, 2023 · 9 comments · Fixed by #19588
Labels
type: feature-request

Comments

@FloChehab
Copy link

What's the use case?

Python 3.12 is officially out.
We are looking forward for dagster to officially support this release.

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
@yuhan yuhan mentioned this issue Oct 31, 2023
Closed
@hbruch
Copy link

hbruch commented Nov 10, 2023

Note: due to python-pendulum/pendulum#696, this probably might require upgrading to pendulum v3

@hbruch hbruch mentioned this issue Nov 10, 2023
Closed
hbruch added a commit to mobidata-bw/ipl-dagster-pipeline that referenced this issue Nov 10, 2023
@hbruch
Fix python version
ae0aca8 
see also dagster-io/dagster#17350
hbruch added a commit to mobidata-bw/ipl-dagster-pipeline that referenced this issue Nov 13, 2023
@hbruch
Fix python version
3c47c29 
see also dagster-io/dagster#17350
@cbini
Copy link
Contributor

cbini commented Nov 30, 2023

I think the Dagster team might want to consider moving away from pendulum. I love the library, but it's not actively developed. Looking through the repo issues and discussion, they originally planned to release v3 early 2022 and the main dev doesn't appear to have time to dedicate to the project as their preoccupied with Poetry.

This discussion is kind of concerning: python-pendulum/pendulum#771

@cbini
Copy link
Contributor

cbini commented Dec 18, 2023

Christmas came early: pendulum 3.0

@ogenstad ogenstad mentioned this issue Dec 22, 2023
Closed
@streitl
Copy link

streitl commented Jan 12, 2024

Hey Dagster team, any news on when pendulum will be updated to v3 and Python 3.12 supported? Thanks!

@alangenfeld
Copy link
Member

latest update here: #19456 (comment)

@zaneselvans zaneselvans mentioned this issue Jan 31, 2024
Closed
@alexismanin
Copy link

alexismanin commented Feb 7, 2024 •
edited
Loading

Does a todo list of actions for Python 3.12 compatibility exist somewhere ?

From what I see, at least universal_path blocks Python 3.12 migration. Trying to use a dagster_aws.s3.S3Resource in Python 3.12 environment causes the following error:

ImportError: cannot import name '_PosixFlavour' from 'pathlib' (/usr/lib64/python3.12/pathlib.py)
...
File ".../venv/lib64/python3.12/site-packages/dagster_aws/s3/__init__.py", line 6, in <module>
  from .io_manager import (
File ".../venv/lib64/python3.12/site-packages/dagster_aws/s3/io_manager.py", line 20, in <module>
  from upath import UPath
File ".../venv/lib64/python3.12/site-packages/upath/__init__.py", line 2, in <module>
  from upath.core import UPath
File ".../venv/lib64/python3.12/site-packages/upath/core.py", line 8, in <module>
  from pathlib import _PosixFlavour  # type: ignore
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UPDATE : The Python 3.12 compatibilty PR on universal_path has been merged last week. If it is released soon, we might hope to reach for Python 3.12 compatibility soon ?

UPDATE 2: Testing universal_path master (i.e. declaring explicit dependency to universal_pathlib@git+https://github.com/fsspec/universal_pathlib.git#egg=f7af1748ba7d2020a126cafdb47bad0261aa198e) solves above error.

@rexledesma rexledesma mentioned this issue Feb 12, 2024
Merged
@ja-me-sk
Copy link

While not a core dependency, I initially ran into this as a limitation with dagster-duckdb alongside pendulum when creating the dagster_university project.

As suggested by docs, using the supported version of 3.11.x works fine on setup.

@cbini
Copy link
Contributor

cbini commented Feb 14, 2024

universal_pathlib released an update yesterday with 3.12 support: https://github.com/fsspec/universal_pathlib/releases/tag/v0.2.0

Haven't tried it yet, but will today and keep you posted.

This was referenced Feb 14, 2024
Merged
Merged
gibsondan added a commit that referenced this issue Feb 15, 2024
@gibsondan
Add python 3.12 testing to buildkite (#19588)
a116c44 
## Summary & Motivation
Leaving the default at 3.11 since not all packages are supported yet,
but with the release of universal_pathlib 0.2 today we are ready to
support 3.12 in many of the core packages. The only changes that were
required here were in tests.

resolves #17350

Test Plan: BK
gibsondan added a commit that referenced this issue Feb 15, 2024
@gibsondan
Add python 3.12 marker to setup.py files (#19780)
069b7d0 
Summary:
Added the 3.12 marker for modules that we now have under test

part of #17350

## Summary & Motivation

## How I Tested These Changes
@gibsondan
Copy link
Member

dagster should work on python 3.12 now!

@Sage0614 Sage0614 mentioned this issue Feb 17, 2024
Closed
PedramNavid pushed a commit that referenced this issue Mar 28, 2024
@gibsondan @PedramNavid
Add python 3.12 testing to buildkite (#19588)
0b1760b 
## Summary & Motivation
Leaving the default at 3.11 since not all packages are supported yet,
but with the release of universal_pathlib 0.2 today we are ready to
support 3.12 in many of the core packages. The only changes that were
required here were in tests.

resolves #17350

Test Plan: BK
PedramNavid pushed a commit that referenced this issue Mar 28, 2024
@gibsondan @PedramNavid
Add python 3.12 marker to setup.py files (#19780)
d330482 
Summary:
Added the 3.12 marker for modules that we now have under test

part of #17350

## Summary & Motivation

## How I Tested These Changes
@agrueneberg agrueneberg mentioned this issue Oct 14, 2024
Merged
garethbrickman pushed a commit that referenced this issue Oct 14, 2024
@agrueneberg
Add support for Python 3.12 in scaffold template (#25251)
4c00783 
It appears that Python 3.12 has been officially supported for months now
[#17350], but the scaffold template has not been updated yet. This
change increases the upper version bound in the template from <3.12 to
<3.13.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: feature-request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add python 3.12 testing to buildkite dagster-io/dagster
8 participants
@alangenfeld @hbruch @cbini @ja-me-sk @gibsondan @streitl @FloChehab @alexismanin