Support pendulum 3

[smackesey]

Pendulum 3 is in beta, we currently have no upper version cap on it, and it is not currently compatible with dagster: https://dagsterlabs.slack.com/archives/C03A0D72A6T/p1697637653174099

[matthias-Q]

Pendulum 3.0.0 has been released 2 days ago an now Dagster breaks for me (in CI) because of the pendulum dependency.

[matthias-Q]

it looks like it is fixed in the latest dagster release.

[smackesey]

Yes, we pinned it to <3 in 1.5.7.

[pspino]

It seems that this change is affecting prior version (such as 1.3.14). Was the change in the version of pendulum added to "legacy" versions of dagster ? Was it intended ?

==================================== ERRORS ====================================
_________________ ERROR collecting test/extraction_op_test.py __________________
../venv/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
../venv/lib/python3.10/site-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
../venv/lib/python3.10/site-packages/_pytest/python.py:531: in collect
    self._inject_setup_module_fixture()
../venv/lib/python3.10/site-packages/_pytest/python.py:545: in _inject_setup_module_fixture
    self.obj, ("setUpModule", "setup_module")
../venv/lib/python3.10/site-packages/_pytest/python.py:310: in obj
    self._obj = obj = self._getobj()
../venv/lib/python3.10/site-packages/_pytest/python.py:528: in _getobj
    return self._importtestmodule()
../venv/lib/python3.10/site-packages/_pytest/python.py:617: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
../venv/lib/python3.10/site-packages/_pytest/pathlib.py:565: in import_path
    importlib.import_module(module_name)
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
../venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
    exec(co, module.__dict__)
extraction_op_test.py:7: in <module>
    from dagster import DagsterTypeCheckDidNotPass
../venv/lib/python3.10/site-packages/dagster/__init__.py:84: in <module>
    from dagster._config.config_schema import ConfigSchema as ConfigSchema
../venv/lib/python3.10/site-packages/dagster/_config/__init__.py:5: in <module>
    from .config_type import (
../venv/lib/python3.10/site-packages/dagster/_config/config_type.py:6: in <module>
    from dagster._annotations import public
../venv/lib/python3.10/site-packages/dagster/_annotations.py:9: in <module>
    from dagster._utils.backcompat import (
../venv/lib/python3.10/site-packages/dagster/_utils/__init__.py:50: in <module>
    import dagster._seven as seven
../venv/lib/python3.10/site-packages/dagster/_seven/__init__.py:16: in <module>
    from .compat.pendulum import PendulumDateTime as PendulumDateTime  # re-exported
../venv/lib/python3.10/site-packages/dagster/_seven/compat/pendulum.py:31: in <module>
    pendulum.DateTime if _IS_PENDULUM_2 else pendulum.Pendulum  # type: ignore[attr-defined]
E   AttributeError: module 'pendulum' has no attribute 'Pendulum'. Did you mean: '_pendulum'?

[smackesey]

Hi pspino, it affects older versions of Dagster because these versions did not place an upper bound pin on pendulum. Therefore unless the user pins it, during installation the newest version of pendulum is downloaded. In the past this was the 2.x series, which Dagster is compatible with.

In the last few days pendulum released the 3.x series, so now older versions of Dagster will download the 3.x pendulum, which they are not compatible with. This problem doesn't exist post-1.5.7 because we put an upper bound of 3 on pendulum.

[ei-grad]

Looks like this issue could be closed - #19480