You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When #152 is merged, user subclasses that override upath.core._FSSpecAccessor will very likely not work correctly with Python3.12, because the 3.12 implementation doesn't use the accessor.
There should be a recommended way for users to subclass UPath for Python3.12 AND Python3.8-Python3.11 without having to maintain two code paths.
Searching GitHub, I could collect an example set of user subclasses that can be split into 3 4 intended use cases:
(1) customize fsspec filesystem creation
here the user wants to change the way the fsspec filesystem instance is created:
upath.UPath should offer a classmethod _make_fs() (?) to allow fsspec filesystem instance customization.
It should be possible too to support old code on python3.12 by detecting when the _default_accessor attribute is present and use that to make a compatible _make_fs() (?) fallback method.
(2) customize path formatting
here the user wants to change the way the path is formatted. Examples:
Not really specific to the Accessor subclassing, because the implementation is overriding __new__ in the UPath subclass. The intention is to extract additional fsspec storage_options from the provided args when instantiating UPath(). Example:
When #152 is merged, user subclasses that override
upath.core._FSSpecAccessor
will very likely not work correctly with Python3.12, because the 3.12 implementation doesn't use the accessor.There should be a recommended way for users to subclass
UPath
for Python3.12 AND Python3.8-Python3.11 without having to maintain two code paths.Searching GitHub, I could collect an example set of user subclasses that can be split into
34 intended use cases:(1) customize fsspec filesystem creation
here the user wants to change the way the fsspec filesystem instance is created:
upath.UPath
should offer a classmethod_make_fs()
(?) to allow fsspec filesystem instance customization.It should be possible too to support old code on python3.12 by detecting when the
_default_accessor
attribute is present and use that to make a compatible_make_fs()
(?) fallback method.(2) customize path formatting
here the user wants to change the way the path is formatted. Examples:
(3) fix behavior of some accessor methods
here the behavior of a fsspec filesystem method is fixed by fixing the accessor method. Example:
boxfs.mkdir
: https://github.com/IBM/boxfs/blob/718fb0071d20a7004f44fe2fa0eac26dc9c3d5d5/src/boxfs/_upath.py#L4-L12(4) customize storage_options extraction
Not really specific to the Accessor subclassing, because the implementation is overriding
__new__
in the UPath subclass. The intention is to extract additional fsspec storage_options from the provided args when instantiatingUPath()
. Example:ObjectStoragePath
extracts a connection id from the urlpath: https://github.com/apache/airflow/blob/07fd3646fa765859a152731bf135ea944d0753d7/airflow/io/path.py#L96-L153TODO
While this list in not complete, it does cover
34 reasonably easy to solve cases for making the transition to 3.12 less bumpy for custom subclasses._accessor
inpy3.12
classes and provide warning with instructions on how to migrate..path
(with potential customization)UPath.mkdir
, etc... directly.The text was updated successfully, but these errors were encountered: