Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: dask.array: asarray(..., copy=None) copies in dask==2024.12 #211

Merged
merged 4 commits into from
Dec 12, 2024

Conversation

ev-br
Copy link
Contributor

@ev-br ev-br commented Dec 5, 2024

closes gh-209

@ev-br ev-br requested a review from asmeurer December 5, 2024 12:13
@ev-br
Copy link
Contributor Author

ev-br commented Dec 5, 2024

Two issues:

@betatim
Copy link
Member

betatim commented Dec 5, 2024

In the standard for asarray it says that for copy=None you should not copy if possible. Do you think "in dask we always make a copy" is compatible with that? And if no, is there anything we can do in array-api-compat to fix this? It would be nice if we could organise (through the compat layer) for users to get the same behaviour no matter what version of dask they are using.

If there is nothing we can do to fix this then I think updating the test is fine.

@lucascolley
Copy link
Contributor

Yes, I think it is compatible for Dask to say that they will always copy, and just raise an error for copy=False.

@ev-br
Copy link
Contributor Author

ev-br commented Dec 5, 2024

The issue with 2024.12 is copy=None, actually.

@lucascolley
Copy link
Contributor

lucascolley commented Dec 5, 2024

The issue with 2024.12 is copy=None, actually.

It is compatible for them to always copy with copy=None.

EDIT: the wording in the spec is

must reuse existing memory buffer if possible

and the Dask maintainers appear to be saying 'reusing the existing memory buffer is never possible' (or at least never supported)

@ev-br ev-br force-pushed the asarray_copy_dask branch from 2442a92 to fda4ab0 Compare December 5, 2024 19:28
ev-br added 2 commits December 5, 2024 22:09
As of dask >= 2024.12, da.from_array() always copies, while before it did not.
Thus copy too, to make array_api_compat.dask.array version-independent.
It seems to hang on CI with dask==2024.12
@ev-br ev-br force-pushed the asarray_copy_dask branch from 8c3c7dd to bce5dcc Compare December 5, 2024 20:09
@ev-br
Copy link
Contributor Author

ev-br commented Dec 5, 2024

CI is green with three tweaks

  • asarray(..., copy=None) for array-likes always copies
  • the test for copy=None is updated to match
  • a unrelated test needs to be skipped on CI: was xfailed and was hanging with dask 2024.12

tests/test_common.py Outdated Show resolved Hide resolved
@crusaderky
Copy link
Contributor

@lucascolley @rgommers this PR is blocking everything

@ev-br
Copy link
Contributor Author

ev-br commented Dec 12, 2024

Let's unblock everything then :-)

@ev-br ev-br merged commit 940a198 into data-apis:main Dec 12, 2024
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_asarray_copy[dask.array] fails on CI with dask==2024.12
4 participants