-
Notifications
You must be signed in to change notification settings - Fork 588
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Array API extra #3065
Array API extra #3065
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Woohoo! Thanks @honno; this is a really exciting new capability and great way help (test) the array ecosystem 😁
I've dumped some preliminary feedback below, but the key thing is that this looks good - I am confident that we can fix any small concerns and get this merged. Definitely a relief for such a large diff!
Remaining design-level issues:
- Do we really need to expose a module full of functions in addition to the
get_strategies_namespace()
function? It seems to me that in practice this is much more ergonomic than manually passing around thexp
array module, even when you're using several of them. I also want to think about the name -make
feels more appropriate thanget
, but whilestrategies_namespace
seems too vague I don't yet have a better idea. - Moving the non-array-module dependent strategies like
array_shapes()
ormutually_broadcastable_shapes()
into a private namespace (e.g.hypothesis.extra.__array_helpers
) to avoid duplicating code. IMO we can test these only against numpy, saving CI time. This should probably be a precursor PR (Move NumPy extra's strategies which are not dependent on NumPy into a seperate helper module #3067) after which we can rebase this one.
And more trivially, you'll need to add a RELEASE.rst
for a minor
release and add yourself (and any other contributors) to AUTHORS.rst
so we give proper credit 😉
If it's a hack but it works, we'll happily ship it! Especially really awful hacks which somehow improve the user experience; that's basically our signature around here 😉
PEP 562 makes this easy with module- If not I'm going to propose an update to the standard - the setuptools "entry points" API would be perfect for this - and a way to discover the installed array-api implementations seems generally useful. Just drop
Thanks for checking! I think these are mostly fine, it's more array-specific style than bad style.
IMO we should probably add a warning to the docs: "The Array API standard is not yet final. We may therefore make breaking changes in minor releases, if this is necessary to support newer versions of the standard." |
I'm really glad to hear I'm on the right track :) Thanks for the great feedback too!
There is no standard way to discover the Array API implementation from a top-level package, where the only module discovery mechanism was built for array-consuming libraries by way of I was thinking of directly mapping the imports per module, e.g. if Jax had Note to self:
|
The entry points would be much nicer. I imagine some libraries will indeed follow NumPy with |
🤗 In order of priority, it would be great if you could shepard #3067 - just ping me for a final review once it's ready to merge. I'd also appreciate a second set of eyes on this PR for design-level review (and probably eventually nitpicky code review, but after we rebase). If you're up for both I can take a stab at #3066 this weekend 🚀 |
105c5a9
to
80b3017
Compare
@honno - are you up for squash-rebasing on master, to make this a little easier to review? |
I think our main remaining points are
Actually this last thing, together with our mock module, might solve the docs issue - just point Sphinx at @rsokl, have I missed anything? |
This seems like the cleanest easy solution, although I think Also noting the current mock depends on NumPy, so maybe a specific hook triggers for
Incase this could be useful, Array API implementations have to be Python 3.8 or above due to the use of positional-only parameters, although maybe there's a world folk will ship/want to play around with 3.7 partial adopters.
Yep my commit today e2ca6af have the namespaced strategies not show the module name anymore—more of a consequence of how I've dropped top-level strategies than a purposeful decision. Showing the module binds still seems nice tho. |
4cd19d9
to
b252efd
Compare
7818835
to
d98ce1e
Compare
On my end I'm pretty happy with everything for this PR, it's just the docs issue. If we go the entry points route then we can utilise a mock like Zac suggested and generate docs from there at least (note it's not as simple as If we're not doing entry points for this PR, or would rather highlight the The TODO now seems to be:
Future PR ideas:
|
I've got a paper deadline on Saturday (ICSE'22); unlikely I'll have much time before then but happy to implement my getattr idea and see about docs once the paper is in. Thanks again for all your work on this - it's been skilled, patient, valuable, and very much appreciated. I'm really looking forward to shipping it to the array-oriented parts of the Python community |
Co-authored-by: Aaron Meurer <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Ryan Soklaski <[email protected]>
1fefd54
to
520d018
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Time to merge!
And as I've said before, thanks for all your work on this - it's been skilled, patient, valuable, and very much appreciated. I look forward to seeing what you work on next 😁
Oh wow, awesome! Likewise I really appreciated the feedback process for all the PRs, and grateful to of learnt sooo much about Hypothesis and beyond. Down the line feel free to ping me on Array API issues. I completely forgot to ask about the |
I'm happy to pay it forward - expanding on |
What this introduces
This implements strategies for Array API implementations inside
hypothesis.extra.array_api
(closes #3037). As the Array API is largely based on NumPy behaviour, I imitatedhypothesis.extra.numpy
where appropriate and so these strategies will hopefully feel familiar to the extra's contributors and users.Strategies in
array_api
do not import array modules, instead taking them as thexp
argument. For the most part they assume that the user has indeed provided an Array API-compliant module.The strategies would be used by Array API implementers (e.g. @asmeurer's compliance suite) and array-consuming libraries (examples).
Many tests are based on the
test/numpy
ones. There is a mock array-API implementation inxputils.py
. Tests will try to import NumPy's Array API implementation (numpy/numpy#18585 was merged just today) and will fall back to the mocked one. I couldn't easily mock a "compliant-looking" array object so a particular non-compliance warning is suppressed and some assertions are skipped when necessary.cc @mattip
Specific requests for feedback
An immediate concern I have is with how I form pretty ("reflective") reprs for
xp
consuming strategies. Default use of@defines_strategy
(and thusLazyStrategy
) is prone to produce some rather noisy repr strings, so I ended up wrapping these strategies with a custom decorator@pretty_xp_repr
... it seems to be a hacky solution, especially since it gets called multiple times.I'm also wondering if you're happy with the
get_strategies_namespace()
method that Zac suggested. It could be nice to not even have the top-level strategies (i.e. array module needs to be passed) to require users to use it, which could mitigate confusion. There could also be some magic where a user could import sayextra.array_api.pytorch
and have Hypothesis auto import a (future) Array API implementation in PyTorch.I see the NumPy extra (and thus this PR) violates the house API style. Let me know if for
array_api
I should take the oppurtunity to drop potentially undesirable features inarrays()
such as inferring strategies via dtype and passing kwargs viaelements
.The shape/axis/index strategies were implemented namely to avoid importing NumPy by using
extra.numpy
, which also allows use of Array API naming conventions and removes some NumPy-specific limitations. They near-verbatim emulate those in the NumPy extra, so a future PR could haveextra.numpy
wrap them to deal with small differences.The Array API is not finalised yet. Some tests may need slight modification in the future if the consortium decide in data-apis/array-api#212 that they don't want polymorphic return values in
xp.unique()
.