Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] update autodiff tutorial #3836

Merged
merged 4 commits into from
Oct 7, 2024
Merged

[docs] update autodiff tutorial #3836

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4883e78
Update autodiff tutorial
gdalle Oct 6, 2024
41fca67
Update operator_ad.jl
odow Oct 6, 2024
b745a2f
Apply suggestions from code review
odow Oct 7, 2024
23c82e8
Update operator_ad.jl
odow Oct 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/src/tutorials/nonlinear/operator_ad.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,9 +331,9 @@ enzyme_rosenbrock()
# Julia offers [many different autodiff packages](https://juliadiff.org/).
# [DifferentiationInterface.jl](https://github.com/gdalle/DifferentiationInterface.jl)
# is a package that provides an abstraction layer across multiple underlying
odow marked this conversation as resolved.
Show resolved Hide resolved
# autodiff libaries.
# autodiff libraries.

Copy link
Contributor

@wsmoses wsmoses Oct 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add a disclaimer:

While DI makes it easier to experiment with different autodiff tools, this comes at a cost. In particular, DI may produce slower derivatives than using an AD tool directly, and sometimes fail to work at all. For example, in this code passing AutoEnzyme to DI fails to differentiate the code which is supported when directly calling Enzyme, like above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way do you have any idea why it fails here? As usual it's a segfault so not particularly user friendly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea, I haven’t even tried to run it or seen what the error message, just took your word that it failed xD.

open an issue with the mwe and error and can take a look

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and sometimes fail to work at all

I don't want to mention something in our documentation if this is the case.

What is the failure mode? Will we gracefully report errors? Segfaults are not good.

Copy link
Contributor

@wsmoses wsmoses Oct 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh agreed in that segfaults are bad, in that case @gdalle just drop DI docs then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized my changes mean we never actually mention Enzyme in the context of DI, so perhaps that's okay.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that it is a segfault is not due to DI per se, it's Enzyme not catching the error gracefully. Sure, DI might be using Enzyme wrong in this particular case (perhaps because of a function returning two closures), but I too would very much prefer a clean error message from Enzyme to a segfault. Will try to reproduce tomorrow.

Again haven't seen the error message so not clear what's up, but I'm unaware of any segfaults caused by Enzyme atm. Most the time people saw segfaults in the past were actually bugs in the Julia compiler itself (e.g. JuliaLang/julia#55306 ).

That said unfortunately even if you're not the underlying cause of an error, the top level package you use is going to be the one people blame for issues (especially if code works otherwise without it). Analogously, there's a ton of code in Enzyme working around bugs in the Julia and LLVM compiler errors =/. Perhaps something similar would be good in DI?

Copy link
Contributor Author

@gdalle gdalle Oct 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not DI's job to work around LLVM errors, it's Enzyme's or Julia's. DI is a fancy argument-passer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here's the MWE: EnzymeAD/Enzyme.jl#1942

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, pure Enzyme also fails on this one when one uses splat(f)

# All the necessary information about your choice of underlyingg autodiff
# All the necessary information about your choice of underlying autodiff
# package is encoded in a "backend object" like this one:

DifferentiationInterface.AutoForwardDiff()
Expand Down
Loading