Building docs when module compiled using +deterministic fails

[garazdawi]

In erlang/otp#8295 I discovered that it is not possible to build documentation for Erlang (and possibly Elixir?) modules build using the +deterministic compiler flag.

When +deterministic is used, it will no longer be possible to locate the location of the source file, so I think that in such case ExDoc should not emit any links to the source code and all warnings/errors have to be emitted as best they can be.

[josevalim]

Sounds good to me. Although missing the source code links would certainly be a shame.

[garazdawi]

Not much that can be done about that without fixing the Erlang compiler to not strip the path from file names when compiling using +deterministic.

[josevalim]

One option perhaps is to store the relative path to the file in relation to the cwd (assuming the file is inside cwd). This way it is always deterministic if you compile from the same directory (which is the most common scenario anyway).

[garazdawi]

I don't recall the details on why it is done the way it is, but part of the answer is in this pr: erlang/otp#1976

[jvantuyl]

I like the "relative path" idea.

Though I think it's less important for it to be "cwd" and more useful to be "project root". I know that's the convention that Mix is built around, but I'd like to be clear that's what we mean. I think there distinction is subtle enough that lots of people would miss it.

As a stopgap, I define a special environment docs in addition to the conventional dev / prod / test. For that environment, I specifically disable +deterministic. It burns some more cycles on the CI server, but it works for getting full path and debugging information.

Of course, this depends heavily on how you're turning +deterministic on in the first place. I do it with a conditional on Mix.env in my project's erlc_options, so environments are easy for me. If your build is doing it some other way, then getting that environment may be more difficult.

[jvantuyl]

For anyone who runs across this and doesn't know what the point of deterministic builds is, it's a big thing with the reproducible builds crowd.

I actually quite like reproducible builds. I've got BuildKite configured to use the hash of the file to do caching. When you've got 40,000 builds where 90% of the built artifacts never change, this can save tons of space and network bandwidth.

There are also strong needs for this in certain regulatory regimes. If you ever want the FCC to approve a device to use firmware with Elixir in it, you need to provide a "trusted binary". Reproducible builds give users a way to generate a certified binary without having to just download it.

Similarly, you can use detached signatures to allow you to build some source and determine if it exactly matches what's expected. This can even detect situations where your Elixir or Erlang compiler chain or network infrastructure have been hacked. Together with a strong approval process and some out-of-band way of distributing those signatures, you can ensure your endpoints don't trust binaries that have been tampered with.