Use FlexiBLAS to pick a BLAS and LAPACK at runtime.

[staticfloat]

It would be neat to attempt switching BLAS backends by creating a libblas.{so,dylib,dll} symlink in JULIA_HOME/lib/julia and linking to that instead of directly to a system library or OpenBLAS. Using this method, we could then select a BLAS library to dynamically link against at startup time. (Unfortunately, I don't think this would allow us to switch BLAS implementations after we've already started up, but correct me if there's some strange dynamic "unlinking" magic I don't know about!)

A good way to test this is to install ATLAS, OpenBLAS and Reference BLAS on a Linux computer and symlink to one of them in the Julia build directory, while overriding the Makefile variables for which BLAS library to use to point to that symlink. If you can change the symlink to point to different BLAS implementations, we have a winner.

There's some added complexity involved with LAPACK, to start with I think we can just always compile our own and have it link to our symlinked BLAS implementation. If whoever wants to tackle this happens to have access to an MKL installation, that would be great to test as well.

[ViralBShah]

It would be great if this works. I have always wanted this.

[lindahua]

+1 for this. Very useful.

[StefanKarpinski]

Doing it with symlinks seems like asking for trouble to me – what happens when you start two different julia instances with different BLAS flags at the same time?

[staticfloat]

I'm not sure of a better way to do it, other than writing (yet another) wrapper around the BLAS libraries that is able to intelligently forward requests for a function to another library.

The worst thing I can imagine with the symlink idea is that someone could change the BLAS provider, and then start up worker processes which would then have separate BLAS providers. That's not really the worst thing in the world, but could give rise to subtle bugs just by virtue of having different workers using different implementations. Changing BLAS implementations underneath a running Julia shouldn't be any problem at all, as once Julia is loaded up, the shared libraries are loaded into memory. You can delete the julia executable and all its (opened) libraries and it doesn't care.

[StefanKarpinski]

Hmm. It just seems like it's ripe for a race condition, but if there's only one object file to load, then I suppose that can't happen – the symlink points to one version or the other at load time.

[staticfloat]

I'm closing this as, between the dimensions of specialty BLAS calls, ILP64, and different LAPACK implementations, it's best to not pull the rug out from under Julia by changing the linked BLAS library willy-nilly. We have good support for building against different BLAS implementations, let's leave it at that until we have a compelling use case for something else.

[tkelman]

At a minimum it requires rebuilding the system image because the threading API's vary quite a bit between different implementations. Now if rebuilding the system image incrementally didn't take quite so long, that would be satisfactory I think. I've written a few helper routines somewhere for doing this.

[gramian]

Maybe the FlexiBLAS project ( https://github.com/mpimd-csc/flexiblas ) helps with this as well as issue #595 .

[ViralBShah]

@gramian Does FlexiBLAS work with Windows too? I have been reading a bit more about it - and I always wished something like this existed. Thanks for putting it together. We probably want to get it working in BinaryBuilder and then bring it into the Julia source build.

Also, the README for FlexiBLAS suggests that it needs to use the reference LAPACK - unless I misread it. So, does it not use the LAPACK from MKL or patched LAPACK from OpenBLAS?

[gramian]

Hi,
first of all, I am a mere contributor, and I guess we should add the lead developer @grisuthedragon to this discussion.
As far as I know, FlexiBLAS is not compatible to Windows, only to Linux and likely BSD. I am not sure if the WSL is changing the game in this regard.
In response to the LAPACK question:
FlexiBLAS provides the reference LAPACK functions as a fall-back in case the selected LAPACK implementation does not provide the called function. So, if for example MKL is set as FlexiBLAS backend, all MKL LAPACK functions are used.
If this can be misunderstood, we will update the README in this regard.
@grisuthedragon please correct me or expand if necessary.

[StefanKarpinski]

We can't rely on WSL on Windows — anything we use has to work on Windows natively. However, BinaryBuilder allows us to cross-compile from Linux to any other platform, which can often be easier than getting something to compile on Windows using MSVC. So it might not be that hard to BinaryBuild FlexiBLAS for all platforms.

[ViralBShah]

Things are going to also get a bit complicated on Macs with the new processors and the current state of Fortran.

[ViralBShah]

Various flexiblas issues opened as part of my experimentation to see if we can adopt it: mpimd-csc/flexiblas#9, mpimd-csc/flexiblas#10, mpimd-csc/flexiblas#11

[ViralBShah]

With LBT, we now have a solution in place for 1.7 onwards.