Cannot detect openmp on OSX Catalina

[dipterix]

Heads up:

For those who found this issue. Basically starting from R 4.0, CRAN uses the compilers shipped with system to compile R packages. This is a bad news for Apple users as Apple's "damned" compilers miss some important features, for example, OpenMP. Therefore all the compiled packages on CRAN don't have OpenMP enabled on OSX by default. Well, you have probably found that packages such as data.table, fst are using single thread on OSX.

To re-enable such convenient OpenMP feature, you have to install these packages from source and compile them by yourself on the local computer. However, your system's compilers probably miss OpenMP as well. Therefore, the very first thing is to find a compiler that has OpenMP features enabled.

Thanks for the CRAN team, they provided native solutions to the problem. The key is to re-enable OpenMP in your native compilers! This solution does NOT require third-party tools such as brew, and it does not need complicated set-ups. Personally I highly, strongly recommend this approach, especially for new R users.

Detailed steps are listed here:

#242 (comment)

PS: There is also a brew version solution below or somewhere I posted years ago, please do not use it unless you have no choice, or you are familiar with compiler flags. Although it might work in many cases. I often found myself in a situation where I have to change the compiler flags back and forth with brew for different packages because it overwrites the default behaviors of package configurations.

---------------------------------------- Below is the original post ------------------------------------

Hi Mark,

Not sure if this is my own case, but neither fst nor fstcore can detect openmp automatically on mac (R 4.0.0). I have brew installed llvm and libomp, but not symlinked to the path. Apple native clang doesn't support openmp.

I tried install from both source and cran binary. Both versions failed to compile with openmp.

> require(fst)
Loading required package: fst
fst package v0.9.2
(OpenMP was not detected, using single threaded mode)

> require(fstcore)
Loading required package: fstcore
fstcore package v0.9.4
(OpenMP was not detected, using single threaded mode)

[dipterix]

Found in R documentation:

note that some toolchains used for R (including Apple’s for macOS and some others using clang31) have no OpenMP support at all, not even omp.h.

I guess you are using $(SHLIB_OPENMP_CFLAGS) in Makevars. R 4.0 starts to use system tools instead of its own, which provides empty flag on MacOSX as Apple doesn't provide full tool chain unless you brew install it. However, then we need to manually configure it up.

[MarcusKlik]

Hi @dipterix, thanks for reporting! Did you have these problems with earlier versions of R as well? And can package data.table be installed without any problems on your system?

(if so, I might adopt a similar strategy as data.table uses, by using a configure script to detect OpenMP compatibility)

thanks!

[dipterix]

No, data.table also failed. There are two problems:

1. Apple's default tool sets support no openmp. This can be fixed by detecting whether brew has llvm and libomp installed and we can use brew's compilers
2. Since R 4.0.0, CRAN's OSX machine which produces binary repository doesn't have openmp library, this means even your configure files detects openmp on my machine, it I use install.packages('fstcore', type='binary'), you still end up with single core.

CRAN R 4.0.0 builds and higher no longer use any custom compilers and thus this directory is no longer relevant. We now use Apple Xcode 10.1 and GNU Fortran 8.2 from https://github.com/fxcoudert/gfortran-for-macOS/releases. For more details on compiling R, please see also https://mac.R-project.org/tools/

For the first problem, it's easy to solve. I actually managed to install with openmp support by set ting the followings in ~/.R/Makevars and install from source.

CC=/usr/local/opt/llvm/bin/clang
CXX=/usr/local/opt/llvm/bin/clang++
CXX1X=/usr/local/opt/llvm/bin/clang++
LDFLAGS=-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
CXXFLAGS=-I/usr/local/opt/llvm/include

SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp
SHLIB_OPENMP_FCFLAGS= -fopenmp
SHLIB_OPENMP_FFLAGS= -fopenmp

The second problem is hard as it's CRAN's work and it's up to them whether they support it or not.

[MarcusKlik]

Hi @dipterix,

thanks for sharing your exact setup, it's good to read that you got the source installation to work with OpenMP enabled using those flags. And yes indeed, it's very unfortunate that the CRAN binaries only provide single threaded mode, a configure script won't help much there as you say.

Perhaps it would be helpful to OSX users to add the exact installation instructions to the relevant section in the README file (including a link to the data.table wiki for completeness)?

[dipterix]

Hi @MarcusKlik,

You might want to give Mac users some instructions on how to get these environment variable set up in your README. An alternative is to compile the code by yourself and let them install it. For example, I'm using my personal repo, which contains several compiled packages I daily work on. Just compiled your fstcore and tried. It worked great.

install.packages('fstcore', repo = c(
  'CRAN' = 'https://cran.rstudio.com/',
  'dipterix' = 'https://dipterix.github.io/drat/'
))

You can compile your own repo and let Mac users use that one. If you only compile mac version, then it won't affect windows/linux users. They still install from CRAN.

See drat package for more details.

[MarcusKlik]

Hi @dipterix,

that's interesting, so you can effectively create a binary package repository for Mac users and host your own pre-compiled packages there. That might be a solution.

On the other hand, there are also some very good guides available about how to setup a system using R and OpenMP (this one looks very complete for example).

For all these solutions, some instructions in the README are definitely needed.

[sheffe]

Adding a FWIW report here -- I am out of my depth in understanding everything that follows, but it magically worked and might be a useful datapoint.

I was able to get fst built from CRAN with install.packages("fst", type = "source") following @dipterix 's nice instructions above, but not data.table. Installing data.table and fst from source with OpenMP support for both required the following for Makevars:

LDFLAGS= -L/usr/local/clang4/lib
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin16/6.3.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CC=/usr/local/clang4/bin/clang
CXX=/usr/local/gfortran/bin/g++
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++

LLVM_LOC = /usr/local/opt/llvm
CC=/usr/local/gfortran/bin/gcc -fopenmp
CXX=/usr/local/gfortran/bin/g++ -fopenmp
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/usr/local/opt/gettext/lib -L$(LLVM_LOC)/lib -Wl,-rpath,$(LLVM_LOC)/lib
CPPFLAGS=-I/usr/local/opt/gettext/include -I$(LLVM_LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp
SHLIB_OPENMP_FCFLAGS= -fopenmp
SHLIB_OPENMP_FFLAGS= -fopenmp

For context, compiling data.table v1.12.8 and 1.12.9 on MacOS/R4.0 from source with OpenMP support now requires slightly different Makevars parameters from the original instructions -- see (see this thread for details on the changes

I'm using MacOS Mojave 10.14.6, R4.0, fst current CRAN version 0.9.2, data.table Github dev version 1.12.9.

(BTW, fst is one of the fastest-to-adopt packages I've ever used, especially for something so hardwired in my workflow as rds. Amazing work on your end to ensure compatibility and performance.)

[ssh352]

macOS Catalina R 4.0.2 user, can't get openmp to work with fst, even after modifying the Makevars as instructed in this thread:

[dipterix]

@ssh352 You need to install llvm and libomp using brew:
Open Terminal.app, type and enter

brew install llvm
brew install libomp

If you don't have brew, run this command-line to install brew first.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

[mimaque]

I followed the instructions on this link and successfully ran install.packages("fst", type = "source") and have been able to use multithredding with fst.

Of course, you'll need to have the llvm and libomp installs from brew, as well as gcc and gfortran.

I had previously tried the steps above and some other links as well, but kept getting this error: Installation Error: clang: error: unsupported option '-fopenmp'

[eipi10]

I was able to install data.table with OpenMP support using the instructions here (but I changed the gcc version from 7 to 11). I installed fst from source using the same Makevars settings but fst did not detect OpenMP. I then tried using @dipterix's and then @mimaque's Makevars instructions, but fst still did not detect OpenMP with either approach.

I wanted to see if anyone has any other ideas on how to get OpenMP detected by fst.

I'm running R 4.1.2 on a Macbook Pro with Catalina 10.15.7 (and RStudio 2022.02.0-preview+392 in case that matters.)

[dipterix]

@eipi10 The instruction you found might overwrite the makevars found by the package configure files. I don't think it's recommended anymore. I would suggest go with R official's guide.

If you have sudo privilege, try uninstall your fst and use the following steps:

General steps:

1. Go to https://mac.r-project.org/openmp/ , Download the "Official" xcode OpenMP accordingly and extract them using the instructions provided (see below for how to find correct versions and how to download)
2. Edit your ~/.R/Makevars, remove all your current flags, replace with the following two lines:

CPPFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp

How to determine your clang/OpenMP version?

Open your system terminal, type and enter:

clang --version

You will see something like

Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin21.2.0
Thread model: posix
InstalledDir: ...

The version 1300.0.29.30 and darwin21 are the keywords that help you identify the OMP archive version. For example, mine is openmp-13.0.0-darwin21-Release.tar.gz. If you are on darwin20 and clang is 1205.xxx, then it's openmp-12.0.1-darwin20-Release.tar.gz

How to download and extract OMP library

In your system terminal, copy and enter the following commands line-by-line. REMEMBER to edit omp_version using the method above

omp_version="openmp-13.0.0-darwin21-Release.tar.gz"
curl -O "https://mac.r-project.org/openmp/$omp_version"
sudo tar fvxz "$omp_version" -C /
rm "$omp_version"

[eipi10]

Thanks @dipterix. My clang is v13 from Homebrew and my system is x86_64-apple-darwin19.6.0:

clang --version

Homebrew clang version 13.0.1
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

There doesn't seem to be an OpenMP version for OSX and clang 13. Do I need to downgrade to clang 12 and use openmp-10.0.0-darwin17-Release.tar.gz (which is for LLVM 10.0.0; Xcode 12.0-12.4 (Apple clang 1200.x))? Also, does this need to be Apple's version of clang 12 or can I install with Homebrew?

[dipterix]

@eipi10 Wait a second, as of R 4.0, CRAN uses system's native build tools.

(Source: https://cran.r-project.org/bin/macosx/tools/) CRAN R 4.0.0 builds and higher no longer use any custom compilers and thus this directory is no longer relevant. We now use Apple Xcode 10.1 and GNU Fortran 8.2 from https://github.com/fxcoudert/gfortran-for-macOS/releases. For more details on compiling R, please see also https://mac.r-project.org/tools/

So unless asked, the system compilers are most likely to be to best candidate to build R packages. The following command should get you the Apple's build of xcode command-line tool

xcode-select --install

If you run this command and get errors like "command line tools are already installed, use "Software Update" to install updates". It means you've already had xcode command-line tools installed.

If you don't have other software or packages depending on HomeBrew version LLVM, you can uninstall that brew one, or unlink (use system's default)

[tbaldenius]

Thanks @dipterix for pointing out big alterations to the Makevars file are not recommended anymore. Just writing here to inform other users facing problems installing fst with openmp support on Mac: I am on an Intel Mac running macOS 12.2.1 and R 4.1.2, I can now use fst in multi-threaded mode after following @dipterix's instructions above (i.e. installing OpenMP and putting just the mentioned two lines in Makevars).

[pat-s]

My favorite approach is using clang combined with the libomp formula from brew:

1. Install libomp via brew:

brew install libomp

2. Add the followiong options to ~/.R/Makevars:

LDFLAGS += -L/opt/homebrew/opt/libomp/lib -lomp
CPPFLAGS += -Xclang -fopenmp

3. Now data.table and fst can be installed from source

install.packages("data.table", type = "source")
[...]

clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include -I/opt/homebrew/include -I/opt/homebrew/opt/sqlite/include -Xclang -fopenmp    -fPIC  -I/opt/homebrew/include -I/opt/homebrew/opt/sqlite/include -Xclang -fopenmp -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -L/opt/homebrew/opt/jpeg/lib -L/opt/homebrew/opt/libpng/lib -L/opt/homebrew/opt/pcre2/lib -L/opt/homebrew/opt/sqlite/lib -L/opt/homebrew/opt/libomp/lib -lomp -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o ijoin.o init.o inrange.o nafill.o nqrecreateindices.o openmp-utils.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
PKG_CFLAGS =
PKG_LIBS = -lz

library(data.table)
data.table 1.14.2 using 4 threads (see ?getDTthreads).  Latest news: r-datatable.com

^{Created on 2022-02-28 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.1.2 (2021-11-01)
#>  os       macOS Monterey 12.2.1
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Europe/Zurich
#>  date     2022-02-28
#>  pandoc   2.17.1.1 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  backports     1.4.1      2021-12-13 [1] CRAN (R 4.1.1)
#>  cli           3.2.0      2022-02-14 [1] CRAN (R 4.1.1)
#>  crayon        1.5.0      2022-02-14 [1] CRAN (R 4.1.2)
#>  data.table  * 1.14.2     2021-09-27 [1] CRAN (R 4.1.2)
#>  digest        0.6.29     2021-12-01 [1] CRAN (R 4.1.2)
#>  ellipsis      0.3.2      2021-04-29 [1] CRAN (R 4.1.2)
#>  evaluate      0.15       2022-02-18 [1] CRAN (R 4.1.2)
#>  fansi         1.0.2      2022-01-14 [1] CRAN (R 4.1.2)
#>  fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.1.2)
#>  fs            1.5.2      2021-12-08 [1] CRAN (R 4.1.1)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.1.2)
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.1.2)
#>  htmltools     0.5.2      2021-08-25 [1] CRAN (R 4.1.2)
#>  knitr         1.37       2021-12-16 [1] CRAN (R 4.1.1)
#>  lifecycle     1.0.1      2021-09-24 [1] CRAN (R 4.1.2)
#>  magrittr      2.0.2      2022-01-26 [1] CRAN (R 4.1.2)
#>  pillar        1.7.0      2022-02-01 [1] CRAN (R 4.1.2)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.1.2)
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.1.2)
#>  R.cache       0.15.0     2021-04-30 [1] CRAN (R 4.1.2)
#>  R.methodsS3   1.8.1      2020-08-26 [1] CRAN (R 4.1.2)
#>  R.oo          1.24.0     2020-08-26 [1] CRAN (R 4.1.2)
#>  R.utils       2.11.0     2021-09-26 [1] CRAN (R 4.1.2)
#>  reprex        2.0.1      2021-08-05 [1] CRAN (R 4.1.2)
#>  rlang         1.0.1.9000 2022-02-28 [1] Github (r-lib/rlang@a5a6d82)
#>  rmarkdown     2.11       2021-09-14 [1] CRAN (R 4.1.2)
#>  rstudioapi    0.13       2020-11-12 [1] CRAN (R 4.1.2)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.1.2)
#>  stringi       1.7.6      2021-11-29 [1] CRAN (R 4.1.2)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.1.2)
#>  styler        1.6.2      2021-09-23 [1] CRAN (R 4.1.1)
#>  tibble        3.1.6      2021-11-07 [1] CRAN (R 4.1.2)
#>  utf8          1.2.2      2021-07-24 [1] CRAN (R 4.1.2)
#>  vctrs         0.3.8      2021-04-29 [1] CRAN (R 4.1.2)
#>  withr         2.4.3      2021-11-30 [1] CRAN (R 4.1.2)
#>  xfun          0.29       2021-12-14 [1] CRAN (R 4.1.1)
#>  yaml          2.3.5      2022-02-21 [1] CRAN (R 4.1.2)
#> 
#>  [1] /Users/pjs/Library/R/arm64/4.1/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[eipi10]

@dipterix instructions worked for me on MacOS 10.15.7. I used [openmp-12.0.1-darwin20-Release.tar.gz](https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz) even though my system is darwin19, because my clang version is 13.0.1. Thanks!

[jzadra]

@eipi10 The instruction you found might overwrite the makevars found by the package configure files. I don't think it's recommended anymore. I would suggest go with R official's guide.

If you have sudo privilege, try uninstall your fst and use the following steps:

General steps:

1. Go to https://mac.r-project.org/openmp/ , Download the "Official" xcode OpenMP accordingly and extract them using the instructions provided (see below for how to find correct versions and how to download)
2. Edit your ~/.R/Makevars, remove all your current flags, replace with the following two lines:

CPPFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp

How to determine your clang/OpenMP version?

Open your system terminal, type and enter:

clang --version

You will see something like

Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin21.2.0
Thread model: posix
InstalledDir: ...

The version 1300.0.29.30 and darwin21 are the keywords that help you identify the OMP archive version. For example, mine is openmp-13.0.0-darwin21-Release.tar.gz. If you are on darwin20 and clang is 1205.xxx, then it's openmp-12.0.1-darwin20-Release.tar.gz

How to download and extract OMP library

In your system terminal, copy and enter the following commands line-by-line. REMEMBER to edit omp_version using the method above

omp_version="openmp-13.0.0-darwin21-Release.tar.gz"
curl -O "https://mac.r-project.org/openmp/$omp_version"
sudo tar fvxz "$omp_version" -C /
rm "$omp_version"

I've followed these steps, but I'm still getting the OpenMP was not detected.

I checked my clang --version:

Apple clang version 13.1.6 (clang-1316.0.21.2)
Target: arm64-apple-darwin21.4.0

And then used the following code in terminal:

omp_version="openmp-13.0.0-darwin21-Release.tar.gz"
curl -O "https://mac.r-project.org/openmp/$omp_version"
sudo tar fvxz "$omp_version" -C /
rm "$omp_version"

I then added this to my R Makevars (there wasn't anything in it before):

CPPFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp

I restarted R, then installed fst from source:

install.packages("fst", type = "source")

but I get an error during the process:

trying URL 'https://cran.rstudio.com/src/contrib/fst_0.9.8.tar.gz'
Content type 'application/x-gzip' length 274495 bytes (268 KB)
==================================================
downloaded 268 KB

* installing *source* package ‘fst’ ...
** package ‘fst’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/fstcore/include' -I/usr/local/include -Xclang -fopenmp   -fPIC  -Wall -g -O2  -c RcppExports.cpp -o RcppExports.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/fstcore/include' -I/usr/local/include -Xclang -fopenmp   -fPIC  -Wall -g -O2  -c flex_store.cpp -o flex_store.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/fstcore/include' -I/usr/local/include -Xclang -fopenmp   -fPIC  -Wall -g -O2  -c fst_compress.cpp -o fst_compress.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/fstcore/include' -I/usr/local/include -Xclang -fopenmp   -fPIC  -Wall -g -O2  -c openmp.cpp -o openmp.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -lomp -o fst.so RcppExports.o flex_store.o fst_compress.o openmp.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: dylib (/usr/local/lib/libomp.dylib) was built for newer macOS version (12.0) than being linked (10.13)
installing to /Library/Frameworks/R.framework/Versions/4.1/Resources/library/00LOCK-fst/00new/fst/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (fst)

I'm on MacOS 12.3.

[dipterix]

Hi @jzadra , This is because the author has split fst into fst and fstcore.

You might want to install fstcore using omp because all the omp components are in that package, fst is just a wrapper now.

install.packages("fstcore", type = "source")

# Restart your R session
fstcore::threads_fstlib()

[jzadra]

That did the trick, thanks!