Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configure: test linking with -fopenmp, test the resulting shared object #6642

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

aitap
Copy link
Contributor

@aitap aitap commented Dec 9, 2024

  • When testing various OpenMP-related flags, dyn.load() the resulting shared object and run code from it before deciding to use these flags for data.table
  • When testing -fopenmp, add it to both compiler and linker flags

Hopefully, this will fix #6622 and prevent similar problems in the future. A more conservative fix is #6633.

Upsides:

  • A configuration that successfully compiles but later doesn't work is much less likely to be chosen
    • Instead of compiling a shared object that fails to load, data.table will install without OpenMP support
  • At least on GNU/Linux and (seemingly) @dvg-p4's Homebrew toolchain, -fopenmp is required in both compiler and linker flags

Downsides:

  • If something makes omp_get_max_threads() return a number ≤ 0 during data.table installation, the build process will consider OpenMP not working and disable it.
    • On my GNU/Linux setup, setting environment variables OMP_NUM_THREADS=0 or OMP_THREAD_LIMIT=0 results in warnings, but omp_get_max_threads() remains positive.
  • -fopenmp is now tested unconditionally, not limited to uname %in% c('Linux', 'Darwin'). Something that previously built single-threaded data.table may now enable OpenMP and surprise someone.
  • configure now runs an R script. This can be rewritten back into a shell script if we can guarantee mktemp -d available on everything that runs data.table.
  • Only tested on GNU/Linux. Given enough free time, can test on FreeBSD, OpenBSD VMs, but not a Mac.

aitap added 4 commits December 5, 2024 16:01
* Test loading the resulting shared objects; make sure they return an
  above-zero number of threads
* Test -fopenmp on all Unix-like operating systems
* Make sure to include user-supplied PKG_{CFLAGS,LIBS}
* Be consistent about calling the linker flags LIBS, not LDFLAGS
@aitap aitap requested a review from MichaelChirico as a code owner December 9, 2024 08:05
Copy link

codecov bot commented Dec 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.61%. Comparing base (a36caac) to head (4bd336f).
Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6642   +/-   ##
=======================================
  Coverage   98.61%   98.61%           
=======================================
  Files          79       79           
  Lines       14533    14558   +25     
=======================================
+ Hits        14332    14357   +25     
  Misses        201      201           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Dec 9, 2024

Comparison Plot

Generated via commit 4bd336f

Download link for the artifact containing the test results: ↓ atime-results.zip

Task Duration
R setup and installing dependencies 4 minutes and 51 seconds
Installing different package versions 8 minutes and 2 seconds
Running and plotting the test cases 2 minutes and 31 seconds

@dvg-p4
Copy link
Contributor

dvg-p4 commented Dec 9, 2024

Works with my R4.2 installation on Mac.

Doesn't work on my R4.4 or 4.3 installations (nothing I've tried so far has), still yields symbol not found in flat namespace '___kmpc_dispatch_deinit' when the installation tests run.

Full output attempting to install on R4.4.1

> install.packages('.', type = 'source', repos = NULL)
* installing *source* package ‘data.table’ ...
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation is configured to use OpenMP... no
* checking if R installation supports OpenMP with "-fopenmp" flag... yes
** libs
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
PKG_CFLAGS = -fopenmp
PKG_LIBS = -fopenmp -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_dispatch_deinit'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
Warning message:
In install.packages(".", type = "source", repos = NULL) :
  installation of package ‘.’ had non-zero exit status

@aitap
Copy link
Contributor Author

aitap commented Dec 9, 2024

> install.packages('.', type = 'source', repos = NULL)

<...>

using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

That's much fewer object files being re-compiled than linked into the final shared object. The danger and the usefulness of being able to run install.packages() on a directory is that it's not cleaned afterwards. Could you please run rm src/*.o src/*.so src/Makevars (git clean -fx may also work if you don't have anything valuable in the repository clone), then install.packages() again, and then show me the installation output and the contents of config.log?

A comparison with the build output from R-4.2 might also help.

@dvg-p4
Copy link
Contributor

dvg-p4 commented Dec 9, 2024

Clean R4.2 build (successful)

R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages('.', type = 'source', repos = NULL)
Installing package into ‘/Users/dgealow/Library/R/arm64/4.2/library’
(as ‘lib’ is unspecified)
* installing *source* package ‘data.table’ ...
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation is configured to use OpenMP... no
* checking if R installation supports OpenMP with "-fopenmp" flag... yes
** libs
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c assign.c -o assign.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c bmerge.c -o bmerge.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c chmatch.c -o chmatch.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c cj.c -o cj.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c coalesce.c -o coalesce.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c dogroups.c -o dogroups.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fastmean.c -o fastmean.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fcast.c -o fcast.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fifelse.c -o fifelse.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fmelt.c -o fmelt.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frank.c -o frank.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fread.c -o fread.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c freadR.c -o freadR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c froll.c -o froll.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frolladaptive.c -o frolladaptive.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fsort.c -o fsort.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwrite.c -o fwrite.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwriteR.c -o fwriteR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c gsumm.c -o gsumm.o
gsumm.c:1177:38: warning: implicit conversion from 'long long' to 'long double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
 1177 |       ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_INTEGER64 : (int64_t)s[i];
      |                                     ~^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: expanded from macro 'INT64_MAX'
   94 | #define INT64_MAX        9223372036854775807LL
      |                          ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c idatetime.c -o idatetime.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c ijoin.c -o ijoin.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c inrange.c -o inrange.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nafill.c -o nafill.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c negate.c -o negate.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nqrecreateindices.c -o nqrecreateindices.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c openmp-utils.c -o openmp-utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c programming.c -o programming.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c quickselect.c -o quickselect.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c reorder.c -o reorder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c shift.c -o shift.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c snprintf.c -o snprintf.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c subset.c -o subset.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c transpose.c -o transpose.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c types.c -o types.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c uniqlist.c -o uniqlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c vecseq.c -o vecseq.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
PKG_CFLAGS = -fopenmp
PKG_LIBS = -fopenmp -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (data.table)
> library(data.table)
data.table 1.16.99 IN DEVELOPMENT built 2024-12-09 15:44:49 UTC using 5 threads (see ?getDTthreads).  Latest news: r-datatable.com

Clean R4.4 build (failure)

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages('.', type = 'source', repos = NULL)
* installing *source* package ‘data.table’ ...
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation is configured to use OpenMP... no
* checking if R installation supports OpenMP with "-fopenmp" flag... yes
** libs
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c assign.c -o assign.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c bmerge.c -o bmerge.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c chmatch.c -o chmatch.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c cj.c -o cj.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c coalesce.c -o coalesce.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c dogroups.c -o dogroups.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fastmean.c -o fastmean.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fcast.c -o fcast.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fifelse.c -o fifelse.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fmelt.c -o fmelt.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frank.c -o frank.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fread.c -o fread.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c freadR.c -o freadR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c froll.c -o froll.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frolladaptive.c -o frolladaptive.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fsort.c -o fsort.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwrite.c -o fwrite.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwriteR.c -o fwriteR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c gsumm.c -o gsumm.o
gsumm.c:1177:38: warning: implicit conversion from 'long long' to 'long double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
 1177 |       ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_INTEGER64 : (int64_t)s[i];
      |                                     ~^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: expanded from macro 'INT64_MAX'
   94 | #define INT64_MAX        9223372036854775807LL
      |                          ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c idatetime.c -o idatetime.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c ijoin.c -o ijoin.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c inrange.c -o inrange.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nafill.c -o nafill.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c negate.c -o negate.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nqrecreateindices.c -o nqrecreateindices.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c openmp-utils.c -o openmp-utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c programming.c -o programming.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c quickselect.c -o quickselect.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c reorder.c -o reorder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c shift.c -o shift.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c snprintf.c -o snprintf.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c subset.c -o subset.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c transpose.c -o transpose.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c types.c -o types.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c uniqlist.c -o uniqlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c vecseq.c -o vecseq.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
PKG_CFLAGS = -fopenmp
PKG_LIBS = -fopenmp -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_dispatch_deinit'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
Warning message:
In install.packages(".", type = "source", repos = NULL) :
  installation of package ‘.’ had non-zero exit status

@dvg-p4
Copy link
Contributor

dvg-p4 commented Dec 9, 2024

R 4.2 config.log

0.29.2
3
Testing if OpenMP works with CFLAGS=' $(SHLIB_OPENMP_CFLAGS)' and LIBS=' $(SHLIB_OPENMP_CFLAGS)':
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
Result: no
Testing if OpenMP works with CFLAGS=' -fopenmp' and LIBS=' -fopenmp':
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
Result: yes

R 4.4 config.log

0.29.2
3
Testing if OpenMP works with CFLAGS=' $(SHLIB_OPENMP_CFLAGS)' and LIBS=' $(SHLIB_OPENMP_CFLAGS)':
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Result: no
Testing if OpenMP works with CFLAGS=' -fopenmp' and LIBS=' -fopenmp':
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Result: yes

It turns out it's possible for omp_get_num_threads() to succeed while
failing to load the shared library that does more involved OpenMP
operations.
@aitap
Copy link
Contributor Author

aitap commented Dec 10, 2024 via email

@dvg-p4
Copy link
Contributor

dvg-p4 commented Dec 10, 2024

Have you got two different arm64 Macs with different R versions on
them? Otherwise it's a bit surprising to see the same include and
library paths on the compiler command lines.

Nope, this is all on the same physical machine.

% ls -l /Library/Frameworks/R.framework/Versions 
total 0
drwxrwxr-x  6 root     admin  192 Jan 10  2023 3.6
drwxrwxr-x  6 root     admin  192 Jan 10  2023 4.1-arm64
drwxrwxr-x  6 root     admin  192 Apr 26  2023 4.2-arm64
drwxrwxr-x  6 root     admin  192 Jul 15 11:35 4.3-arm64
drwxrwxr-x  6 root     admin  192 Jul 15 11:37 4.4-arm64
lrwxr-xr-x@ 1 dgealow  admin   50 Dec 10 10:23 Current -> /Library/Frameworks/R.framework/Versions/4.4-arm64

Is it a CRAN build of R or the Homebrew build of R?

Pretty sure I installed all of these from the CRAN MacOS installers

@dvg-p4
Copy link
Contributor

dvg-p4 commented Dec 10, 2024

No dice with the new commit. Same error.

Output and log:

> install.packages('.', type = 'source', repos = NULL)
* installing *source* package ‘data.table’ ...
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation is configured to use OpenMP... no
* checking if R installation supports OpenMP with "-fopenmp" flag... yes
** libs
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c assign.c -o assign.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c bmerge.c -o bmerge.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c chmatch.c -o chmatch.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c cj.c -o cj.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c coalesce.c -o coalesce.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c dogroups.c -o dogroups.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fastmean.c -o fastmean.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fcast.c -o fcast.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fifelse.c -o fifelse.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fmelt.c -o fmelt.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frank.c -o frank.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fread.c -o fread.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c freadR.c -o freadR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c froll.c -o froll.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c frolladaptive.c -o frolladaptive.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fsort.c -o fsort.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwrite.c -o fwrite.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c fwriteR.c -o fwriteR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c gsumm.c -o gsumm.o
gsumm.c:1177:38: warning: implicit conversion from
      'long long' to 'long double' changes value from 9223372036854775807 to
      9223372036854775808 [-Wimplicit-const-int-float-conversion]
 1177 |       ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_...
      |                                     ~^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: 
      expanded from macro 'INT64_MAX'
   94 | #define INT64_MAX        9223372036854775807LL
      |                          ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c idatetime.c -o idatetime.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c ijoin.c -o ijoin.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c inrange.c -o inrange.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nafill.c -o nafill.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c negate.c -o negate.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c nqrecreateindices.c -o nqrecreateindices.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c openmp-utils.c -o openmp-utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c programming.c -o programming.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c quickselect.c -o quickselect.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c reorder.c -o reorder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c shift.c -o shift.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c snprintf.c -o snprintf.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c subset.c -o subset.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c transpose.c -o transpose.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c types.c -o types.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c uniqlist.c -o uniqlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c vecseq.c -o vecseq.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp  -fPIC  -falign-functions=64 -Wall -g -O2  -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -fopenmp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
PKG_CFLAGS = -fopenmp
PKG_LIBS = -fopenmp -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_dispatch_deinit'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’
Warning message:
In install.packages(".", type = "source", repos = NULL) :
  installation of package ‘.’ had non-zero exit status
> 
Save workspace image? [y/n/c]: n
dgealow@Dans-P4-MacBook-Pro data.table % cat config.log
0.29.2
3
Testing if OpenMP works with CFLAGS=' $(SHLIB_OPENMP_CFLAGS)' and LIBS=' $(SHLIB_OPENMP_CFLAGS)':
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Test result: 0 (must be 3; should be 0 for disabled OpenMP)
Error: Test failed
Execution halted
Testing if OpenMP works with CFLAGS=' -fopenmp' and LIBS=' -fopenmp':
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fopenmp -fPIC  -falign-functions=64 -Wall -g -O2  -c test.c -o test.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o test.so test.o -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Test result: 3 (must be 3; should be 0 for disabled OpenMP)
Success

@aitap aitap marked this pull request as draft December 11, 2024 10:07
@aitap
Copy link
Contributor Author

aitap commented Dec 11, 2024

Thanks again!

Part of the problem may be that R.framework version 4.4 comes with libomp.dylib while 4.2 doesn't. R-4.4 says your toolchain comes from Homebrew; perhaps Homebrew has its own OpenMP runtime that is incompatible with the one brought in by R? You may have better chances if you use R-installed-from-Homebrew with the Homebrew toolchain.

I'm currently out of ideas how to detect configurations like this one that seem to compile, link, load and perform OpenMP operations during ./configure, and later fail as part of a larger R package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

data.table installation from source fails on Mac, symbol not found in flat namespace '___kmpc_barrier'
2 participants