make test fails on macOS Catalina: Library not loaded: @rpath/libmysqlclient.21.dylib

[hakonhagland]

I am on macOS 10.15.3, perlbrew with perl version 5.30. I first installed mysql from

https://dev.mysql.com/downloads/mysql/

Downloaded disk image file: mysql-8.0.19-macos10.15-x84_64.dmg
After the installation I added to my ~/.zshrc

 export PATH="$PATH:/usr/local/mysql-8.0.19-macos10.15-x86_64/bin"
export LD_LIBRARY_PATH=/usr/local/mysql-8.0.19-macos10.15-x86_64/lib
export DYLD_LIBRARY_PATH=/usr/local/mysql-8.0.19-macos10.15-x86_64/lib

Then I ran perl Makefile.PL; make; make test:

"/Users/hakonhaegland/perl5/perlbrew/perls/perl-5.30.1/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- mysql.bs blib/arch/auto/DBD/mysql/mysql.bs 644
PERL_DL_NONLAZY=1 "/Users/hakonhaegland/perl5/perlbrew/perls/perl-5.30.1/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00base.t .............................. 1/6
#   Failed test 'use DBD::mysql;'
#   at t/00base.t line 15.
#     Tried to use 'DBD::mysql'.
#     Error:  Can't load '/Users/hakonhaegland/perl/DBD-mysql-4.050/blib/arch/auto/DBD/mysql/mysql.bundle' for module DBD::mysql: dlopen(/Users/hakonhaegland/perl/DBD-mysql-4.050/blib/arch/auto/DBD/mysql/mysql.bundle, 2): Library not loaded: @rpath/libmysqlclient.21.dylib
#   Referenced from: /Users/hakonhaegland/perl/DBD-mysql-4.050/blib/arch/auto/DBD/mysql/mysql.bundle
#   Reason: image not found at /Users/hakonhaegland/perl5/perlbrew/perls/perl-5.30.1/lib/5.30.1/darwin-2level/DynaLoader.pm line 197.
#  at t/00base.t line 15.
# Compilation failed in require at t/00base.t line 15.
# BEGIN failed--compilation aborted at t/00base.t line 15.
Bailout called.  Further testing stopped:  Unable to load DBD::mysql
FAILED--Further testing stopped: Unable to load DBD::mysql
make: *** [test_dynamic] Error 255

It is strange that it cannot find libmysqlclient.21.dylib since I set LD_LIBRARY_PATH and the directory indeed has the library:

$ ls /usr/local/mysql-8.0.19-macos10.15-x86_64/lib
libcrypto.1.1.dylib          libmysqlclient.a             libprotobuf-lite.3.6.1.dylib libprotobuf.dylib            mecab
libcrypto.dylib              libmysqlclient.dylib         libprotobuf-lite.dylib       libssl.1.1.dylib             pkgconfig
libmysqlclient.21.dylib      libmysqlservices.a           libprotobuf.3.6.1.dylib      libssl.dylib                 plugin

[hakonhagland]

The problem seems to be that DYLD_LIBRARY_PATH is not inherited by the generated Makefile.
On line 1222 of the generated Makefile the test target is defined as:

test_dynamic :: subdirs-test_dynamic
	PERL_DL_NONLAZY=1 $(FULLPERLRUN) "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness($(TEST_VERBOSE), '$(INST_LIB)', '$(INST_ARCHLIB)')" $(TEST_FILES)

where $FULLPERLRUN is the path to the perl executable, in my case

/Users/hakonhaegland/perl5/perlbrew/perls/perl-5.30.1/bin/perl

This command does not inherit DYLD_LIBRARY_PATH even if I have exported it in the shell where I run make test from. It can be confirmed by changing the given line in the Makefile to include DYLD_LIBRARY_PATH as follows:

test_dynamic :: subdirs-test_dynamic
	DYLD_LIBRARY_PATH=/usr/local/mysql-8.0.19-macos10.15-x86_64/lib PERL_DL_NONLAZY=1 $(FULLPERLRUN) "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness($(TEST_VERBOSE), '$(INST_LIB)', '$(INST_ARCHLIB)')" $(TEST_FILES)

Running make test on this modified Makefile does not produce the error with missing libraries as given above in my first post.

[etlflow]

@hakonhagland were you able to find a resolution? I am facing the same problem. I'm on macOS 10.15.6.

[hakonhagland]

@ra1fee Apparently it is a security measure introduced in newer versions of macOS, see this post for more information.

[hakonhagland]

I think we should try to find a fix for this, more people are running into this. See for example this issue

[cpeng13]

@hakonhagland is there any update regarding this issue? I am encountering the same issue now (on M1 MacBook Pro, macOS Big Sur)

[hakonhagland]

@cpeng13 I have added a PR to ExtUtils::MakeMaker, see Perl-Toolchain-Gang/ExtUtils-MakeMaker#403. But unfortunately when I tested that PR with this module (DBD::mysql) it does not seem to work for some reason. I am looking into what can be the problem here.

[hakonhagland]

@cpeng13 Seems to me like a problem with the mysql_config script with is used to determine which libraries to link with. I have reported the issue here. As a quick fix you can do:

$ cd $(mysql_config --variable=pkglibdir)/../bin
$ perl -pi -E 'if (/^libs=.*-lssl/) { s/-lssl//; s/-lcrypto//; s/-lresolv//; s/ \s+/ /; s/\s+\x22$/\x22/ }' mysql_config

to patch mysql_config