"error while loading shared libraries: libmysqlclient.so.18" when I try to run the miniExample

[sz-1002]

Hi,

I am trying to see if cesar2 works correctly on my computer, and encountered the following error with the mini example. I was wondering if you could help? Thank you so much!

Running "annotateGenesViaCESAR.pl POLR3K hg38_oryAfe1.bb twoGenes.gp.forCESAR hg38 oryAfe1 CESARoutput 2bitDir $profilePath -maxMemory 1" gave me the following error:

Processing gene 'POLR3K'
twoBitToFa: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory
substr outside of string at /home/sidi/programs/CESAR2.0/tools/annotateGenesViaCESAR.pl line 169.
Use of uninitialized value $seq in uc at /home/sidi/programs/CESAR2.0/tools/annotateGenesViaCESAR.pl line 750.
mafSpeciesSubset: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory
Error running '/bin/bash -c 'set -o pipefail; mafExtract -region=chr16:53475-53586 hg38_oryAfe1.bb stdout|mafSpeciesSubset stdin NULL /dev/shm/exon.maf.Rwsm2jQ1w -speciesList=oryAfe1,hg38''

I already installed libmysqlclient by:
sudo apt-get install libmysqlclient-dev

libmysqlclient.so is in the following path, and "/usr/lib/x86_64-linux-gnu" is in my $LD_LIBRARY_PATH. I am on Ubuntu 20.04 LTS system.
/usr/lib/x86_64-linux-gnu/libmysqlclient.so

(I'm not sure if this is related, but I did not compile kent from source and I assume the command above is using the pre-compiled version in tools?)

Thank you!!

[MichaelHiller]

Pls try to install
apt-get install libmysql-dev
or
apt-get install default-libmysqlclient-dev

[sz-1002]

Thanks for the reply!

I tried apt-get install libmysql-dev and apt-get install default-libmysqlclient-dev (the latter installed successfully it seems; the former gave me "E: Unable to locate package libmysql-dev").

But, I'm still seeing the same error:

Processing gene 'POLR3K'
twoBitToFa: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory
substr outside of string at /home/sidi/programs/CESAR2.0/tools/annotateGenesViaCESAR.pl line 169.
Use of uninitialized value $seq in uc at /home/sidi/programs/CESAR2.0/tools/annotateGenesViaCESAR.pl line 750.
mafSpeciesSubset: error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory
Error running '/bin/bash -c 'set -o pipefail; mafExtract -region=chr16:53475-53586 hg38_oryAfe1.bb stdout|mafSpeciesSubset stdin NULL /dev/shm/exon.maf.LglZAebLb -speciesList=oryAfe1,hg38''

Is this maybe a problem with the version of libmysqlclient.so? There is libmysqlclient.so.21 in my system.
ls -l -h /usr/lib/x86_64-linux-gnu/libmysqlclient*
-rw-r--r-- 1 root root 9.4M 7月 27 01:14 /usr/lib/x86_64-linux-gnu/libmysqlclient.a
lrwxrwxrwx 1 root root 20 7月 27 01:14 /usr/lib/x86_64-linux-gnu/libmysqlclient.so -> libmysqlclient.so.21
lrwxrwxrwx 1 root root 25 7月 27 01:14 /usr/lib/x86_64-linux-gnu/libmysqlclient.so.21 -> libmysqlclient.so.21.2.30
-rw-r--r-- 1 root root 7.3M 7月 27 01:14 /usr/lib/x86_64-linux-gnu/libmysqlclient.so.21.2.30

[MichaelHiller]

Looks like mysql is still not properly installed on your system. The kent src tools need that.
I asked our admin who said
"the precompiled kent tools on github were compiled using dynamic linking, that means that the user requires the same library versions on his computer.

For ubuntu the required library version for libmysqlclient can be downloaded here:
https://launchpad.net/ubuntu/xenial/amd64/libmysqlclient18/5.6.25-0ubuntu1

(multiarch package has to be installed prior to this)
http://archive.ubuntu.com/ubuntu/pool/main/g/glibc/multiarch-support_2.27-3ubuntu1.5_amd64.deb

But there might be more libraries missing...
The best way would be compiling kent tools on the target machine

Another way would be using static linking on our kent compiles (gcc -static)
https://opensource.com/article/22/6/static-linking-linux
Not sure about the side effects (besides the larger binary size)
"

Hope that helps. Essentially, you need to be able to install kent src.

[sz-1002]

Thank you very much for your reply!

Indeed, after manually installing libmysqlclient from the link you provided, annotateGenesViaCESAR.pl keeps complaining that more libraries are missing...

So, I decided to compile kent, and it went okay after I uninstalled Anaconda and switched to system gcc. (For some reason, ld in Anaconda cannot find lz and lcrypto, but system ld could, and I haven't figured out why).

Now, I am able to run annotateGenesViaCESAR.pl POLR3K hg38_oryAfe1.bb twoGenes.gp.forCESAR hg38 oryAfe1 CESARoutput 2bitDir $profilePath -maxMemory 1 and annotateGenesViaCESAR.pl SNRNP25 hg38_oryAfe1.bb twoGenes.gp.forCESAR hg38 oryAfe1 CESARoutput 2bitDir $profilePath -maxMemory 1 without error.

But, bed2GenePred.pl oryAfe1 CESARoutput oryAfe1.gp gave me:

executing  'find CESARoutput -type f -path '*/oryAfe1/*' -exec cat {} \;'  ...
gives 8 exons
overlapSelect: /lib/x86_64-linux-gnu/libssl.so.10: version libssl.so.10' not found (required by overlapSelect)
overlapSelect: /lib/x86_64-linux-gnu/libcrypto.so.10: version `libcrypto.so.10' not found (required by overlapSelect)
ERROR!! 'overlapSelect /dev/shm/HOuvs.bed /dev/shm/HOuvs.bed stdout -mergeOutput' failed in filterOverlappingExons function

This is after I followed https://askubuntu.com/questions/339364/libssl-so-10-cannot-open-shared-object-file-no-such-file-or-directory and installed libssl1.0.0, libssl-dev and created the links:

sudo ln -s libssl.so.1.0.0 libssl.so.10
sudo ln -s libcrypto.so.1.0.0 libcrypto.so.10

So my question now is:
Do you happen to know how to get libssl.so.10 properly?

Thank you and sorry for a lot of questions!

[MichaelHiller]

Good to hear that there is progress.
We don't have Ubuntu, so all I can do is google the error messages (lib *** not found), which brings
sudo apt-get install libssl1.0.0 libssl-dev
and likely something similar to crypto.

Pls ask your admin to install these packages.

[alejandrogzi]

Hi guys,

I have the same exact problem with the same libraries. Making symlinks as describe above help me out but then libpng15.so started failing. After finding a solution for the latter, libssl and crypto failed again.

Just wanted to address the problem. I am using Ubuntu 20.04. Hope to find a solution quickly, I am very interested in using CESAR.

Alejandro

[MichaelHiller]

Sorry to hear that running it on Ubuntu is apparently not easy. Hard for me to debug as we are having CentOS.
These libs are needed by UCSC kent src tools and libs, which we need. These tools are also useful for a lot of genomic tasks.

I found this recipe for how to install UCSC kent on Ubuntu. http://genomewiki.ucsc.edu/index.php/Source_tree_compilation_on_Debian/Ubuntu

Pls try that and also ask your sysadmins to check whether your system and these libs are correctly installed.
Pls post how this is going.
Thx

[sz-1002]

Hi!

Sorry for the late reply. I tried compiling openssl from source (https://www.openssl.org/source/old/1.0.0/) and that seems to solve the problem for me. (For some unknown reason, the version installed from apt-get which I used earlier does not work properly).

Now, bed2GenePred.pl oryAfe1 CESARoutput oryAfe1.gp gives me:

executing  'find CESARoutput -type f -path '*/oryAfe1/*' -exec cat {} \;'  ...
gives 8 exons
overlapSelect: /lib/x86_64-linux-gnu/libssl.so.10: no version information available (required by overlapSelect)
overlapSelect: /lib/x86_64-linux-gnu/libcrypto.so.10: no version information available (required by overlapSelect)
overlapSelect: /lib/x86_64-linux-gnu/libssl.so.10: no version information available (required by overlapSelect)
overlapSelect: /lib/x86_64-linux-gnu/libcrypto.so.10: no version information available (required by overlapSelect)
Results for oryAfe1 are in oryAfe1.gp

I assume the "no version information available"s are just warnings, as oryAfe1.gp is generated normally.

Thank you so much for your help!

[MichaelHiller]

overlapSelect is a UCSC tool and the libssl / libcrypto errors indicate that these 2 libs are still not properly installed on your system (did you recompile the tools after installing libssl / libcrypto?)

[alejandrogzi]

I have not been able to solve the problem, even compiling the UCSC kent code. I will keep trying.

Michael, I also wanted to ask you - in the first open issue here in CESAR´s github you pointed that TOGA has CESAR2.0 implemented, and it can be run through a Docker container - it is possible to run only CESAR2.0 with TOGA? I asked this in TOGA´s git but haven´t receive a response yet.

Regards,
Alejandro

[MichaelHiller]

Yes, absolutely. Pls try the TOGA container (I think we had users successfully running it on Ubuntu).
TOGA also provides much more than CESAR such as gene loss classification, orthology inference, etc

[sz-1002]

Hmmm I recompiled kent, but the "no version information available (required by overlapSelect)" message doesn't go away.
But TOGA installation seems to be working.

Thank you very much!

[MichaelHiller]

Where is the "no version information available" error coming from? From which tool? I'll have a look.
But good to know that TOGA works.
Thx

[sz-1002]

It is the same error from overlapSelect. I searched online a little bit and it seems that the Ubuntu version of openssl should always have the version information, but maybe the one I downloaded and compiled does not...not sure about this.

Thank you!

bed2GenePred.pl oryAfe1 CESARoutput oryAfe1.gp

executing  'find CESARoutput -type f -path '*/oryAfe1/*' -exec cat {} \;'  ...
gives 8 exons
overlapSelect: /usr/local/ssl/lib/libssl.so.10: no version information available (required by overlapSelect)
overlapSelect: /usr/local/ssl/lib/libcrypto.so.10: no version information available (required by overlapSelect)
overlapSelect: /usr/local/ssl/lib/libssl.so.10: no version information available (required by overlapSelect)
overlapSelect: /usr/local/ssl/lib/libcrypto.so.10: no version information available (required by overlapSelect)
Results for oryAfe1 are in oryAfe1.gp

[kirilenkobm]

Hi @alejandrogzi @sz-1002

Regarding UCSC binaries: pls try to download precompiled binaries from
https://hgdownload.cse.ucsc.edu/admin/exe/
Alternatively, some binaries can be installed using conda (such as ucsc-faToTwoBit and others).

I used this strategy to install dependencies for https://github.com/hillerlab/make_lastz_chains because this strategy seemed to be the most reproducible given that UCSC binaries are indeed very hard to compile.

[sz-1002]

Thank you, this is indeed very helpful!

After replacing overlapSelect in CESAR2.0/tools with the precompiled binary, bed2GenePred.pl oryAfe1 CESARoutput oryAfe1.gp finishes without complaints:

executing  'find CESARoutput -type f -path '*/oryAfe1/*' -exec cat {} \;'  ...
gives 8 exons
Results for oryAfe1 are in oryAfe1.gp