Cannot find zlib.h.

[ArnieBerg]

When I run the 'make test' after a successful setup of the environment and pipeline, I get the following error:
(raseenv) analynano@analynano:/media/analynano/Data500GB/ArnieFiles/rase-pipeline$ prophyle decompress "database/spneumoniae-sparc.k18.tar.gz" database
[prophyle] 2019-10-15 16:15:26 Shell command: make -j -s -C /home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle
prophyle_assembler.cpp:43:10: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^~~~~~~~
compilation terminated.
make[1]: *** [Makefile:13: prophyle_assembler.o] Error 1
make: *** [Makefile:14: compile_assembler] Error 2
[prophyle] 2019-10-15 16:15:26 Unfinished, an error occurred (error code 2, 0.8 MB used): make -j -s -C /home/analynano/miniconda3/e
nvs/raseenv/lib/python3.7/site-packages/prophyle
ProPhyle Error: A command failed, see messages above.

There is a zlib.h in /home/analynano/miniconda3/envs/raseenv/include. And it is referenced in $CFLAGS:

(raseenv) analynano@analynano:/media/analynano/Data500GB/ArnieFiles/rase-pipeline/rase$ echo $CFLAGS
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/analynano/miniconda3/envs/raseenv/include

What could I be missing?

[karel-brinda]

Thanks for reporting the problem. What operating system do you use?

[ArnieBerg]

Ubuntu 18.04.

[karel-brinda]

It's weird that zlib is not installed on your system. Could you run sudo apt-get install libz-dev and try it again?

[ArnieBerg]

It seems that zlib.h is installed on the system, but is not included in the search path for the make environment.

Notice that if it helps you can setup an environment with docker and replicate the issue by running 'docker build' with this file:
https://gist.github.com/sbuller/a7b769b9c6be7162949084d68daaece8

It looks like a problem with the prophyle Makefiles not working with conda environments.
Conda puts zlib.h in $ENV/include, but the compiler is built to expect the system includes under $ENV/x86_64-conda_cos6-linux-gnu. Conda seems to take care of this by setting conventional environment variables for the compiler. These environment variables are picked up by default make rules, but are not present in the rules in the Makefiles in $ENV/lib/python3.7/site-packages/prophyle/*. See https://www.gnu.org/software/make/manual/make.html#Implicit-Rules for more on implicit rules. In particular conda relies on CFLAGS and LDFLAGS being used, but this is just convention due to the aforementioned implicit rules.

Does that help point to a solution?

[ArnieBerg]

Here is what fixed it for my environment. I modified the Makefiles to reflect the include location.

export PROPHYLE=/home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle

append LDFLAGS to compilation commands

sed -i 's/($(CC).$)/\1 $(LDFLAGS)/' $PROPHYLE/{,/}/Makefile
sed -i 's/($(CXX).$)/\1 $(LDFLAGS)/' $PROPHYLE/{,/}/Makefile

append to CFLAGS rather than overwriting

sed -i 's/CFLAGS=/CFLAGS+=/' $PROPHYLE/prophyle_index/{,bwa/}Makefile
sed -i 's/CXXFLAGS =/CXXFLAGS +=/' $PROPHYLE/prophyle_as*/Makefile

[karel-brinda]

Hi Arnie, thanks for the solution! I will keep the ticket open as other users might be experiencing the same and the issue needs to be fixed.

[ArnieBerg]

Hi Karel, Now that I have Rase working well, I am faced with an operational issue. Your paper illustrates well how to use Rase with two sample bacteria, but how does one go about creating the input database for a different bacterium? Is there a best practice guide that explains the detailed steps? My background is coming from a long Computer Science career combined with a recent Bioinformatics degree. Thanks for your guidance on this, Arnie From: Karel Břinda <[email protected]> Sent: October 17, 2019 2:57 PM To: c2-d2/rase-pipeline <[email protected]> Cc: ArnieBerg <[email protected]>; State change <[email protected]> Subject: Re: [c2-d2/rase-pipeline] Cannot find zlib.h. (#14) Hi Arnie, thanks for the solution! I will keep the ticket open as other users might be experiencing the same and the issue needs to be fixed. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=AFQ724J44KWTPB4S47Z6PUTQPDGRHA5CNFSM4JBOPNU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRP4YQ#issuecomment-543358562> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQ724MQX5K7TZQBS25IDCTQPDGRHANCNFSM4JBOPNUQ> .

[MortenEneberg]

Hi Karel and Arnie,

I receive the exact same error also running Ubuntu (20.04) in a VM.

Which of the Makefiles is it that has to be altered to fix the error?

Kind regards,
Morten

[MortenEneberg]

I tried using Arnie's solution as I realized (at least that is what I think) it was supposed to replace all occurrences. However, I receive an error upon execution, can you help:

Thanks!

[MortenEneberg]

@karel-brinda , @ArnieBerg
? :-)

[karel-brinda]

I think the best solution would be to edit the corresponding file manually (note that you will have to identify the directory of the source codes in your log messages, it no longer will be /home/analynano/miniconda3/envs/raseenv/lib/python3.7/site-packages/prophyle).

On most computers the current way of packaging ProPhyle works, but sometimes there are issues like this one. As ProphAsm (formerly called prophyle_assembler) now exists as a standalone program, future versions of ProPhyle will no longer be distributed with its source codes, but use ProphAsm as a dependency. But it will require changes in the architecture of ProPhyle.

[karel-brinda]

Concerning the sed error message, my best guess is that on your system there's a slightly different variant of sed with a different syntax (it might be a GNU vs. BSD issue like with many other Unix programs).

[MortenEneberg]

Hi again,
I did locate the 4(?) files that are to be edited, however I am unsure what substitutions the sed command actually makes to the flags, and thus what manual substitutions I should conduct. Could you help me with what exact changes I need to make to these files?

Sincerely, Morten

Prophyle makefile

Prophyle/prophyle_index/bwa makefile

Prophyle/prophyle_assigment makefile

Prophyle/prophyle_assembler makefile

[MortenEneberg]

Or maybe a hint to what version of sed will work. This is my installed version:

[karel-brinda]

I am sorry for the difficulties. Maybe try if some params like -E are not necessary: https://unix.stackexchange.com/questions/615977/invalid-reference-1-on-s-commands-rhs-error-for-sed-script. On some systems, these might be passed automatically (e.g., through aliasing).