Example for Aarch64 compile

[Desperado17]

Greetings,

I need to cross-compile kcov for an armv8 Aarch64 yocto linux on an X64 Ubuntu Linux. Can someone point me to an example for how to run cmake to set all necessary parameters?

Thank you.

[SimonKagstrom]

Unfortunately, I've never cross-compiled it myself, so can't really say what's needed. Feel free to fill in here how it's solved, but in general I think it can be done using generic (i.e., what you find by googling) cmake cross compilation instructions. At least as long as the required libraries are available as well.

[Desperado17]

Ok, I arrived at the compile step and ran into the first problem (I had to censor proprietary parts with XXXXXX.):

[ 76%] Building CXX object src/CMakeFiles/kcov.dir/parsers/elf.cc.o
cd /home/user/Downloads/kcov/kcov/build/src && aarch64-linux-gnu-g++ --sysroot=XXXXXXXXXX/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/aarch64-linux-gnu/libc -DPACKAGE -DPACKAGE_VERSION -I/XXXXXXXXXX -IXXXXXXXXXX -I/XXXXXXXXXX -I/XXXXXXXX/dist-libelf/usr/include -I/XXXXXXXXXXcurl-dev/include -std=c++0x -g -Wall -D_GLIBCXX_USE_NANOSLEEP -DKCOV_LIBRARY_PREFIX=/tmp -DKCOV_HAS_LIBBFD=0 -O3 -DNDEBUG -o CMakeFiles/kcov.dir/parsers/elf.cc.o -c /home/user/Downloads/kcov/kcov/src/parsers/elf.cc
/home/user/Downloads/kcov/kcov/src/parsers/elf.cc:5:10: fatal error: elfutils/libdw.h: No such file or directory
 #include <elfutils/libdw.h>

It seems that the offending header is from the path that is provided via LIBDW_INCLUDE_DIR but the directory in the -I parameter is from LIBELF_INCLUDE_DIR.

Does this step require both headers from libdw and libelf? If yes, is this accounted for in the makefiles?

[SimonKagstrom]

Yes, both are required. I guess the CMakeLists.txt should be better and stop before compilation starts though...

But elfutils (libdw.h) is used for parsing DWARF debugging information, so that's fundamental to kcov.

[Desperado17]

Can you tell me in which Cmake file the -I includes for src/CMakeFiles/kcov.dir/parsers/elf.cc.o are specified?

[SimonKagstrom]

Sure! src/CMakeLists.txt is the main build file, while the cmake/-directory contains the helpers to find libraries, FindLibElf.cmake and FindElfutils.cmake are of interest here.

[Desperado17]

CmakeLists.txt:294

if (LIBDW_FOUND)
include_directories(${LIBDW_INCLUDE_DIRS})
endif (LIBDW_FOUND)

Is this what should append the path including <elfutils/libdw.h> ?

[Desperado17]

Do these indicate a problem that may cause libdw include not getting added?

CMake Warning (dev) at /opt/cmake-3.19.3-Linux-x86_64/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to find_package_handle_standard_args (LIBBFD)
does not match the name of the calling package (Bfd). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindBfd.cmake:79 (find_package_handle_standard_args)
src/CMakeLists.txt:3 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) at /opt/cmake-3.19.3-Linux-x86_64/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to find_package_handle_standard_args (ElfUtils)
does not match the name of the calling package (Elfutils). This can lead
to problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindElfutils.cmake:70 (find_package_handle_standard_args)
src/CMakeLists.txt:143 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

[Desperado17]

Ok, apparently LIBDW_FOUND is never set. What determines whether this is found? LIBDW_LIBRARY and LIBDW_INCLUDE_DIR? If yes, how should these look like? Currently LIBDW_LIBRARY points directly at libdw.so.

[SimonKagstrom]

I think it's probably this part in FindElfutils.cmake that is not properly working,

find_path (DWARF_INCLUDE_DIR
    NAMES
      dwarf.h
    HINTS
      ${PC_LIBDW_INCLUDE_DIRS}
    PATHS
      /usr/include
      /usr/local/include
      /opt/local/include
      /sw/include
      ENV CPATH) # PATH and INCLUDE will also work

Probably you can pass CPATH before cmake to allow it to find elfutils/libdw.h, i.e., if you have your target devel headers in

/home/xxx/devel/cross-compilation-path/sdk/usr/include

and your libraries in

/home/xxx/devel/cross-compilation-path/sdk/lib

then run cmake with

CPATH=/home/xxx/devel/cross-compilation-path/sdk/usr/include LIBRARY_PATH=/home/xxx/devel/cross-compilation-path/sdk/lib cmake <other args as before>

edit: My example was for dwarf.h, but the same goes for libdw.h. Also added LIBRARY_PATH

[Desperado17]

Which command would set LIBDW_FOUND to true anyways?

Edit: Overriding LIBDW_FOUND manually will allow compilation to proceed.

[SimonKagstrom]

It's should be set by FindElfutils.cmake if it manages to find the headers and libraries, but good that you found a workaround! Probably the Find utilities doesn't work very well in your cross compilation setting.

[Desperado17]

According to https://cmake.org/cmake/help/latest/command/find_package.html find_package would set the LIBDW_FOUND variable but I cannot find a find_package call with LIBDW. Where did you intend to have it set?

[Desperado17]

Another question: I managed to run kcov with the intended executable but it doesn't produce any coverage output (it does create the file structure in the output directory like index html, coverage.json etc.). Does kcov work with stripped binaries? Some other tools allow adding symbols later on.

[SimonKagstrom]

The find_package_handle_standard_args should do that if I remember correctly.

Running without symbols isn't really supported. There is the kcov-system-daemon which tries to do this, but it's not quite functional.

It does support debug links though (https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html), which might be useful depending on how you build your system. With those, you could e.g., NFS mount the debug directories and kcov will detect and use those if the binaries are stripped.

[Desperado17]

Where is find_package_handle_standard_args called for LIBDW?

https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
"The _FOUND variable will be set to TRUE if all the variables ... are valid and any optional constraints are satisfied, and FALSE otherwise."

It's a pity that kcov doesn't have a no symbols mode. We have tests conducted by external staff and handing out our symbols would be a problem.

[SimonKagstrom]

It's here,

https://github.com/SimonKagstrom/kcov/blob/master/cmake/FindElfutils.cmake#L70

unfortunately, it won't work without symbols. Kcov uses breakpoints to get coverage data, and the symbols (well, the DWARF debug information) are needed in order to provide the translation source-file:line -> address.

The source doesn't need to be available when collecting, with --collect-only, but the debug information is needed.

The idea with system mode was to overcome this limitation (well, not only, but as well). In that case, you would run kcov on your build output (the sysroot, with debug symbols) to create a list of breakpoints to be set in a file.

On target, the kcov system daemon would then read this file and produce coverage output to some directory which can later be copied over to your host machine and analyzed there.

But unfortunately, it's still just a proof of concept and hasn't been useful to me personally, so I haven't finished implementing it.

[Desperado17]

find_package_handle_standard_args(ElfUtils DEFAULT_MSG
LIBDW_LIBRARY
LIBDW_INCLUDE_DIR)

If I understand the documentation of find_package_handle_standard_args right then the first parameter, ElfUtils, would have to be equivalent to LIBDW so LIBDW_FOUND=1 is set. Is that the case? Otherwise the result should be something like ELFUTILS_FOUND which is not used anywhere I think.

We are coding for embedded environment where a lot of testing is done in the wild and with production software so a mode where the symbols could be resoved later would be invaluable.