Compilation error on Ubuntu 22.10

[coco21]

Hello,

I am unable to build MoonGen and libmoon on my system.
Same system but with a former Ubunut 20.04 LTS that all worked here but no longer with this very OS (see below).

Installed all dependencies and also DPDK that I tested is working with my Intel ixgbe 10G NIC, no problems from there.

Here my OS details:

chris@chris-OptiPlex-9020:$ uname -a
Linux chris-OptiPlex-9020 5.19.0-23-generic #24-Ubuntu SMP PREEMPT_DYNAMIC Fri Oct 14 15:39:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
chris@chris-OptiPlex-9020:$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.10"
NAME="Ubuntu"
VERSION_ID="22.10"
VERSION="22.10 (Kinetic Kudu)"
VERSION_CODENAME=kinetic
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=kinetic
LOGO=ubuntu-logo
chris@chris-OptiPlex-9020:~$

And here what I get as build output log:

root@chris-OptiPlex-9020:/home/chris/MoonGen# ./build.sh
Submodule 'libmoon' (https://github.com/libmoon/libmoon) registered for path 'libmoon'
Cloning into '/home/chris/MoonGen/libmoon'...
Submodule path 'libmoon': checked out 'dfceafcbde59a6d50a793160382571ce45561b4d'
Build libmoon with MoonGen
Submodule 'deps/dpdk' (https://github.com/emmericp/dpdk) registered for path 'deps/dpdk'
Submodule 'deps/highwayhash' (https://github.com/google/highwayhash.git) registered for path 'deps/highwayhash'
Submodule 'deps/luajit' (https://github.com/emmericp/LuaJIT) registered for path 'deps/luajit'
Submodule 'deps/pciids' (https://github.com/emmericp/pciids) registered for path 'deps/pciids'
Submodule 'deps/tbb' (https://github.com/01org/tbb.git) registered for path 'deps/tbb'
Submodule 'lua/lib/pf' (https://github.com/emmericp/pflua) registered for path 'lua/lib/pf'
Submodule 'lua/lib/syscall' (https://github.com/emmericp/ljsyscall) registered for path 'lua/lib/syscall'
Submodule 'lua/lib/turbo' (https://github.com/emmericp/turbo) registered for path 'lua/lib/turbo'
Cloning into '/home/chris/MoonGen/libmoon/deps/dpdk'...
Cloning into '/home/chris/MoonGen/libmoon/deps/highwayhash'...
Cloning into '/home/chris/MoonGen/libmoon/deps/luajit'...
Cloning into '/home/chris/MoonGen/libmoon/deps/pciids'...
Cloning into '/home/chris/MoonGen/libmoon/deps/tbb'...
Cloning into '/home/chris/MoonGen/libmoon/lua/lib/pf'...
Cloning into '/home/chris/MoonGen/libmoon/lua/lib/syscall'...
Cloning into '/home/chris/MoonGen/libmoon/lua/lib/turbo'...
Submodule path 'deps/dpdk': checked out '6d6c77b373757ec98010b4bf4fcb671423fa3ce2'
Submodule path 'deps/highwayhash': checked out '14dedecd1de87cb662f7a882ea1578d2384feb2f'
Submodule path 'deps/luajit': checked out '8271c643c21d1b2f344e339f559f2de6f3663191'
Submodule path 'deps/pciids': checked out 'eea0f7199045fbaf3800a499e867ef1a4e5d1f9e'
Submodule path 'deps/tbb': checked out 'd7e150913467f3c168ba6319c985cf341fd81012'
Submodule path 'lua/lib/pf': checked out 'b16b31aed678a3b8ce809391c9e6bcd67f70be0e'
Submodule path 'lua/lib/syscall': checked out '2a18b1b62947c597b4e573bb042483ff18434b6f'
Submodule path 'lua/lib/turbo': checked out 'd69c85a255a6ceed329813f04e40bd2a9cefcf5e'
==== Building LuaJIT 2.1.0-beta3 ====
...
OK Successfully built LuaJIT
make[1]: Leaving directory '/home/chris/MoonGen/libmoon/deps/luajit/src'
==== Successfully built LuaJIT 2.1.0-beta3 ====
==== Installing LuaJIT 2.1.0-beta3 to /usr/local ====
...
==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local ====

Note: the development releases deliberately do NOT install a symlink for luajit
You can do this now by running this command (with sudo):

ln -sf luajit-2.1.0-beta3 /home/chris/MoonGen/libmoon/deps/luajit/usr/local/bin/luajit

Configuration done using x86_64-native-linuxapp-gcc
== Build lib
== Build lib/librte_compat
== Build lib/librte_eal
SYMLINK-FILE include/rte_compat.h
== Build lib/librte_eal/common
SYMLINK-FILE include/generic/rte_atomic.h
SYMLINK-FILE include/generic/rte_byteorder.h
SYMLINK-FILE include/generic/rte_cycles.h
SYMLINK-FILE include/generic/rte_prefetch.h
SYMLINK-FILE include/generic/rte_spinlock.h
SYMLINK-FILE include/generic/rte_memcpy.h
SYMLINK-FILE include/generic/rte_cpuflags.h
SYMLINK-FILE include/generic/rte_rwlock.h
SYMLINK-FILE include/generic/rte_vect.h
SYMLINK-FILE include/generic/rte_pause.h
SYMLINK-FILE include/generic/rte_io.h
SYMLINK-FILE include/rte_branch_prediction.h
SYMLINK-FILE include/rte_common.h
SYMLINK-FILE include/rte_debug.h
SYMLINK-FILE include/rte_eal.h
SYMLINK-FILE include/rte_errno.h
SYMLINK-FILE include/rte_launch.h
SYMLINK-FILE include/rte_lcore.h
SYMLINK-FILE include/rte_log.h
SYMLINK-FILE include/rte_memory.h
SYMLINK-FILE include/rte_memzone.h
SYMLINK-FILE include/rte_pci.h
SYMLINK-FILE include/rte_per_lcore.h
SYMLINK-FILE include/rte_random.h
SYMLINK-FILE include/rte_tailq.h
SYMLINK-FILE include/rte_interrupts.h
SYMLINK-FILE include/rte_alarm.h
SYMLINK-FILE include/rte_string_fns.h
SYMLINK-FILE include/rte_version.h
SYMLINK-FILE include/rte_eal_memconfig.h
SYMLINK-FILE include/rte_malloc_heap.h
SYMLINK-FILE include/rte_hexdump.h
SYMLINK-FILE include/rte_devargs.h
SYMLINK-FILE include/rte_bus.h
SYMLINK-FILE include/rte_dev.h
SYMLINK-FILE include/rte_vdev.h
SYMLINK-FILE include/rte_pci_dev_feature_defs.h
SYMLINK-FILE include/rte_pci_dev_features.h
SYMLINK-FILE include/rte_malloc.h
SYMLINK-FILE include/rte_keepalive.h
SYMLINK-FILE include/rte_time.h
SYMLINK-FILE include/rte_service.h
SYMLINK-FILE include/rte_service_component.h
SYMLINK-FILE include/rte_atomic_32.h
SYMLINK-FILE include/rte_atomic_64.h
SYMLINK-FILE include/rte_atomic.h
SYMLINK-FILE include/rte_byteorder_32.h
SYMLINK-FILE include/rte_byteorder_64.h
SYMLINK-FILE include/rte_byteorder.h
SYMLINK-FILE include/rte_cpuflags.h
SYMLINK-FILE include/rte_cycles.h
SYMLINK-FILE include/rte_io.h
SYMLINK-FILE include/rte_memcpy.h
SYMLINK-FILE include/rte_pause.h
SYMLINK-FILE include/rte_prefetch.h
SYMLINK-FILE include/rte_rtm.h
SYMLINK-FILE include/rte_rwlock.h
SYMLINK-FILE include/rte_spinlock.h
SYMLINK-FILE include/rte_vect.h
== Build lib/librte_eal/linuxapp
== Build lib/librte_eal/linuxapp/eal
== Build lib/librte_eal/linuxapp/igb_uio
CC eal.o
CC eal_hugepage_info.o
CC eal_memory.o
CC eal_thread.o
CC eal_log.o
CC eal_vfio.o
CC eal_vfio_mp_sync.o
CC eal_pci.o
CC eal_pci_uio.o
CC eal_pci_vfio.o
CC eal_debug.o
CC eal_lcore.o
CC eal_timer.o
CC eal_interrupts.o
CC eal_alarm.o
CC eal_common_lcore.o
CC eal_common_timer.o
CC eal_common_memzone.o
CC eal_common_log.o
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_memzone.c: In function ‘find_heap_max_free_elem’:
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_memzone.c:115:39: warning: taking address of packed member of ‘struct rte_mem_config’ may result in an unaligned pointer value [-Waddress-of-packed-member]
115 | malloc_heap_get_stats(&mcfg->malloc_heaps[i], &stats);
| ^~~~~~~~~~~~~~~~~~~~~~
...
CC [M] /home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.o
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/malloc_heap.c: In function ‘rte_eal_malloc_heap_init’:
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/malloc_heap.c:224:40: warning: taking address of packed member of ‘struct rte_mem_config’ may result in an unaligned pointer value [-Waddress-of-packed-member]
224 | malloc_heap_add_memseg(&mcfg->malloc_heaps[ms->socket_id], ms);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c: In function ‘igbuio_msix_mask_irq’:
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:100:29: error: ‘struct msi_desc’ has no member named ‘masked’
100 | u32 mask_bits = desc->masked;
| ^~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:101:31: error: ‘struct msi_desc’ has no member named ‘msi_attrib’
101 | unsigned offset = desc->msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE +
| ^~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:109:30: error: ‘struct msi_desc’ has no member named ‘masked’
109 | if (mask_bits != desc->masked) {
| ^~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:110:39: error: ‘struct msi_desc’ has no member named ‘mask_base’
110 | writel(mask_bits, desc->mask_base + offset);
| ^~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:111:27: error: ‘struct msi_desc’ has no member named ‘mask_base’
111 | readl(desc->mask_base);
| ^~
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:112:21: error: ‘struct msi_desc’ has no member named ‘masked’
112 | desc->masked = mask_bits;
| ^~
In file included from ./include/linux/kernel.h:21,
from ./arch/x86/include/asm/percpu.h:27,
from ./arch/x86/include/asm/current.h:6,
from ./include/linux/sched.h:12,
from ./include/linux/ratelimit.h:6,
from ./include/linux/dev_printk.h:16,
from ./include/linux/device.h:15,
from /home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:27:
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c: In function ‘igbuio_pci_irqcontrol’:
/home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:146:53: error: ‘struct device’ has no member named ‘msi_list’
...
cc1: some warnings being treated as errors
make[7]: *** [scripts/Makefile.build:257: /home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.o] Error 1
make[6]: *** [Makefile:1851: /home/chris/MoonGen/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio] Error 2
make[5]: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.module.mk:79: igb_uio.ko] Error 2
make[4]: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:65: igb_uio] Error 2
make[4]: *** Waiting for unfinished jobs....
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_options.c: In function ‘eal_plugindir_init’:
/home/chris/MoonGen/libmoon/deps/dpdk/lib/librte_eal/common/eal_common_options.c:236:9: warning: ‘__builtin_strncpy’ output may be truncated copying 4095 bytes from a string of length 4095 [-Wstringop-truncation]
236 | strncpy(solib->name, path, PATH_MAX-1);
| ^
AR librte_eal.a
INSTALL-LIB librte_eal.a
make[3]: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:65: linuxapp] Error 2
make[2]: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.subdir.mk:63: librte_eal] Error 2
make[1]: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.sdkbuild.mk:78: lib] Error 2
make: *** [/home/chris/MoonGen/libmoon/deps/dpdk/mk/rte.sdkroot.mk:129: all] Error 2
make -C deps/http-parser library
make[1]: Entering directory '/home/chris/MoonGen/libmoon/lua/lib/turbo/deps/http-parser'
...
213 | memset(my_rep,0,sizeof(concurrent_queue_rep));
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/chris/MoonGen/libmoon/deps/tbb/./src/old/concurrent_queue_v2.cpp:102:7: note: ‘class tbb::internal::concurrent_queue_rep’ declared here
102 | class concurrent_queue_rep {
| ^~~~~~~~~~~~~~~~~~~~
/home/chris/MoonGen/libmoon/deps/tbb/./src/tbbmalloc/proxy.cpp:234:32: error: ISO C++17 does not allow dynamic exception specifications
234 | void * operator new(size_t sz) throw (std::bad_alloc) {
| ^~~~~
/home/chris/MoonGen/libmoon/deps/tbb/./src/tbbmalloc/proxy.cpp:242:33: error: ISO C++17 does not allow dynamic exception specifications
242 | void* operator new[](size_t sz) throw (std::bad_alloc) {
| ^~~~~
...
| ^~~~
make[1]: *** [/home/chris/MoonGen/libmoon/deps/tbb/./build/Makefile.tbbmalloc:66: proxy.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:36: tbbmalloc] Error 2

CMake Warning at libmoon/CMakeLists.txt:67 (find_package):
By not providing "FindTBB.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "TBB", but
CMake did not find one.

Could not find a package configuration file provided by "TBB" with any of
the following names:

TBBConfig.cmake
tbb-config.cmake

Add the installation prefix of "TBB" to CMAKE_PREFIX_PATH or set "TBB_DIR"
to a directory containing one of the above files. If "TBB" provides a
separate development package or SDK, be sure it has been installed.

-- Configuring done
CMake Warning (dev) at CMakeLists.txt:48 (add_executable):
Policy CMP0115 is not set: Source file extensions must be explicit. Run
"cmake --help-policy CMP0115" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

File:

/home/chris/MoonGen/src/moongen-main.cpp

This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) at CMakeLists.txt:48 (add_executable):
Policy CMP0115 is not set: Source file extensions must be explicit. Run
"cmake --help-policy CMP0115" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

File:

/home/chris/MoonGen/src/software-timestamping.c

This warning is for project developers. Use -Wno-dev to suppress it.

...
-- Generating done
-- Build files have been written to: /home/chris/MoonGen/build
[ 10%] Building C object libmoon/CMakeFiles/moon.dir/src/i40e.c.o
[ 10%] Building C object libmoon/CMakeFiles/moon.dir/src/util.c.o
[ 17%] Building C object libmoon/CMakeFiles/moon.dir/src/memory.c.o
[ 17%] Building CXX object libmoon/CMakeFiles/moon.dir/src/main.cpp.o
[ 17%] Building CXX object libmoon/CMakeFiles/moon.dir/src/lifecycle.cpp.o
[ 20%] Building C object libmoon/CMakeFiles/moon.dir/src/device.c.o
[ 24%] Building CXX object libmoon/CMakeFiles/moon.dir/src/barrier.cpp.o
[ 27%] Building CXX object libmoon/CMakeFiles/moon.dir/src/task.cpp.o
/home/chris/MoonGen/libmoon/src/memory.c:2:10: fatal error: rte_mbuf.h: No such file or directory
2 | #include <rte_mbuf.h>
| ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [libmoon/CMakeFiles/moon.dir/build.make:90: libmoon/CMakeFiles/moon.dir/src/memory.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/chris/MoonGen/libmoon/src/device.c:2:10: fatal error: rte_ethdev.h: No such file or directory
2 | #include <rte_ethdev.h>
| ^~~~~~~~~~~~~~
compilation terminated.
/home/chris/MoonGen/libmoon/src/i40e.c:2:10: fatal error: rte_ethdev.h: No such file or directory
2 | #include <rte_ethdev.h>
| ^~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [libmoon/CMakeFiles/moon.dir/build.make:132: libmoon/CMakeFiles/moon.dir/src/i40e.c.o] Error 1
make[2]: *** [libmoon/CMakeFiles/moon.dir/build.make:118: libmoon/CMakeFiles/moon.dir/src/device.c.o] Error 1
/home/chris/MoonGen/libmoon/src/util.c:3:10: fatal error: rte_ip.h: No such file or directory
3 | #include <rte_ip.h>
| ^~~~~~~~~~
compilation terminated.
make[2]: *** [libmoon/CMakeFiles/moon.dir/build.make:146: libmoon/CMakeFiles/moon.dir/src/util.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:126: libmoon/CMakeFiles/moon.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
Trying to bind interfaces, this will fail if you are not root
Try sudo ./bind-interfaces.sh if this step fails
insmod: ERROR: could not load module ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko: No such file or directory
/usr/bin/env: ‘python’: No such file or directory
Could not find any inactive interfaces to bind to DPDK. Note that this script does not bind interfaces that are in use by the OS.
Delete IP addresses from interfaces you would like to use with libmoon and run this script again.
You can also use the script dpdk-devbind.py in deps/dpdk/usertools manually to manage interfaces used by libmoon and the OS.
root@chris-OptiPlex-9020:/home/chris/MoonGen#

[huberste]

Hi @coco21 ,

The error seems to persist. The problem is, that the old, with MoonGen / libmoon packaged dependencies, won't compile on a newer kernel or with a newer C compiler. Since ubuntu (bginning with 22.04, i guess) ships all those, right now, MoonGen does not build on those machines. You could either downgrade the machines that need MoonGen to ubuntu 18.04 or 20.04, or try "fixing" MoonGen for newer kernel headers and gcc versions.

For a start, in ubuntu 22.04, most dependencies kome packaged with ubuntu and can be installed with apt install dpdk dpdk-dev dpdk-kmods-dkms liblibluajit-5.1-dev libssl-dev libhighwayhash-dev libtbb-dev. You will then have to fix the CMake files from MoonGen and libmoon, and then still have to fix some C files in MoonGen, though.

Good Luck!