ld.so.cache not updating properly for emerge-root

[Kangie]

Hi All,

I've been trying to build an image with the following build.sh (superfluous comments removed):

#
# Kubler phase 1 config, pick installed packages and/or customize the build
#

_packages="
    app-crypt/libb2
    app-dicts/myspell-en
    dev-build/meson
    dev-libs/openssl
    dev-python/sphinx
    dev-python/sphinx-rtd-theme
    dev-python/sphinxcontrib-spelling
    sys-devel/binutils
"

BOB_INSTALL_BASELAYOUT=true

configure_builder()
{
    :
}


configure_rootfs_build()
{
    update_use 'app-dicts/myspell-en' '+l10n::en-AU'
    update_use 'sys-apps/util-linux' '-su'

    :
}

finish_rootfs_build()
{
    :
}

Unfortunately, I run into issues where python is unable to find libb2 (blake2{b,s}) when loading hashlib:

ERROR:root:code for hash blake2b was not found.
Traceback (most recent call last):
  File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2b
ERROR:root:code for hash blake2s was not found.
Traceback (most recent call last):
  File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
usage: sphinx-build [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]

From within the container I see

306d835e5dc8 / # ldd /usr/lib64/libb2.so
        linux-vdso.so.1 (0x00007ffe94bd9000)
        libgomp.so.1 => not found
        libc.so.6 => /lib64/libc.so.6 (0x00007d1a249b9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007d1a24ba0000)

which is odd as libgomp definitely exists:

306d835e5dc8 / # LD_PRELOAD=/usr/lib/gcc/x86_64-pc-linux-gnu/13/libgomp.so.1 python
Python 3.11.8 (main, Mar 13 2024, 06:07:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>>

and is in /etc/ld.so.conf:

8835fb8f3e6b / # cat /etc/ld.so.conf
# contents of /etc/env.d directory.
include ld.so.conf.d/*.conf
# ld.so.conf autogenerated by env-update; make all changes to
/lib
/lib64
/usr/lib
/usr/lib64
/usr/lib/gcc/x86_64-pc-linux-gnu/13
/usr/local/lib
/usr/local/lib64

but not in the cache:

8835fb8f3e6b / # strings /etc/ld.so.cache 
glibc-ld.so.cache1.1.
/usr/lib64/libresolv.so
/usr/lib64/libcrypt.so
/lib64/libskarnet.so
/usr/lib64/libnss_compat.so
/lib64/libs6auto.so
/usr/lib64/libanl.so
/usr/lib64/libc_malloc_debug.so
/usr/lib64/libunistring.so
/lib64/libexecline.so
/lib64/libpcprofile.so
/usr/lib64/libBrokenLocale.so
/lib64/libmemusage.so
/usr/lib64/libnss_hesiod.so
/usr/lib64/libmvec.so
/usr/lib64/libnss_db.so
/usr/lib64/libthread_db.so
/lib64/libs6.so
/usr/lib64/libidn2.so
/lib64/libexecline.so.2.9
/lib64/libm.so.6
/lib64/libc.so.6
/usr/lib64/libunistring.so.5
/lib64/libskarnet.so.2.14
/lib64/libs6auto.so.2.12
/lib64/libs6.so.2.12
/lib64/libresolv.so.2
/usr/lib64/libcrypt.so.2
/lib64/libnss_compat.so.2
/lib64/libnss_dns.so.2
/lib64/libnss_files.so.2
/lib64/libdl.so.2
/lib64/libnss_hesiod.so.2
/lib64/libnss_db.so.2
/lib64/ld-linux-x86-64.so.2
/lib64/librt.so.1
/lib64/libcrypt.so.1
/lib64/libnsl.so.1
/lib64/libanl.so.1
/lib64/libutil.so.1
/lib64/libBrokenLocale.so.1
/lib64/libmvec.so.1
/lib64/libthread_db.so.1
/lib64/libc_malloc_debug.so.0
/lib64/libpthread.so.0
/usr/lib64/libidn2.so.0
ldconfig (Gentoo 2.38-r10 (patchset 10)) stable release version 2.38

Regenerating the cache inside the container resolves the issue, but I shouldn't need to do that:

docker run -it esphpc/sphinx-docs-builder /bin/bash
2ab3065ffba0 / # cat /etc/ld.so.conf.d/05gcc-x86_64-pc-linux-gnu.conf
/usr/lib/gcc/x86_64-pc-linux-gnu/13
2ab3065ffba0 / # ldconfig ; python
Python 3.11.8 (main, Mar 13 2024, 06:07:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>>

I'm also not sure where the cache is being generated, as we remove the cache from emerge-root, maybe that can shed some light on the situation:

kubler/engine/docker/bob-core/build-root.sh

Lines 708 to 709 in 3aa97bb

	rm -rf \
	"${_EMERGE_ROOT}"/etc/ld.so.cache \

Any insight on what's happening? Portage's env-update should take care of this and it's ROOT aware.

In the short term I will try to generate the cache as a postinst step before a rootfs tarball is taken.

[Kangie]

Following along with that, adding ldconfig -X -r "/emerge-root" --verbose to finish_rootfs_build detects the library:

+ ldconfig -X -r /emerge-root --verbose
ldconfig: Path `/lib' given more than once
(from <builtin>:0 and /etc/ld.so.conf:4)
ldconfig: Path `/lib64' given more than once
(from <builtin>:0 and /etc/ld.so.conf:5)
ldconfig: Can't stat /libx32: No such file or directory
ldconfig: Path `/usr/lib' given more than once
(from <builtin>:0 and /etc/ld.so.conf:6)
ldconfig: Path `/usr/lib64' given more than once
(from <builtin>:0 and /etc/ld.so.conf:7)
ldconfig: Can't stat /usr/libx32: No such file or directory
/lib: (from /etc/ld.so.conf:4)
/lib64: (from /etc/ld.so.conf:5)
/usr/lib: (from /etc/ld.so.conf:6)
/usr/lib64: (from /etc/ld.so.conf:7)
        libenchant-2.so.2 -> libenchant-2.so.2.5.0
        libgio-2.0.so.0 -> libgio-2.0.so.0.7800.3
        libglib-2.0.so.0 -> libglib-2.0.so.0.7800.3
        libgmodule-2.0.so.0 -> libgmodule-2.0.so.0.7800.3
        libgobject-2.0.so.0 -> libgobject-2.0.so.0.7800.3
        libgthread-2.0.so.0 -> libgthread-2.0.so.0.7800.3
        libxml2.so.2 -> libxml2.so.2.12.5
        libpython3.11.so.1.0 -> libpython3.11.so.1.0
        libb2.so.1 -> libb2.so.1.0.4
        libmpc.so.3 -> libmpc.so.3.3.1
        libmpfr.so.6 -> libmpfr.so.6.2.1
        libblkid.so.1 -> libblkid.so.1.1.0
        libfdisk.so.1 -> libfdisk.so.1.1.0
        libmount.so.1 -> libmount.so.1.1.0
        libsmartcols.so.1 -> libsmartcols.so.1.1.0
        libuuid.so.1 -> libuuid.so.1.3.0
        libasm.so.1 -> libasm-0.190.so
        libdw.so.1 -> libdw-0.190.so
        libelf.so.1 -> libelf-0.190.so
        libpkgconf.so.4 -> libpkgconf.so.4.0.0
        libhunspell-1.7.so.0 -> libhunspell.so
        libsqlite3.so.0 -> libsqlite3.so.0.8.6
        libexpat.so.1 -> libexpat.so.1.8.10
        libexpatw.so.1 -> libexpatw.so.1.8.10
        libffi.so.8 -> libffi.so.8.1.2
        libpcre2-16.so.0 -> libpcre2-16.so.0.11.2
        libpcre2-32.so.0 -> libpcre2-32.so.0.11.2
        libpcre2-8.so.0 -> libpcre2-8.so.0.11.2
        libpcre2-posix.so.3 -> libpcre2-posix.so.3.0.4
        libgmp.so.10 -> libgmp.so.10.5.0
        libgmpxx.so.4 -> libgmpxx.so.4.7.0
/usr/lib/gcc/x86_64-pc-linux-gnu/13: (from /etc/ld.so.conf:8)
        libasan.so.8 -> libasan.so.8.0.0
        libatomic.so.1 -> libatomic.so.1.2.0
        libcc1.so.0 -> libcc1.so.0.0.0
        libgcc_s.so.1 -> libgcc_s.so.1
        libgfortran.so.5 -> libgfortran.so.5.0.0
        libgomp.so.1 -> libgomp.so.1.0.0
        libhwasan.so.0 -> libhwasan.so.0.0.0
        libitm.so.1 -> libitm.so.1.0.0
        liblsan.so.0 -> liblsan.so.0.0.0
        libquadmath.so.0 -> libquadmath.so.0.0.0
        libstdc++.so.6 -> libstdc++.so.6.0.32
        libtsan.so.2 -> libtsan.so.2.0.0
        libubsan.so.1 -> libubsan.so.1.0.0
/usr/local/lib: (from /etc/ld.so.conf:9)
/usr/local/lib64: (from /etc/ld.so.conf:10)

but the ld.so.cache that ends up in the rootfs tarball does not match this. Are we unpacking a cached copy somewhere? Edit: we have to be, because we rming it after running finish_rootfs_build as per the above post.

Moving my workaround to the dockerfile template...

[berney]

Is this issue still present in latest kubler and with latest portage? What parent image are you using? Which builder (bob or bob-musl)? It looks like it must be bob (glibc).

When you ran the commands in the container is that the built container, and not the build container (e.g. kubler build -i python)? It looks like it's the built python image, but I would expect the GCC libraries to not exist unless you used the copy_gcc_libs function, which I don't see in your build.sh.

Did you work any more details out?

I think copy_gcc_libs doesn't handle multilib (having both 32-bit and 64-bit libraries) well, so it might be related to that.

[berney]

I built an image using your build.sh and python works, but import hashlib gets the error you showed.

Debugging with LD_DEBUG=libs the libb2 library is found. But it depends on libgomp.so.1, which isn't found, because it is in /usr/lib/gcc/x86_64-pc-linux-gnu/14/, which isn't in the system search path. It's in /etc/ld.so.conf.d/05gcc-x86_64-pc-linux-gnu.conf (created by the GCC install), which is included by /etc/ld.so.conf, but the /etc/ld.so.cache is missing (because kubler deletes it as you pointed out). This is why your workaround of running ldconfig works, building the cache is all that's needed.

This is covered in https://github.com/edannenberg/kubler?tab=readme-ov-file#common-build-pitfalls

kubler/README.md

Line 449 in cc6ae36

* Image was successfully built but can't find it's libraries on image run

Your workaround is an alternative to the Dockerfile approach, and probably better way.