[FR] extensions: general way to provide library paths that are searched before LDSHARED's paths

[lgarrison]

What's the problem this feature will solve?

Currently, if sysconfig's LDSHARED contains a -L flag, that flag will appear before user-provided flags when building an extension. If the user provides an additional path to search (e.g. via LDFLAGS), that path will have lower priority than paths in LDSHARED since library paths are searched left to right. This makes it hard for a user to prefer their own library, even when an undesired version is being found via LDSHARED.

Describe the solution you'd like

Ideally, there would be a way to search user-provided paths first. But I'm honestly not sure what this would look like, implementation-wise. Since LDSHARED contains both the linker invocation and flags to the linker, any solution would seem to need to parse LDSHARED to insert -L flags early. That seems fragile. However, there is precedent in the sense that LDSHARED is already automatically rewritten with a user-provided CC:

setuptools/setuptools/_distutils/sysconfig.py

Line 331 in ba24375

ldshared = newcc + ldshared[len(cc) :]

Alternative Solutions

Users can currently pass LDSHARED to fully specify the linker command and the order of -L flags. This is currently the workaround I'm using. However, it's verbose (see below) and arguably error-prone, since it requires manually discovering the LDSHARED value and modifying it to insert your own paths first.

Additional context

Here's an example LDSHARED on my system, which contains a lot of paths a user might want to override (notably -L/usr/lib64):

> python -m sysconfig
...
LDSHARED = "/mnt/sw/nix/store/qgcgvdlgz6546x6mzjiknvrq218j9nx8-gcc-11.4.0/bin/gcc -pthread -shared -L/mnt/sw/nix/store/30zwhjaf2jh84r0bws1yz2dwgqhw23f0-bzip2-1.0.8/lib -L/mnt/sw/nix/store/q0d0y49rsz7291c00kr0560y2q6bix58-expat-2.6.2/lib -L/mnt/sw/nix/store/snfa89v7a38qmzwqsj7006awc2z77rf4-gdbm-1.23/lib -L/mnt/sw/nix/store/zdnba5ggf37q6cxx1rszxabi8m260cfh-gettext-0.22.4/lib -L/mnt/sw/nix/store/d9n53v93vxra9a0madzpkicmrxxgbd61-libffi-3.4.6/lib64 -L/mnt/sw/nix/store/6416cbjlqkw7l9q7naz81ha4mknw3pkr-libxcrypt-4.4.35/lib -L/mnt/sw/nix/store/hn8dq06sw3hyx5z21z1n2kf62d37n8sv-ncurses-6.4/lib -L/usr/lib64 -L/mnt/sw/nix/store/nfr8l146l2lfxhdphgmd63qb8awy07w7-readline-8.2/lib -L/mnt/sw/nix/store/i117x95p5jmjpqmkdp8iyvr6n4yhzn6i-sqlite-3.43.2/lib -L/mnt/sw/nix/store/0swn26vgd2lwfnindbxcgzbpz2ra2qmc-tcl-8.6.12/lib -L/mnt/sw/nix/store/wl8bvjd4iwsx496lbpy641ihpswldqry-tk-8.6.11/lib -L/mnt/sw/nix/store/rrkjvd4n6dg3y0wvr2r34zq6zfz7864a-util-linux-uuid-2.38.1/lib -L/mnt/sw/nix/store/pmzfq3af88pyl5ykcrxqjqa1k4639nnl-xz-5.4.6/lib -L/mnt/sw/nix/store/2i17p7mws5s8x6y68md7vyclaim4b7nk-zlib-1.3.1/lib"
...

Code of Conduct

• I agree to follow the PSF Code of Conduct