pkg-config does not update MacOS SDK paths after major macOS upgrades

[shenlebantongying]

brew doctor output

It is single warning that is impossible to deal as of right now.

> brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  [email protected]

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

> brew config
HOMEBREW_VERSION: 4.1.17
ORIGIN: https://github.com/Homebrew/brew
HEAD: 35746e0a6ba6c3c5cfe56d99f79d9ec9f52ee15f
Last commit: 3 days ago
Core tap JSON: 25 Oct 01:23 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.RB7odOrKDO/org.xquartz:0
HOMEBREW_EDITOR: nano
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 14.0-arm64
CLT: 15.0.0.0.1.1694021235
Xcode: 15.0.1
Rosetta 2: false

What were you trying to do (and why)?

Trying to use pkg-config via cmake to find pango which depends on libffi from macOS SDK, but failed because pkg-config will provides invalid path pointing to older macOS SDK

Minimum CMakeLists.txt to reproduce the problem:

cmake_minimum_required(VERSION 3.27)
project(test)

set(CMAKE_C_STANDARD 23)
find_package(PkgConfig REQUIRED)
pkg_check_modules(pango IMPORTED_TARGET pango)
add_executable(main main.c)
target_link_libraries(main PkgConfig::pango)

Will gives

CMake Error in CMakeLists.txt:
  Imported target "PkgConfig::pango" includes non-existent path

    "/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/ffi"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

What happened (include all command output)?

The pkg-config tries to uses libffi.pc from macOS 11 while i am using the current Xcode and macOS 14.

The pkg-config will give cmake a none-exists path because this line got included:

https://github.com/Homebrew/brew/blob/master/Library/Homebrew/os/mac/pkgconfig/11/libffi.pc#L1

instead of

https://github.com/Homebrew/brew/blob/master/Library/Homebrew/os/mac/pkgconfig/14/libffi.pc#L1

The pkg-config's path falsely includes /opt/homebrew/Library/Homebrew/os/mac/pkgconfig/11

pkg-config --variable pc_path pkg-config
/opt/homebrew/lib/pkgconfig:/opt/homebrew/share/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig:/opt/homebrew/Library/Homebrew/os/mac/pkgconfig/11

What did you expect to happen?

I think pkg-config's search path should be /opt/homebrew/Library/Homebrew/os/mac/pkgconfig/14 instead of /opt/homebrew/Library/Homebrew/os/mac/pkgconfig/11 on newest macOS & XCode.

Step-by-step reproduction instructions (by running brew commands)

brew install pkg-config
pkg-config --variable pc_path pkg-config

The path outputted should includes /opt/homebrew/Library/Homebrew/os/mac/pkgconfig/14 instead of /opt/homebrew/Library/Homebrew/os/mac/pkgconfig/11

For extra info, my XCode 15 installtion includes those things, the 11 got removed recently? i don't know.

/Library/Developer/CommandLineTools/SDKs
> ls -al
total 0
drwxr-xr-x  9 root  wheel  288 27 Sep 10:50 ./
drwxr-xr-x  5 root  wheel  160 27 Sep  2022 ../
lrwxr-xr-x  1 root  wheel   14 27 Sep 10:50 MacOSX.sdk@ -> MacOSX14.0.sdk
drwxr-xr-x  7 root  wheel  224 24 Jun 11:04 MacOSX12.3.sdk/
lrwxr-xr-x  1 root  wheel   14 24 Jun 11:03 MacOSX12.sdk@ -> MacOSX12.3.sdk
drwxr-xr-x  7 root  wheel  224 27 Sep 10:50 MacOSX13.3.sdk/
lrwxr-xr-x  1 root  wheel   14 27 Sep 10:49 MacOSX13.sdk@ -> MacOSX13.3.sdk
drwxr-xr-x  7 root  wheel  224 31 Aug 15:12 MacOSX14.0.sdk/
lrwxr-xr-x  1 root  wheel   14 27 Sep 10:49 MacOSX14.sdk@ -> MacOSX14.0.sdk

[Bo98]

Were you previously running macOS 11? Currently you need to brew reinstall pkg-config when you do major macOS updates.

[shenlebantongying]

@Bo98 Yes, it fixed the problem, thank you very much :)

Were you previously running macOS 11?

I forget. I got my machine around 2021 Sept, it might be, but I upgraded every versions in between.

---

However this is so frustrating. The process of deep diving into pkg-config to find a workaround is so long :(

Should this be closed? Or i just change the title.

[carlocab]

I tried to fix the handling of this a while back, but ran out of steam. I could try looking at it again, though I'm happy for someone else to take a stab at it.

We can keep this issue open for now.

[alebcay]

I'm interested in having a stab at this, but I'm not sure what the best path would be to fix it. The search path for pkg-config is baked into pkg-config binary at build time, currently set to include #{HOMEBREW_LIBRARY}/Homebrew/os/mac/pkgconfig/#{MacOS.version}.

I would propose that we instead include something like #{HOMEBREW_LIBRARY}/Homebrew/os/mac/pkgconfig/current, which will be a symlink to #{MacOS.version}. (The symlink should not be checked into Git - or we can put this symlink somewhere else - maybe somewhere in Cellar? - to avoid this.)

Then the question would be at what point to update the symlink to point at the current version of #{HOMEBREW_LIBRARY}/Homebrew/os/mac/pkgconfig/#{MacOS.version}. Maybe:

• Provide a command for this. brew reconfigure-pkg-config or something.
  • If we see this pattern with more packages, perhaps its worth making this a more general solution, something like dpkg-reconfigure - brew reconfigure pkg-config. This would probably need some DSL additions for more formulae to be able to specify/leverage this.
  • From UX perspective, I think this is not much of an improvement over brew reinstall pkg-config. User still needs to discover the command, and run it themselves.
• Build this check into some brew command that runs often. Somewhere in brew update maybe?
  • Overloading update to do other tasks might be a bad pattern/smell, especially since OS upgrades don't happen often.
  • User could still be stuck with broken/impaired pkg-config until the brew command runs.
• Add a launchd item to contain this and possible future things that need to run outside of the lifecycle of brew commands (e.g. once per boot or something).
  • Need something Linux-equivalent. systemd probably covers most people, but maybe provide a manual option/command that users can hook into their own preferred init systems.
  • Expands the scope of when/how brew-related code is running on the system. Also one more thing to clean up on uninstall, and not contained neatly in HOMEBREW_PREFIX.
• Something else...?

Or maybe there's a better way than this symlink approach, but for most other ideas I can think of, I still see the same issue of user needing to manually run a command (maybe brew can prompt/show a message, but still requires user to run a brew command) or we need some way to automate/do stuff at the right time.

[Bo98]

Alternative, which could apply to non-pkg-config cases like GCC and Clang's DEFAULT_SYSROOT, could be a way of mark formulae that need reinstalling on newer macOS and those will appear as "outdated" if current OS != tab.dig("built_on", "os_version")

[carlocab]

I would propose that we instead include something like #{HOMEBREW_LIBRARY}/Homebrew/os/mac/pkgconfig/current, which will be a symlink to #{MacOS.version}. (The symlink should not be checked into Git - or we can put this symlink somewhere else - maybe somewhere in Cellar? - to avoid this.)

This was essentially the approach I took in #13664. Fine with doing this, but I remember there being concerns about when that symlink would be updated.

[MikeMcQuaid]

Alternative, which could apply to non-pkg-config cases like GCC and Clang's DEFAULT_SYSROOT, could be a way of mark formulae that need reinstalling on newer macOS and those will appear as "outdated" if current OS != tab.dig("built_on", "os_version")

This is the approach I'd like to see: a new DSL that forces an upgrade or reinstall in the case mentioned here.