Process.findExecutable(arguments[0]) gives an unexpected result

[RemiBardon]

Hello 👋🏻

I am investigating a bug and I'm not sure if I'm doing something wrong or if the code is not working correctly, can you help me understanding what's happening?

Context

I am packaging Swift for Exherbo.

For various reasons, I'm still working on the Swift 5.7, using the release/5.7 branches.

My problem

When I run swift build, I get error: unknown or missing subcommand 'swift-build'

Investigation

The error is thrown from apple/swift-driver/Sources/swift-driver/main.swift#L68-L75.

I added a few print statements so I can see what's happening:

  // We are running as a subcommand, try to find the subcommand adjacent to the executable we are running as.
  // If we didn't find the tool there, let the OS search for it.
+ print("arguments", arguments)
+ print("path1", String(describing: Process.findExecutable(arguments[0])?.parentDirectory.appending(component: subcommand)))
  let subcommandPath = Process.findExecutable(arguments[0])?.parentDirectory.appending(component: subcommand)
                         ?? Process.findExecutable(subcommand)

To my surprise, while I expected arguments to be something like ["/usr/x86_64-pc-linux-gnu/lib/swift-5.7/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swift", "build"], it says ["swift-build"]!
To my surprise, while I expected arguments to be something like ["/usr/x86_64-pc-linux-gnu/lib/swift-5.7/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swift", "build"] (allowing swift-driver to find the adjacent swift-build binary), it says ["swift-build"]!

$ /usr/x86_64-pc-linux-gnu/lib/swift-5.7/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swift build
arguments ["swift-build"]
path1 nil
error: unknown or missing subcommand 'swift-build'

(The same happens with swift build by the way)

$ swift build
arguments ["swift-build"]
path1 nil
error: unknown or missing subcommand 'swift-build'

FAQ

Yes, swift-build exists and is next to swift and swift-driver.

Yes, adding swift-build' parent directory (/usr/x86_64-pc-linux-gnu/lib/swift-5.7/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin) to the PATH fixes the issue, but it's not what I want.

Am I missing something?

This seems like a bug to me, CommandLine.arguments should not return swift-build yet 🤔

[RemiBardon]

I see that Driver.swift#L1002-L1006 is the reason why I see ["swift-build"], but it hides the initial binary path so I still think this is not what's expected.

I tried Process.findExecutable(CommandLine.arguments[0]) instead of Process.findExecutable(arguments[0]):

  // We are running as a subcommand, try to find the subcommand adjacent to the executable we are running as.
  // If we didn't find the tool there, let the OS search for it.
  print("arguments", arguments)
  print("path1", String(describing: Process.findExecutable(arguments[0])?.parentDirectory.appending(component: subcommand)))
+ print("path2", String(describing: Process.findExecutable(CommandLine.arguments[0])?.parentDirectory.appending(component: subcommand)))
- let subcommandPath = Process.findExecutable(arguments[0])?.parentDirectory.appending(component: subcommand)
+ let subcommandPath = Process.findExecutable(CommandLine.arguments[0])?.parentDirectory.appending(component: subcommand)
                         ?? Process.findExecutable(subcommand)

Unfortunately, when running swift build, it results in Optional(<AbsolutePath:"/usr/host/bin/swift-build">), which is still not what I'd expect…

When running /usr/x86_64-pc-linux-gnu/lib/swift-5.7/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swift build it obviously works, but it's unfortunate I have to use the absolute path…

Could we easily resolve the symlinks? Would you be against the idea for some reason?

[RemiBardon]

I tried many different things to patch my issue, but I could not properly resolve the symlink without taking the risk of introducing platform-specific code.

Guess my only option is to add swift-* binaries to the PATH…

[RemiBardon]

I was about to propose a PR for:

- if subcommandPath == nil || !localFileSystem.exists(subcommandPath!) {
+ guard let subcommandPath = subcommandPath, localFileSystem.exists(subcommandPath) else {
    throw Driver.Error.unknownOrMissingSubcommand(subcommand)
  }

  // Execute the subcommand.
- try exec(path: subcommandPath?.pathString ?? "", args: arguments)
+ try exec(path: subcommandPath.pathString, args: arguments)

then I quickly checked if release/5.9 was still like this and I stumbled upon e0de0af and f207c2a 🙄

Seems like I wasn't the first to have those issues 🥲

Could we then merge them into release/* branches to fix the issue for previous releases?

[finagolfin]

Those two commits you mention are in 5.8.1, which is the latest release, while 5.7 is no longer updated. Any reason you're not using 5.8.1?

[MaxDesiatov]

Could we then merge them into release/* branches to fix the issue for previous releases?

Depends on the timeline. At this stage issues that corresponding PRs are fixing have to be quite critical if not blocking to be considered for release/5.8 and release/5.9. No more releases are cut from release/5.7, so it's highly unlikely any PR would be accepted to a branch that old.