[Cxx Interop]: Support a Seamless Integration with Existing Cxx Projects

[furby-tm]

Description

Some Background

Swift/Cxx Interop is currently plagued with the current limitation of its especially strict requirement that a project adhere exactly to the structural demands in the following form:

ProjectRoot/
  Sources/
    CxxLibrary/
      include/
        CxxLibrary/
          CxxLibrary.h (umbrella)
      CxxSource.cpp (source)
  Package.swift

While there is some bit of flexibility one can achieve through a carefully crafted module.modulemap file, it is not a proper solution in terms of allowing the flexibility necessary in order for existing Cxx projects to interoperate with the Swift programming language, maintainers of these existing projects require a seamless integration if they are to incorporate a SwiftPM Package manifest file at the root of their project repositories, and most importantly -- to see rapid adoption of the Swift programming language and its open source ecosystem flourish.

Flexible Configurations

While this flexibility could potentially be leveraged from within the Package manifest files themselves through exposing more configuration options in terms of how headers are realized by the tooling (ex. publicHeadersPath: ""), I believe there might be a far more flexible solution, one tailored specifically for the nature of existing Cxx projects (ex. CMake), rather than pigeonholing existing Cxx projects to attempt to fit into a standard SwiftPM project paradigm.

Powerful Package Plugins

Which finally leads me to say, that it might be in the best interest for both Swift as well as Cxx if:

If it is deemed that exposing more flexible (Cxx) configurations to package manifest files could prove problematic or incompatible with what's best for Swift and/or Cxx, maybe things could be more tooled out on the PackagePlugin side of things.

Perhaps with either:

• the existing BuildToolPlugin API.
• or perhaps a brand new API tailored around CMake and other tooling commonly found in existing Cxx projects.
• or perhaps even allowing custom designed Build Plugins to be written in Swift, exposing some of the internal tooling of what PackagePlugin currently encompasses.

BuildToolPlugin

^{Perhaps Low Hanging Fruit}

Tip

I have provided an example project with CI hooked up to help aid in reproducing this issue.

For now, so long as this following documentation note holds true for the prebuildCommand() documented below:

outputFilesDirectory
A directory into which the command writes its output files.
Any files there recognizable by their extension as source files (e.g. . swift) are
compiled into the target for which this command was generated as if in its source
directory; other files are treated as resources
as if explicitly listed in Package.swift using .process(...).

Then theoretically it should be relatively simple to support the generation of (*.c, *.cpp, *.h, *.hpp) files to that directory, one such implementation example that could prove incredibly beneficial - could be git cloning a Cxx project from source, and into that directory with prebuildCommand().

Expected behavior

In it's simplest form, I expected the generation of a header file for a Cxx module to be correctly picked up by SwiftPM:

import PackagePlugin

@main
struct GenerateUmbrellaHeadersPlugin: BuildToolPlugin
{
  func createBuildCommands(context: PluginContext, target: Target) throws -> [Command]
  {
    guard let target = target.sourceModule
    else { return [] }

    let outputDir = context.pluginWorkDirectory.appending("\(target.name)/include/\(target.name)")

    return [
      .prebuildCommand(
        displayName: "Generating Cxx Umbrella Header for: \(target.name)",
        executable: .init("/usr/bin/touch"),
        arguments: ["\(outputDir.appending("\(target.name).h").string)"],
        outputFilesDirectory: outputDir
      ),
    ]
  }
}

As the docs indicate:

Returns a command that runs unconditionally before every build.

static func prebuildCommand (
  displayName: String?,
  executable: Path,
  arguments: [any CustomStringConvertible],
  environment: [String: any CustomStringConvertible] = [:],
  outputFilesDirectory: Path
) -> Command

Parameters

displayName
An optional string to show in build logs and other status areas.

executable
The absolute path to the executable to be invoked.

arguments
Command-line arguments to be passed to the executable.

environment
Environment variable assignments visible to the executable.

workingDirectory
Optional initial working directory when the executable runs.

outputFilesDirectory
A directory into which the command writes its output files. Any files there recognizable by their extension as source files (e.g. . swift) are compiled into the target for which this command was generated as if in its source directory; other files are treated as resources as if explicitly listed in Package. swift using process (...).

Actual behavior

Instead, while the header file properly generated into the outputFilesDirectory, I received the following warning:

warning: Headers generated by plugins are not supported at this time:
.build/plugins/.../CxxModule/include/CxxModule/CxxModule.h

Steps to reproduce

1. Clone the following SwiftPM project.

   git clone https://github.com/wabiverse/SwiftBuildToolPluginNoInterop.git

2. Then, "cd" into it.

   cd SwiftBuildToolPluginNoInterop

3. Build it.

   swift build

4. Notice the warning.

   warning: Headers generated by plugins are not supported at this time:
   .build/plugins/.../CxxModule/include/CxxModule/CxxModule.h

Swift Package Manager version/commit hash

Swift 5.9.2

Swift & OS version (output of swift --version && uname -a)

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx14.0

[furby-tm]

I cloned SwiftPM and am attempting to enable Cxx Interop for generated headers/source files for BuildToolPlugin:

$ swift-package-manager-main/.build/arm64-apple-macosx/debug/swift-build        
Building for debugging...
In file included from SwiftBuildToolPluginNoInterop/.build/plugins/.../CxxModule/GenerateUmbrellaHeadersPlugin/CxxModule/include/CxxModule/CxxModule.h:6:
SwiftBuildToolPluginNoInterop/Sources/CxxModule/include/CxxModule/cxxA.h:9:1: error: unknown type name 'class'
class Example
^
SwiftBuildToolPluginNoInterop/Sources/CxxModule/include/CxxModule/cxxA.h:9:14: error: expected ';' after top level declarator
class Example
             ^
             ;
2 errors generated.
[3/9] Write swift-version--58304C5D6DBC2206.txt

As far as I can tell this appears quite close, what I do know for certain is that this is definitely stuck in C Interop mode, not sure how trivial it would be to punch this into Cxx Interop mode??

I believe the generated source (and resources) are just missing their computeRule, perhaps?:

private static func computeRule(
    for path: AbsolutePath, 
    toolsVersion: ToolsVersion,
    rules: [FileRuleDescription],
    declaredResources: [(path: AbsolutePath, rule: TargetDescription.Resource.Rule)],
    declaredSources: [AbsolutePath]?,
    matchingResourceRuleHandler: (AbsolutePath) -> () = { _ in },
    observabilityScope: ObservabilityScope
) -> FileRuleDescription.Rule

[code-per-day]

In migrating a few libraries, the biggest issue I found is that only a single public header path is allowed. Having the ability to add more would solve most of the problematic libs I've encountered.

[furby-tm]

In migrating a few libraries, the biggest issue I found is that only a single public header path is allowed. Having the ability to add more would solve most of the problematic libs I've encountered.

Exactly! And the ability to ensure if multiple include paths share a same root directory, which is typically the case, ex:

ProjectRoot/
  src/
    CxxLibA/
      CxxLibA.h(umbrella)
      CxxLibA.cpp (source)
    CxxLibB/
      CxxLibB.h(umbrella)
      CxxLibB.cpp (source)
  Package.swift

In C++ with includes such as these:

#include <CxxLibA/CxxLibA.h>
#include <CxxLibB/CxxLibB.h>

To be able to disambiguate the following Package manifest config header paths (so one target doesn't incorrectly include the other target's headers), for example, import CxxLibA should not also include CxxLibB's publicHeadersPath, but there's no way to prevent that from happening:

.target(
  name: "CxxLibA",
  path: "src",
  exclude: [
    "CxxLibB" /* XXX this does NOT exclude CxxLibB's headers below (it should). */
  ],
  publicHeadersPath: "." /* XXX conflicts with CxxLibB headers. */
),

.target(
  name: "CxxLibB",
  dependencies: [
    /* okay, we need CxxLibA's public headers path. */
    .target(name: "CxxLibA")
  ],
  path: "src",
  exclude: [
    "CxxLibA" /* XXX this does NOT exclude CxxLibA's headers below (it should). */
  ],
  publicHeadersPath: "." /* XXX conflicts with CxxLibA headers. */
),

Perhaps this could be resolved via a brand new additional parameter:

publicHeadersExclude: [
  "CxxLibA"
]

Or possibly even by just allowing the existing exclude parameter to additionally exclude any header files within any of its specified directories, meaning the following would not only exclude all source files in the CxxLibA directory, it would additionally exclude any other potential header files in that same directory:

exclude: [
  "CxxLibA"
]