swift test --filter does not find tests with names containing ',' when using v5.6.3

[ysabih]

Description

When using Swift 5.3, tests with names containning "," are filtered correctly when using swift test --filter, while when using Swift 5.6.3, they are not. The same tests are found and executed when running swift test with no filter or when executing swift test --list-tests.

Please note that the same problem happens with '.' character on Swift 5.3. I would like to know if this behavior is normal or if I'm not using --filter correctly.

Expected behavior

Tests with names containing special characters like '.' or ',' are filtered correctly, or the unsupported characters are listed on the docs and swift test --help content.

Actual behavior

Tests with names containing special characters like '.' or ',' are filtered correctly when swift test --filter is used.

Steps to reproduce

1. Create the following files:

• Package.swift

// swift-tools-version: 5.6.3
import PackageDescription

let package = Package(
    name: "SolutionTests",
    dependencies: [
    ],
    targets: [
        .testTarget(
            name: "SolutionTests"),
    ]
)

• Tests\LinuxMain.swift (I know LinuxMain.swift is deprecated and that tests are automatically discovered but I'm keeping it because when I use auto-discovery, only the test function names are shown, and it's important for me to keep showing the full names)

import XCTest
@testable import SolutionTests

@main
class App {
    static func main() {
        XCTMain([
            testCase(SolutionTests.allTests)
        ])
    }
}

• Tests\SolutionTests\SolutionTests.swift

import XCTest

class SolutionTests : XCTestCase {
    static var allTests = {
        return [
            ("test1 No special characters", test1),
            ("test2 Won't be executed, because of the comma", test2),
        ]
    }()

    func test1() {
        XCTAssertTrue(true)
    }
    func test2() {
        XCTAssertTrue(true)
    }
}

2. swift tests --list-tests produces the following the output:

Building for debugging...
Build complete! (0.11s)
SolutionTests.SolutionTests/test1 No special characters
SolutionTests.SolutionTests/test2 Won't be executed, because of the comma

3. swift test --filter test1 produces the following output:

...
Test Suite 'Selected tests' started at 2022-10-03 16:00:54.823
Test Suite 'SolutionTests' started at 2022-10-03 16:00:54.824
Test Case 'SolutionTests.test1 No special characters' started at 2022-10-03 16:00:54.824
Test Case 'SolutionTests.test1 No special characters' passed (0.001 seconds)
Test Suite 'SolutionTests' passed at 2022-10-03 16:00:54.826
...

4. swift test --filter test2 produces the following output (test2 is not found):

...
Test Suite 'Selected tests' started at 2022-10-03 16:02:18.262
Test Suite 'Selected tests' passed at 2022-10-03 16:02:18.263
         Executed 0 tests, with 0 failures (0 unexpected) in 0.0 (0.0) seconds
...

5. swift test finds and executes both tests:

...
Test Suite 'All tests' started at 2022-10-03 16:03:02.526
Test Suite 'debug.xctest' started at 2022-10-03 16:03:02.528
Test Suite 'SolutionTests' started at 2022-10-03 16:03:02.528
Test Case 'SolutionTests.test1 No special characters' started at 2022-10-03 16:03:02.529
Test Case 'SolutionTests.test1 No special characters' passed (0.002 seconds)
Test Case 'SolutionTests.test2 Won't be executed, because of the comma' started at 2022-10-03 16:03:02.530
Test Case 'SolutionTests.test2 Won't be executed, because of the comma' passed (0.0 seconds)
Test Suite 'SolutionTests' passed at 2022-10-03 16:03:02.531
         Executed 2 tests, with 0 failures (0 unexpected) in 0.002 (0.002) seconds
Test Suite 'debug.xctest' passed at 2022-10-03 16:03:02.531
         Executed 2 tests, with 0 failures (0 unexpected) in 0.002 (0.002) seconds
Test Suite 'All tests' passed at 2022-10-03 16:03:02.531
         Executed 2 tests, with 0 failures (0 unexpected) in 0.002 (0.002) seconds

Swift Package Manager version/commit hash

5.6.3

Swift & OS version (output of swift --version && uname -a)

Swift version 5.6.3 (swift-5.6.3-RELEASE)
Target: x86_64-unknown-linux-gnu
Linux 89043a4e645e 5.10.16.3-microsoft-standard-WSL2 apple/swift-package-manager#1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[grynspan]

Duplicate of #417