swift build --build-tests doesn't work with --enable-test-discovery flag

[MaxDesiatov]

It appears the lack of tests hurt us here: this pull broke swift build --build-tests registering the --enable-test-discovery flag also. My Android CI builds swift-numerics trunk daily with the latest Swift trunk snapshot builds and those flags: it now errors with the March 1 snapshot.

I can reproduce the crash on linux x86_64 with the following commands, which work with the last Feb. 29 trunk snapshot before this pull:

> ../swift-DEVELOPMENT-SNAPSHOT-2024-03-01-a-ubi9/usr/bin/swift build --build-tests --enable-test-discovery
> .build/x86_64-unknown-linux-gnu/debug/swift-numericsPackageTests.xctest

Originally posted by @finagolfin in #7377 (comment)

[grynspan]

--enable-test-discovery is deprecated and has no effect anymore; it will be removed in a future update. We should move this over to swift-numerics to remove the LinuxMain.swift file. I'm guessing WindowsMain.swift can be updated too.

@MaxDesiatov I don't seem to be able to move the issue over myself (presumably a permissions issue.)

[MaxDesiatov]

I can't move that either. Created apple/swift-numerics#280 in the meantime.

[grynspan]

Reopening while I make sure that --enable-test-discovery doesn't outright break the build. It'll either end up being a one-line fix or a complete removal of the option (save for a vestigial deprecation warning.)

[MaxDesiatov]

With #7391 under review, I don't think this issue should stay open.

[grynspan]

Sure thing. I didn't realize you'd opened that PR! We can forward-dup this issue to that PR then.

[grynspan]

Duplicate of #7391

[finagolfin]

--enable-test-discovery is deprecated and has no effect anymore

The very fact disabling the flag in your pull broke running a package's tests shows it had an effect prior to your pull.

I don't think removing the --enable-test-discovery flag yet is a good idea, eg the most recent 1.2.1 tag of swift-system from a couple years ago has a LinuxMain.swift and so does the most recent 1.0.2 tag of swift-numerics, so you will no longer be able to test those tags with a trunk toolchain if you remove this flag.

There should be a push in the Swift community to remove those files first, particularly in Apple's own repos like these, before you can remove the flag. @tomerd?

[MaxDesiatov]

so you will no longer be able to test those tags with a trunk toolchain if you remove this flag.

You won't be able to tests those packages with swift test --enable-test-discovery, but you can still test them with plain swift test.

[finagolfin]

You won't be able to tests those packages with swift test --enable-test-discovery, but you can still test them with plain swift test.

How so? While that flag is currently unnecessary generally, it is still useful to override older tags of packages that still have a LinuxMain.swift, like the two I just linked. Unless you start ignoring those old files too, this flag is still needed to resolve the ambiguity of whether to use those manual files or not.

[bnbarham]

Reverting this in #7395 as it's blocking our PR testing.

We also seem to be using LinuxMain.swift here, as @finagolfin noted in #7394 (comment) (we check for XCTMain.swift and LinuxMain.swift). This will of course then cause the original issue that @finagolfin originally opened.

As far as I understand, it seems the intention of the --enable-test-discovery deprecation was to also deprecate the entry point files, ie. the tests are automatically discovered on all platforms meant "you don't need --enable-test-discovery or the entry-point files". It's not clear to me whether this was/is clear enough to users though.

I'd say we have a couple of choices here:

1. Remove this flag but also remove the default entry point handling
2. Fix the original issue without removing the flag, @grynspan sounded like he had a fix in mind for that above

As an aside, is --enable-experimental-test-entry-point also broken after #7377?