Multi-target tests to netcoreapp3.1 and net48

[mauroservienti]

There is no need to target net5 or net6, netcoreapp3.1 is sufficient unless we explicitly need new SDK features. Targeting net48 allows testing endpoints still using the .NET Framework.

[Tobias-08]

Hi @mauroservienti ,

is it possible to multitarget net48, too?

We have still endpoints that use net48 because of certain dependencies and testing them with netcoreapp3.1-NServiceBus.IntegrationTesting seems to cause some issues.

[mauroservienti]

It's technically possible (see #191 and #192) the only issue I see is that it would be impossible to have in the same acceptance test a net48 endpoint and a netcoreapp3.1 one.

[Tobias-08]

Great, LGTM. That it's impossible to mix endpoints with different target framework is OK for me (and by design). But that hint could be added to the readme.

[mauroservienti]

@Tobias-08 I just created 1.2.0, once the build is completed, I'll push to nuget.

[Tobias-08]

@mauroservienti Thanks! Unfortunately I have issues in test execution while using 1.2.0 with net48:

System.TimeoutException : Starting endpoints took longer than 2 minutes.

I have seen this with netcoreapp3.1, too, but not in every test run.

This might be specific to my setup but I have no concrete idea what might be wrong on my side.

[mauroservienti]

I'm sorry about that @Tobias-08. I faced a similar issue when implementing multi-targeting in tests running on Linux. All tests were timing out no matter what. The fix for the Linux tests was to force tests to run on a specific test framework, e.g.:

dotnet test [projectname] --framework netcoreapp3.1 --configuration Release

Commit here

I have no idea what could be the underlying issue, but that fixed the specific problem.

That being said, you should have the full test logs available, if it's possible would you mind sharing those with me?

Another thing, would you be able to download the bits from #194 and run the tests in the MySystem.AcceptanceTests project and report back here what's happening?

[Tobias-08]

The --framework parameter does not make a difference on my side (and my scenario does not use multi-targeting). Excerpt from the log:

<ErrorInfo> <Message>System.TimeoutException : Starting endpoints took longer than 2 minutes.</Message> <StackTrace> at NServiceBus.AcceptanceTesting.Support.TaskExtensions.&lt;Timebox&gt;d__0.MoveNext() in /_/src/NServiceBus.AcceptanceTesting/Support/TaskExtensions.cs:line 23&#xD; --- End of stack trace from previous location where exception was thrown ---&#xD; at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)&#xD; at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)&#xD; at NServiceBus.AcceptanceTesting.Support.ScenarioRunner.&lt;PerformScenarios&gt;d__2.MoveNext() in /_/src/NServiceBus.AcceptanceTesting/Support/ScenarioRunner.cs:line 72&#xD; --- End of stack trace from previous location where exception was thrown ---&#xD; at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()&#xD; at NServiceBus.AcceptanceTesting.Support.ScenarioRunner.&lt;PerformScenarios&gt;d__2.MoveNext() in /_/src/NServiceBus.AcceptanceTesting/Support/ScenarioRunner.cs:line 105&#xD; --- End of stack trace from previous location where exception was thrown ---&#xD; at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)&#xD; at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)&#xD; at NServiceBus.AcceptanceTesting.Support.ScenarioRunner.&lt;PerformTestRun&gt;d__1.MoveNext() in /_/src/NServiceBus.AcceptanceTesting/Support/ScenarioRunner.cs:line 49&#xD; --- End of stack trace from previous location where exception was thrown ---&#xD; at NServiceBus.AcceptanceTesting.ScenarioWithContext1.<Run>d__2.MoveNext() in /_/src/NServiceBus.AcceptanceTesting/ScenarioWithContext.cs:line 55
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
 at Elfe.Integrator.IntegrationTests.IntegrationTests.<TestWholeElfeProcess_ValidAntragVorabmeldungTimeout_ElterngeldAntragVersendet>d__4.MoveNext() in C:\src\Elfe.Integrator\Elfe.Integrator\Elfe.Integrator.IntegrationTests\IntegrationTests.cs:line 63
 --- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
 at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter1.BlockUntilCompleted()
 at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
 at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func1 invoke)
at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
at NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.<Execute>b__0()
at NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)
`

The bits from #194 result in errors:

OneTimeSetUp: System.ComponentModel.Win32Exception : The system cannot find the file specified

[mauroservienti]

That specific exception is thrown by the acceptance testing framework when it tries to start endpoints and one or more fail to start within the allocated timeout of 2 minutes. You should in the logs another error message similar to:

Endpoint {Name} failed to start.

Based on the "the system cannot load the specified file" error it sounds like one of the endpoints is not loading one of the required dependencies. Might that be the case?

Are those .NET 4.8 endpoints using the generic host? Would you be able to share with me, privately, some more information, if needed?

[mauroservienti]

Some more thoughts, mostly for my future self. The "system cannot find the specified file" could also be a docker-compose error, that's the error you get on Linux when docker-compose is not installed. However, if that was the case that would have immediately failed the tests because the compose part happens before the NServiceBus acceptance test framework gets invoked.

[Tobias-08]

My own endpoint uses generic host (on net48, no multi targeting). When I use this endpoint in integration tests with netcoreapp3.1 I get expected warnings regarding referencing net48 in netcoreapp3.1 but the tests succeed (at least most of the time). When I use this endpoint in integration tests with net48 the tests always fail. My endpoint uses UseWindowsService (instead of UseConsoleLifetime). My endpoint uses some internal libs so it's difficult to share it as a repro but I can share additional information if you have an idea what to ask for ;-) .

Regarding #194 files: I have no Docker on my machine. Besides that I have no idea which dependencies might not get loaded (the nuget packages shouldn't be a problem).

[mauroservienti]

@Tobias-08 I made a few changes to #194 to remove the dependency on RabbitMQ and thus on Docker too. Can you try pulling and running it one more time?

When your tests fail, the test should output the full debug logs. Would it be possible to share those test logs with me (privately) alongside the diagnostic file that should be in the bin directory?

[Tobias-08]

@mauroservienti The MySystem.AcceptanceTests in the updated bits from #194 are running (and passing) on my machine. No cannot find the file specified anymore.

[mauroservienti]

Is there any chance that one of the endpoints in the test takes really longer than two minutes to start?

[Tobias-08]

@mauroservienti IMHO not really. Endpoint startup takes only a few seconds.

I tested again with a minimal integration test (endpoint with no test-behaviors) and this succeeds (always, also on net48) in about 2.5 sec. My real tests are quite more complex (multiple behaviors with conditions) and I'm wondering if these cause the test framework to run into the timeout (if run on net48). I could share one of my tests as an example on a private channel.

[mauroservienti]

I could share one of my tests as an example on a private channel.

That would be awesome. You can use this Typeform to get in touch. I'll reply via email.

Thanks.

[mauroservienti]

@Tobias-08 I made a couple of changes to #194 to use both SqlTransport and SqlPersistence. The endpoints are configured with the following connection string:

Data Source=.;Initial Catalog=db;User ID=sa;Password=YourStrong@Passw0rd;Max Pool Size=80

Can you please adapt the connection string to your environment, run the tests, and if they succeed try to tweak them so as to reproduce the failure you are observing in your environment?

[Tobias-08]

@mauroservienti I'll have a look. But as the timeouts seem to have something to do with the complexity of our tests I am not sure if I can reproduce them based on your code.

[Tobias-08]

@mauroservienti Your tests succeed on my machine. I'll try to reproduce the issue based on your code but that might be hard...

[mauroservienti]

Thanks, @Tobias-08. As far as I can tell from the logs you shared the problem is somewhere in the endpoint startup path. The exception is thrown by the acceptance testing framework when it tries to start endpoints and one or more fail to start within the allocated timeout of 2 minutes.

[Tobias-08]

@mauroservienti I had no luck reproducing the issue based on your code. Regarding my code I thought again about my theory from my comment above:

It's still current behavior in my code that a minimal integration test (with identical endpoint startup) succeeds and only the complex tests (with multiple behavior.When(condition) fail. That means in my understanding that the endpoint startup is not the problem. Instead I might be doing something wrong in my complex tests or the framework might have a problem with these kind of complex tests.