Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of process tests #206

Draft
wants to merge 52 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
d261933
Add test proxy endpoint
mauroservienti Oct 16, 2021
a3be73b
Make so endpoints use incompatible target frameworks ad remove not ne…
mauroservienti Oct 16, 2021
7fd2c0c
Stop filtering out assemblies
mauroservienti Oct 16, 2021
e88b4b2
Spike hypothetical test API
mauroservienti Oct 16, 2021
6b4c7c7
Scenario context clients
mauroservienti Oct 16, 2021
aaa4aba
Begin moving types to shared contract classes
mauroservienti Oct 16, 2021
597535f
external process handling
mauroservienti Jan 5, 2022
88d4caa
Update NUnit test adapter
mauroservienti Jan 5, 2022
6cc5eb7
enhance arguments
mauroservienti Jan 5, 2022
e97bb7f
Use gRpc to control remote endpoints
mauroservienti Jan 5, 2022
ec68a71
Add missing reference
mauroservienti Jan 5, 2022
9f48071
Remove the need for configuration preview callback
mauroservienti Jan 6, 2022
fc69253
Make sure the tested endpoint targets only one framework
mauroservienti Jan 6, 2022
2499e70
Shortcut the test proxy startup code
mauroservienti Jan 6, 2022
9022416
Simplify dealing with command line arguments
mauroservienti Jan 6, 2022
c2ed300
Add support for remote endpoint started notification
mauroservienti Jan 6, 2022
0e4c11f
update RabbitMQ image
mauroservienti Jan 6, 2022
c55969f
Update dependencies to re-enable testing against RabbitMQ
mauroservienti Jan 6, 2022
ab6fbf1
Missing reference
mauroservienti Jan 6, 2022
036c0a4
formatting and usings
mauroservienti Jan 6, 2022
3b26769
Minor enhancements to remote endpoint client
mauroservienti Jan 6, 2022
104b748
typos
mauroservienti Jan 6, 2022
ad6415f
Not needed using directives
mauroservienti Jan 6, 2022
3411aca
Make start and stop of remote endpoints more robust
mauroservienti Jan 6, 2022
19007e5
typos
mauroservienti Jan 6, 2022
dbf56f9
Respect the default endpoints startup timeout
mauroservienti Jan 7, 2022
9b90c24
Wait before whens execution, some more logging
mauroservienti Jan 7, 2022
e655441
Required infrastructure for remote whens execution
mauroservienti Jan 7, 2022
e8b701d
Approved API
mauroservienti Jan 7, 2022
80cbad6
in process and out of process in the same test
mauroservienti Jan 7, 2022
cf5db62
not used proto
mauroservienti Jan 7, 2022
1937dec
exclude tests that cannot run on Linux and multi-target tests
mauroservienti Jan 7, 2022
75f77b6
More logging
mauroservienti Jan 7, 2022
6744303
Refactor to stop using streams and first failed attempt at remote send
mauroservienti Jan 7, 2022
bc3a91b
Pilot remote endpoint to send messages and (partially) intercepts sen…
mauroservienti Jan 7, 2022
2e4c186
Check and skip instead of throwing
mauroservienti Jan 8, 2022
5103cca
Do not run Docker ps -a
mauroservienti Jan 8, 2022
84d97bc
Revert "Do not run Docker ps -a"
mauroservienti Jan 8, 2022
2aec7a1
Try to use Windows latest
mauroservienti Jan 8, 2022
276ac97
Use external RabbitMQ
mauroservienti Jan 8, 2022
0e29a23
Use an external action to setup RabbitMQ (I bet it fails...)
mauroservienti Jan 8, 2022
a9daebe
ooops
mauroservienti Jan 8, 2022
34ac646
Do not setup RabbitMQ (not supported on Windows)
mauroservienti Jan 8, 2022
6750df1
Use the learning transport for tests (containers are not supported on…
mauroservienti Jan 8, 2022
cffad8b
Remove tests requirements
mauroservienti Jan 8, 2022
1d2bc46
Note about supported transports
mauroservienti Jan 8, 2022
b71e8b5
MarkdownSnippets documentation changes
actions-user Jan 8, 2022
493c4cf
Introduce the concept of remote operations and do not try to reuse ex…
mauroservienti Jan 8, 2022
d8053df
Approved API
mauroservienti Jan 8, 2022
977f958
Move gRpc types into a Grpc namespace
mauroservienti Jan 9, 2022
fe8ddb1
Create a wrapper type to avoid exposing users directly to gRpc types
mauroservienti Jan 9, 2022
520c488
Approved API
mauroservienti Jan 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
strategy:
matrix:
include:
- os: windows-2019
- os: windows-latest
name: Windows
- os: ubuntu-20.04
name: Linux
Expand Down
32 changes: 20 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ It's important to make sure that each endpoint configuration works as expected,

### Testing message routing

[Message routing](https://docs.particular.net/nservicebus/messaging/routing) is part of the endpoint configuration and is what makes a choreography successful, the only way to test correctness of the message routing set up is to exercise the choreography so that endpoints will use the production routing configuration to exchange messages, and to subscribe to events. If the routing configuration is wrong or is missing pieces the choreography tests wil fail.
[Message routing](https://docs.particular.net/nservicebus/messaging/routing) is part of the endpoint configuration and is what makes a choreography successful, the only way to test correctness of the message routing set up is to exercise the choreography so that endpoints will use the production routing configuration to exchange messages, and to subscribe to events. If the routing configuration is wrong or is missing pieces the choreography tests will fail.

### Testing saga message mappings

Expand All @@ -85,18 +85,18 @@ In theory, it's possible to assert on [saga message mappings](https://docs.parti

Defining an NServiceBus integration test is a multi-step process, composed of:

- Make sure endpoints configuration can be istantiated by tests
- Make sure endpoints configuration can be instantiated by tests
- Define endpoints used in each test; and if needed customize each endpoint configuration to adapt to the test environment
- Define tests and completion criteria
- Assert on tests results

### Make sure endpoints configuration can be istantiated by tests
### Make sure endpoints configuration can be instantiated by tests

One of the goals of end-to-end testing a NServiceBus endpoint is to make sure that what gets tested is the real production code, not a copy of it crafted for the tests. The production endpoint configuration has to be used in tests. To make sure that the testing infrastructure can instantiate the endpoint configuration there are a couple of options, with many variations.

#### Inherit from EndpointConfiguration

It's possible to create a class that inherits from `EndpointConfiguration` and then use it in both the production endpoint and the tests. To make so that the testing infrastructure can automatically instantiate it, the class must have a parameterless constructor, like in the following snippet:
It's possible to create a class that inherits from `EndpointConfiguration` and then use it in both the production endpoint and the tests. To make so that the testing infrastructure can automatically instantiate it, the class must have a parameter-less constructor, like in the following snippet:

<!-- snippet: inherit-from-endpoint-configuration -->
<a id='snippet-inherit-from-endpoint-configuration'></a>
Expand All @@ -109,11 +109,15 @@ public class MyServiceConfiguration : EndpointConfiguration
this.SendFailedMessagesTo("error");
this.EnableInstallers();

this.UseTransport(new RabbitMQTransport(Topology.Conventional, "host=localhost"));
/*
* Any NServiceBus suppported transport can be used. Tests in this
* repostory are using the LearningTransport for the setup simplicity
*/
this.UseTransport(new LearningTransport());
}
}
```
<sup><a href='/src/Snippets/ConfigurationSnippets.cs#L5-L17' title='Snippet source file'>snippet source</a> | <a href='#snippet-inherit-from-endpoint-configuration' title='Start of snippet'>anchor</a></sup>
<sup><a href='/src/Snippets/ConfigurationSnippets.cs#L5-L21' title='Snippet source file'>snippet source</a> | <a href='#snippet-inherit-from-endpoint-configuration' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

Using the above approach can be problematic when configuration values need to be read from an external source, like for example a configuration file. If this is the case the same external configuration source, most of the times with different values, needs to be available in tests too.
Expand All @@ -127,19 +131,23 @@ In case configuration values need to be passed to the endpoint configuration the
```cs
public static class MyServiceConfigurationBuilder
{
public static EndpointConfiguration Build(string endpointName, string rabbitMqConnectionString)
public static EndpointConfiguration Build(string endpointName)
{
var config = new EndpointConfiguration(endpointName);
config.SendFailedMessagesTo("error");
config.EnableInstallers();

config.UseTransport(new RabbitMQTransport(Topology.Conventional, rabbitMqConnectionString));
/*
* Any NServiceBus suppported transport can be used. Tests in this
* repostory are using the LearningTransport for the setup simplicity
*/
config.UseTransport(new LearningTransport());

return config;
}
}
```
<sup><a href='/src/Snippets/ConfigurationSnippets.cs#L19-L33' title='Snippet source file'>snippet source</a> | <a href='#snippet-use-builder-class' title='Start of snippet'>anchor</a></sup>
<sup><a href='/src/Snippets/ConfigurationSnippets.cs#L23-L41' title='Snippet source file'>snippet source</a> | <a href='#snippet-use-builder-class' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

### Define endpoints used in each test
Expand Down Expand Up @@ -310,7 +318,7 @@ An end-to-end test execution can only be terminated by 3 events:
- the test times out
- there are unhandled exceptions

Unhandled exceptions are a sort of problem from the integration testing infrastructure perspecive as most of the times they'll result in messages being retried and eventually ending up in the error queue. Based on this it's better to consider failed messages as part of the done condition:
Unhandled exceptions are a sort of problem from the integration testing infrastructure perspective as most of the times they'll result in messages being retried and eventually ending up in the error queue. Based on this it's better to consider failed messages as part of the done condition:

<!-- snippet: simple-done-condition -->
<a id='snippet-simple-done-condition'></a>
Expand All @@ -323,7 +331,7 @@ Unhandled exceptions are a sort of problem from the integration testing infrastr
<sup><a href='/src/Snippets/DoneSnippets.cs#L15-L20' title='Snippet source file'>snippet source</a> | <a href='#snippet-simple-done-condition' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

Such a done condition has to be read as: "If there are one or more failed messages the test is done, proceed to evaulate the assertions". Obviously this is not enough. In the identified test case scenario the test is done when a saga is invoked (specifically is created, more on this later). A saga invokation can be expressed as a done condition in the following way:
Such a done condition has to be read as: "If there are one or more failed messages the test is done, proceed to evaluate the assertions". Obviously this is not enough. In the identified test case scenario the test is done when a saga is invoked (specifically is created, more on this later). A saga invocation can be expressed as a done condition in the following way:

<!-- snippet: complete-done-condition -->
<a id='snippet-complete-done-condition'></a>
Expand Down Expand Up @@ -453,7 +461,7 @@ The above extension method is far from being complete and doesn't handle all the

## How to install

Using a package manager add a nuget reference to [NServiceBus.IntegrationTesting](https://www.nuget.org/packages/NServiceBus.IntegrationTesting/).
Using a package manager add a NuGet reference to [NServiceBus.IntegrationTesting](https://www.nuget.org/packages/NServiceBus.IntegrationTesting/).

## Background

Expand Down
11 changes: 11 additions & 0 deletions src/MyMessages/Messages/ALonleyMessage.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
using System;
using System.Collections.Generic;
using System.Text;

namespace MyMessages.Messages
{
public class ALonleyMessage
{
public Guid AnIdentifier { get; set; }
}
}
14 changes: 14 additions & 0 deletions src/MyOtherService.TestProxy/MyOtherService.TestProxy.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net48</TargetFramework>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\MyOtherService\MyOtherService.csproj" />
<ProjectReference Include="..\NServiceBus.IntegrationTesting.OutOfProcess.Nsb8\NServiceBus.IntegrationTesting.OutOfProcess.Nsb8.csproj" />
</ItemGroup>

</Project>
15 changes: 15 additions & 0 deletions src/MyOtherService.TestProxy/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
using System;
using Microsoft.Extensions.Hosting;
using NServiceBus;
using Serilog;

namespace MyOtherService.TestProxy
{
public class Program
{
public static void Main(string[] args)
{
MyOtherService.Program.CreateHostBuilder(args).Build().Run();
}
}
}
5 changes: 2 additions & 3 deletions src/MyOtherService/MyOtherService.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>netcoreapp3.1;net48</TargetFrameworks>
<TargetFramework>net48</TargetFramework>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="NServiceBus" Version="8.0.0-alpha.1897" />
<PackageReference Include="NServiceBus.Newtonsoft.Json" Version="3.0.0-alpha.155" />
<PackageReference Include="NServiceBus.RabbitMQ" Version="7.0.0-alpha.124" />
<PackageReference Include="NServiceBus" Version="8.0.0-alpha.1902" />
</ItemGroup>

<ItemGroup>
Expand Down
6 changes: 2 additions & 4 deletions src/MyOtherService/MyOtherServiceConfigurationBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,8 @@ public static EndpointConfiguration Build(string endpointName, string rabbitMqCo
config.UsePersistence<LearningPersistence>();
config.EnableInstallers();

//TODO: renable when a compatible combination of alphas is available
//var transport = new RabbitMQTransport(Topology.Conventional, rabbitMqConnectionString);
//config.UseTransport(transport);
config.UseTransport(new LearningTransport());
var transport = new LearningTransport();
config.UseTransport(transport);

config.SendFailedMessagesTo("error");

Expand Down
14 changes: 14 additions & 0 deletions src/MyService.TestProxy/MyService.TestProxy.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\MyService\MyService.csproj" />
<ProjectReference Include="..\NServiceBus.IntegrationTesting.OutOfProcess.Nsb8\NServiceBus.IntegrationTesting.OutOfProcess.Nsb8.csproj" />
</ItemGroup>

</Project>
16 changes: 16 additions & 0 deletions src/MyService.TestProxy/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
using System;
using System.Diagnostics;
using Microsoft.Extensions.Hosting;
using NServiceBus;
using Serilog;

namespace MyService.TestProxy
{
public class Program
{
public static void Main(string[] args)
{
MyService.Program.CreateHostBuilder(args).Build().Run();
}
}
}
14 changes: 14 additions & 0 deletions src/MyService/ALonleyMessageHandler.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
using MyMessages.Messages;
using NServiceBus;
using System.Threading.Tasks;

namespace MyService
{
class ALonleyMessageHandler : IHandleMessages<ALonleyMessage>
{
public Task Handle(ALonleyMessage message, IMessageHandlerContext context)
{
return Task.CompletedTask;
}
}
}
6 changes: 2 additions & 4 deletions src/MyService/MyService.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,22 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>netcoreapp3.1;net48</TargetFrameworks>
<TargetFramework>netcoreapp3.1</TargetFramework>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="NServiceBus" Version="8.0.0-alpha.1897" />
<PackageReference Include="NServiceBus" Version="8.0.0-alpha.1902" />
<PackageReference Include="NServiceBus.Extensions.Hosting" Version="2.0.0-alpha.144" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="3.1.19" />
<PackageReference Include="NServiceBus.Newtonsoft.Json" Version="3.0.0-alpha.155" />
<PackageReference Include="NServiceBus.RabbitMQ" Version="7.0.0-alpha.124" />
<PackageReference Include="Serilog.Extensions.Hosting" Version="4.2.0" />
<PackageReference Include="Serilog.Settings.Configuration" Version="3.3.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="4.0.1" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\MyMessages\MyMessages.csproj" />
<ProjectReference Include="..\NServiceBus.AssemblyScanner.Extensions\NServiceBus.AssemblyScanner.Extensions.csproj" />
</ItemGroup>

</Project>
8 changes: 2 additions & 6 deletions src/MyService/MyServiceConfiguration.cs
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,13 @@ public MyServiceConfiguration()
: base("MyService")
{
var scanner = this.AssemblyScanner();
scanner.IncludeOnly("MyService.dll", "MyMessages.dll");

this.UseSerialization<NewtonsoftSerializer>();
this.UsePersistence<LearningPersistence>();
this.EnableInstallers();

//TODO: renable when a compatible combination of alphas is available
//var transport = new RabbitMQTransport(Topology.Conventional, "host=localhost;username=guest;password=guest");
//var routing = this.UseTransport(transport);

var routing = this.UseTransport(new LearningTransport());
var transport = new LearningTransport();
var routing = this.UseTransport(transport);

routing.RouteToEndpoint(typeof(AMessage), "MyOtherService");

Expand Down
4 changes: 1 addition & 3 deletions src/MyService/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ public static void Main(string[] args)
CreateHostBuilder(args).Build().Run();
}

public static IHostBuilder CreateHostBuilder(string[] args, Action<EndpointConfiguration> configPreview = null)
public static IHostBuilder CreateHostBuilder(string[] args)
{
var builder = Host.CreateDefaultBuilder(args);
builder.UseConsoleLifetime();
Expand All @@ -25,8 +25,6 @@ public static IHostBuilder CreateHostBuilder(string[] args, Action<EndpointConfi
builder.UseNServiceBus(ctx =>
{
var config = new MyServiceConfiguration();
configPreview?.Invoke(config);

return config;
});

Expand Down
55 changes: 0 additions & 55 deletions src/MySystem.AcceptanceTests/DockerCompose.cs

This file was deleted.

6 changes: 5 additions & 1 deletion src/MySystem.AcceptanceTests/MyOtherServiceTemplate.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
using MyOtherService;
#if NET48_OR_GREATER

using MyOtherService;
using NServiceBus;
using NServiceBus.AcceptanceTesting.Support;
using NServiceBus.IntegrationTesting;
Expand All @@ -18,3 +20,5 @@ protected override Task<EndpointConfiguration> OnGetConfiguration(RunDescriptor
}
}
}

#endif
Loading