@RegisterExtension + WireMockExtension: Timing & state issues when using "static"

[henrykuijpers]

Proposal

There seem to be state/timing issues in WireMockExtension. For the following declaration:

    @RegisterExtension
    final WireMockExtension wireMock = WireMockExtension.newInstance()
        .options(wireMockConfig()
            .disableOptimizeXmlFactoriesLoading(true)
            .dynamicHttpsPort()
            .dynamicPort())
        .build();

it seems that applying "static" to this field, causes a whole other behavior.
To my understanding, all fields and state etc in should be fresh and new for every @test.

Reproduction steps

1. Checkout https://github.com/henrykuijpers/wiremock-timing-issue
2. Run mvn clean install
3. Observe that there is 1 test failure

Also, run the tests in for example IntelliJ IDEA, or similar. Notice that the tests run rather slowly.
Now, remove the "static" keyword from the WireMockExtension, the tests run a lot faster.
Also notice, when removing the "static" keyword, that the one failing test is actually working.

References

No response

[leeturner]

Hi @henrykuijpers

Many thanks for including the example project, it really helps when we are looking into the issue. From looking at your test class I was able to narrow it down to the two tests that had the issue - this test simpleServiceTimeoutConfigurations() that always fails when the WireMockExtension is static and this test timeoutExceptionGetsThrown(). If I disabled timeoutExceptionGetsThrown() and ran the test, all other tests passed regardless of whether it was static or not.

I am now wondering if this is a case of one test leaking into another due to state being changed on a static variable? On line 256 in the timeoutExceptionGetsThrown() test we can see the following line:

wireMock.setGlobalFixedDelay(1000); // Make all requests take way longer than 1s

It looks like it is this line that is causing the issues. Setting the global delay in WireMock on a static WireMockExtension will impact all the tests that run after that line is executed. As you noted in your bug report, the tests seem to run much slower.

The reason the simpleServiceTimeoutConfigurations() test fails is that it is the only other test that sets a timeout (500) on the RestServiceClient:

final RestClientService service = new RestClientService("http://localhost:" + wireMock.getRuntimeInfo().getHttpPort(), 500);

All the other clients are created with no timeout which is why they appear slower but don't fail:

final RestClientService restClientService = new RestClientService("http://localhost:" + wireMock.getRuntimeInfo().getHttpPort(), -1);

To solve this issue and to keep the WireMockExtension static you will need to reset the state at the end of the timeoutExceptionGetsThrown() test by setting the global timeout back to 0:

wireMock.setGlobalFixedDelay(0); // reset global timeout

Or if you want to do it the jUnit way:

    @BeforeEach
    void setUp() {
        wireMock.setGlobalFixedDelay(0);
    }

[leeturner]

Hi, we are planning on releasing 3.6.0 this week if possible. This issue will be closed at that time. If there is anything you think I missed in my comment above which makes you think this is still an issue, please let me know