Test execution does not stop after throwing exception from @AfterAll

[hem-verma]

👓 What did you see?

When an exception is thrown from the method annotated with @afterall hook, then test execution does not stop. It waits indefinitely. I am seeing below behaviour.

✅ What did you expect to see?

I have written a custom logic to check the code coverage and I want to run this logic when all feature files have been run. I am using @afterall hook for this. When the coverage is not desired then I want to throw an AssertionError with a message. After this the test execution should stop and the error should be shown. However the same is not happening. I cant figure out what it is waiting for. I tried to debug in the classes Runtime.Java and CucumberExecutionContext.java but couldnt find anything.

📦 Which tool/library version are you using?

Java 21
cucumber bom 7.29.0

🔬 How could we reproduce it?

No response

📚 Any additional context?

No response

[mpkorstanje]

I can't reproduce your problem by merely throwing an exception from an after all hook. I.e:

public class StepDefinitions {
   
    @AfterAll
    public static void fail(){
        throw new AssertionError();
    }
}

Could you please clone https://github.com/cucumber/cucumber-java-skeleton and provide a minimal reproducer that demonstrates your problem?

I cant figure out what it is waiting for. I tried to debug in the classes Runtime.Java and CucumberExecutionContext.java but couldnt find anything.

Assuming you are running without parallel execution, then you'd only have to check the main thread. If you run the program without any break points, then pause it once it gets stuck, you can inspect the stack trace of the main thread to get an idea of what is going on.

Looking at your output it seems that the Teamcity plugin has already reported the problem. So you're not stuck in the after hook. Perhaps there is another plugin that is stuck processing the test run finished event.

[hem-verma]

@mpkorstanje Thanks for your reply.

If I use the repo you mentioned, you are right that throwing an exception from AfterAll is not causing the program to get stuck in that repo.

In my repo, I tried to pause and look at the stack trace and I can see below:

In my other repo also, same behavior is happening but the flow is stuck at a different point.

One thing is common in both the cases is that a different run method is also triggered and thats where execution gets stuck. I am not sure why does it not get stuck in case of a success run but If I put a break point at these places, these run method are also executed in success run.

[hem-verma]

One additional note I am only facing this issue when I run it from IDE. mvn test runs fine.

[mpkorstanje]

I don't think the threads you are looking at are relevant. You should look at the main thread.

Right now it doesn't look like a Cucumber problem. But I can't say much more without a minimal reproducer of the problem.

https://stackoverflow.com/help/minimal-reproducible-example

[hem-verma]

I followed like you suggested. When the loader is running on the Test Results, I click on pause after the test are done, then it automatically stops into the threads like I shared. Not sure if I am not doing it correctly.

One thing I know is that it never reached to line 42 in Cucumber Main.java in my case after the exception is thrown from AfterAll.

I also tried to take a thread dump and I see below:

Also I have tried to all plugin from the @ConfigurationParameter annotation.

Its a pity/fortunate that it does not happen with minimum reproducible code. Please guide how should I proceed?

[mpkorstanje]

When the loader is running on the Test Results, I click on pause after the test are done, then it automatically stops into the threads like I shared. Not sure if I am not doing it correctly.

The thread the debugger stops in was the one that was executing when you pressed stop. You kinda have to look at all of them, especially the main thread and see what it is doing or waiting for.

One thing I know is that it never reached to line 42 in Cucumber Main.java in my case after the exception is thrown from AfterAll.
I also tried to take a thread dump and I see below:

I don't see a main thread in there. 🤔

And is that Timer-0 a non-daemon thread that keeps the whole JVM alive?

https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#setDaemon-boolean-

Its a pity/fortunate that it does not happen with minimum reproducible code. Please guide how should I proceed?

No, you got that backwards. 😉

You have to provide a project that allows me to reproduce your problem. The skeleton project is usually a good starting point for making one.

[mpkorstanje]

This does reproduce the problem.

public class StepDefinitions {
    @Given("I have {int} cukes in my belly")
    public void I_have_cukes_in_my_belly(int cukes) {
        new Timer(false).scheduleAtFixedRate(new TimerTask() {
            @Override
            public void run() {
                System.out.println("Hello");
            }
        }, 10, 1000);
    }
    
    @AfterAll
    public static void afterAll(){
        throw new RuntimeException();
    }
}

For legacy reasons, before all and after all hooks throw an exception. They currently can't be included in any sort of reporting yet so that is the only way to inform users that they failed. The exception bypasses System.exit, and then the Timer keeps the JVM alive.

This will be fixed when I get around to doing #3054. For now I think you'll have to make sure your timer is a daemon.

[hem-verma]

Thanks for the update. Good to see that you could reproduce it. Much appreciated. Looking forward for the fix. 👍🏻

I will check this further on my side. For now, atleast the pipeline would work fine for me because it uses mvn test where it works. Or I would not throw an exception from AfterAll and generate a report which will then help the pipeline to fail the test phase. Problem remains in IDE.