When a JUnit 5 test class is annotated with @EnabledIf and @DirtiesContext with @EnabledIf's property loadContext set to true, if its expression evaluates to false, the context loaded through the annotation is not cleaned up afterwards.

If the expression evaluates to true and the tests are actually run, the context is cleaned up correctly.

A workaround is to put @EnabledIf on each test method, instead of the class.

I ran into this issue running Kafka container tests conditionally, when Kafka consumers were created through @EnableAutoConfiguration and were left over after the test was skipped. They were consuming messages of other tests that would run afterwards, breaking those tests.

Comment From: sbrannen

That sounds like an interesting combination of actors at play.

I'll investigate if we can improve that.

Thanks for raising the issue!

Comment From: sbrannen

Just to clarify, do you have a bare @DirtiesContext declaration at the class level (or an explicit @DirtiesContext(classMode = ClassMode.AFTER_CLASS))?

Comment From: sbrannen

The reason you are experiencing this behavior is that a JUnit Jupiter ExecutionCondition that evaluates to disabled results in the test class not being executed, which means that class-level callbacks are not executed, which means that the DirtiesContextTestExecutionListener is never invoked for a disabled test class.

So, to get this to work, we'll have to add some special logic to the AbstractExpressionEvaluatingCondition in spring-test to honor the @DirtiesContext annotation in such scenarios.

Comment From: igorakkerman

Just to clarify, do you have a bare @DirtiesContext declaration at the class level (or an explicit @DirtiesContext(classMode = ClassMode.AFTER_CLASS))?

I had tried all three options (bare, AFTER_CLASS and AFTER_EACH_TEST_METHOD) and verified it just now. All three show the same behavior.

Comment From: sbrannen

I had tried all three options (bare, AFTER_CLASS and AFTER_EACH_TEST_METHOD) and verified it just now. All three show the same behavior.

Thanks for the feedback.

See https://github.com/spring-projects/spring-framework/issues/26694#issuecomment-801172981 for my analysis of why it doesn't work for all options.

Comment From: igorakkerman

Wow, amazing response and resolution time. Thank you so much for acting so quickly!

Comment From: sbrannen

Wow, amazing response and resolution time. Thank you so much for acting so quickly!

You're welcome.

Please note that this has been fixed in master and backported to 5.2.x.

Feel free to try it out in the latest snapshots for 5.3.6 and 5.2.14 and let us know if it works for you.

Comment From: igorakkerman

Works for me with Spring Boot 2.4.4 and Spring Test 5.3.6-SNAPSHOT. Thanks!