Environment: Spring Boot Version: 3.3.1 Java SDK Version: 21 MockServer Version: 5.15.0
Issue Summary:
When running more than 32 integration tests with unique @SpringBootTest configurations, subsequent tests involving asynchronous operations, such as MockServer interacting with WebFlux, fail due to a RejectedExecutionException. This suggests a potential issue with thread/resource management in the context of Spring Boot's test framework.
Steps to Reproduce:
Create test @Suite including 32 test classes, each annotated with @SpringBootTest and varying configurations (e.g., different properties, ports, classes). (See Sample Test Configuration)
Create a 33rd test, ensuring some of these tests perform asynchronous operations using WebFlux.
Run the test suite, running the tests in order.
Expected Behavior:
All tests should pass, with Spring Boot managing thread resources appropriately across different test contexts.
Actual Behavior:
After the 32nd test with a unique @SpringBootTest configuration, any test performing asynchronous operations fails with the following exception:
2024-07-17T13:10:04.782+02:00 WARN 97041 --- [service] [actor-tcp-nio-8] io.netty.channel.AbstractChannel : Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xa8a0026b]
java.util.concurrent.RejectedExecutionException: event executor terminated
at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:931)
...
Observations:
The issue does not occur when the @SpringBootTest annotations are configured with identical properties. It appears that a new thread may be retained for each unique test context, leading to exhaustion of available threads for asynchronous operations.
Sample Test Configuration:
@SpringBootTest(properties = "x=1")
class Example1Test {
@Test
void test() {
assertThat(true).isTrue();
}
}
...
// Repeat classes incrementing properties, so they have a different context
...
@SpringBootTest(properties = "x=32")
class Example32Test {
@Test
void test() {
assertThat(true).isTrue();
}
}
@SpringBootTest
class Example33Test {
// Example with MockServer and call to a method using WebFlux
}
Additional Context:
This behavior is consistent across multiple test runs and environments. The test suite runs fine when the number of unique @SpringBootTest configurations is 32 or fewer.
Comment From: wilkinsona
This is a side-effect of the caching that's performed by the TestContext Framework. There's nothing that we can do about this in Spring Boot as exactly when the available resources will become exhausted depends on the nature of the application, the JVM, operating system, kernel tuning, and so on. You should be able to avoid the problem by reducing the size of the context cache using the spring.test.context.cache.maxSize property set either as a system property or in a spring.properties located at the root of the classpath.
If the above does not help and you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue. We can then re-open the issue and take another look.
You may also be interested in https://github.com/spring-projects/spring-framework/issues/32289. In the case you have described, each context wouldn't be cached at all as it is not going to be reused.
Comment From: Thyago
Thanks for the quick and enlightening reply @wilkinsona It seems that the TestContext is able to handle the contexts correctly, although, when reaching the 32 threads limit, it starts to affect my MockServer tests calling client classes using WebFlux.
I will try to find time to extract the smallest possible sample of this issue and publish here.
Comment From: wilkinsona
It seems that the TestContext is able to handle the contexts correctly, although, when reaching the 32 threads limit, it starts to affect my MockServer tests calling client classes using WebFlux
Ah, in that case I suspect I misunderstood the problem and it's actually occurring once the cache is full and contexts start getting closed. Reducing the max size of the cache would then make the problem occur sooner. I wonder if there's a mismatch between the lifecycle of the contexts and the lifecycle of your MockServer?
I will try to find time to extract the smallest possible sample of this issue and publish here.
Thanks.
Comment From: Thyago
Hi @wilkinsona, just to give an update about the issue. I was finishing a project simulating the issue when I saw that the new Spring Boot version 3.3.2 became available, and that version fixed the issue, which seems to be related to this ticket: https://github.com/spring-projects/spring-boot/issues/38199
I found that after creating 32 different Tests with different contexts (eg. @SpringBootTest(properties="a=1"), @SpringBootTest(properties="a=2"), ... @SpringBootTest(properties="a=32")), the following tests calling a WebClient with specific configuration will fail.
The specific configuration is the use of WebClientSsl bundles, just by adding a line like below:
webclientbuilder.apply(webClientSsl.fromBundle("default"))
Anyway, I could not replicate the issue anymore since version 3.3.2, so I guess the mentioned bug ticket solved the issue. Thanks again for your prompt attention and precious help here.
Comment From: wilkinsona
Thanks for following up, @Thyago. I think that confirms my earlier misunderstanding and that it was, in fact, a problem with a mismatched lifecycle. The cache becoming full and starting to close contexts is essentially the same as @DirtiesContext which causes the context to be closed immediately after use and never cached. https://github.com/spring-projects/spring-boot/issues/38199 fixes this by ensuring that the Reactor resources aren't shared. This then aligns their lifecycle with that of the application context. Apologies for heading off in the wrong direction initially. I'm glad we got there in the end.
Comment From: izeye
It would be better to update labels to "duplicate" to reflect that this is a duplicate of https://github.com/spring-projects/spring-boot/issues/38199.
Comment From: wilkinsona
Thanks, @izeye. I've done that.