Since upgrading from Sprint Boot 2.5.6 to Spring Boot 2.7.9, I see numerous warnings on application shutdown notifying me that threads have not been properly stopped. Through testing, I discovered this was related to the usage of Schedulers in reactor-core. I was able to produce a minimal example. I tried opening a ticket with the reactor-core project, but I was unable to produce the error in the absence of Spring Boot.
Behavior
Error messages on shutdown for each time the code that utilized Schedulers is called.
2023-04-10 11:31:40.640 WARN 24492 --- [on(8)-127.0.0.1] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [parallel-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
2023-04-10 11:31:40.641 WARN 24492 --- [on(8)-127.0.0.1] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [parallel-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
2023-04-10 11:31:40.641 WARN 24492 --- [on(8)-127.0.0.1] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [parallel-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
Code to Reproduce
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
import reactor.core.scheduler.Schedulers;
@RestController
public class GreetingController {
@GetMapping("/greeting")
public Flux<Integer> greeting(@RequestParam(value = "name", defaultValue = "World") String name) {
Flux<Integer> flux = Flux.range(0, 2)
.publishOn(Schedulers.parallel());
return flux;
}
}
Your Environment
dependencies from pom.xml:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
</dependency>
</dependencies>
- Spring Boot version used: 2.7.10 (also happened in 2.7.9)
- JVM version (
java -version): 1.8.0_333 - OS and version (eg
uname -a): Windows
Comment From: wilkinsona
Thanks for the report.
I tried opening a ticket with the reactor-core project, but I was unable to produce the error in the absence of Spring Boot.
What did you include in your attempt? The warnings are being logged by Tomcat. Unless you included Tomcat, the threads that were left running would not have been detected automatically.
Comment From: KatieKoslosky
@wilkinsona , that is an excellent note, I did not include tomcat so there would be nothing to log the error. I guess I need to revisit that.
Comment From: wilkinsona
Yeah, that's right. I'll close this one for now then as it sounds like you should be able to proceed with reporting the problem to the Reactor team. If it turns out that some changes are needed in Boot, please comment again here and we can re-open the issue and take another look.
Comment From: briup01
The web application [ROOT] appears to have started a thread named [AIG-TECH-ASYNC-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)
Comment From: briup01
我也遇到同样的问题,请问你有什么好的解决方案吗
Comment From: wilkinsona
@briup01 I don't think that is the same problem as I doubt that AIG-TECH-ASYNC-1 is a Reactor Core thread.
If you have any further questions, please follow up on Stack Overflow or Gitter. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.
Comment From: KatieKoslosky
@wilkinsona , that is an interesting point. Since the Spring Boot upgrade, we also see these errors on shutdown. I had assumed they were reactor core threads that the related libraries were naming, but if they're unrelated, it seems to me to point to something in Spring Boot, rather than just a reactor core issue.
2023-04-18 08:48:29.863 WARN [on(2)-127.0.0.1] o.a.c.loader.WebappClassLoaderBase [] [] [] : The web application [ROOT] appears to have started a thread named [mssql-jdbc-shared-timer-core-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
2023-04-18 08:48:29.864 WARN [on(2)-127.0.0.1] o.a.c.loader.WebappClassLoaderBase [] [] [] : The web application [ROOT] appears to have started a thread named [HikariPool-1 housekeeper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
Comment From: wilkinsona
Those threads are related to JDBC (one is from the Hikari connection pool and the other is from SQL Server's JDBC driver). When using the auto-configured DataSource they should both be tidied up as part of the DataSource being closed when the application context is closed. If that is not happening then please open a new issue with a minimal sample that reproduces the problem and we can take a look.