Since spring-boot 3.3.0 our application fails to start with a similar exception as below. Increasing the max threads of tomcat from 5 to 10 resolves that exception. The problem occurs as soon as there are under 10 threads configured. spring-boot 3.2.6 is not affected. Both versions (3.2.6 and 3.3.0) use the same tomcat version (10.1.24).
Steps to reproduce:
1. Create basic Spring-Boot Project, that uses Apache Tomcat as we Web-Server (https://start.spring.io/ + spring-boot-starter-web dependency)
2. Set server.tomcat.threads.max=9
3. Application fails to start
2024-05-29T17:21:30.629+02:00 INFO 22712 --- [demo] [ main] com.example.demo.DemoApplication : Starting DemoApplication using Java 21.0.3 with ...
2024-05-29T17:21:30.631+02:00 INFO 22712 --- [demo] [ main] com.example.demo.DemoApplication : No active profile set, falling back to 1 default profile: "default"
2024-05-29T17:21:31.285+02:00 INFO 22712 --- [demo] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 8080 (http)
2024-05-29T17:21:31.295+02:00 INFO 22712 --- [demo] [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2024-05-29T17:21:31.295+02:00 INFO 22712 --- [demo] [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.24]
2024-05-29T17:21:31.348+02:00 INFO 22712 --- [demo] [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2024-05-29T17:21:31.349+02:00 INFO 22712 --- [demo] [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 674 ms
2024-05-29T17:21:31.383+02:00 INFO 22712 --- [demo] [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2024-05-29T17:21:31.388+02:00 WARN 22712 --- [demo] [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server
2024-05-29T17:21:31.394+02:00 INFO 22712 --- [demo] [ main] .s.b.a.l.ConditionEvaluationReportLogger :
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-05-29T17:21:31.408+02:00 ERROR 22712 --- [demo] [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.context.ApplicationContextException: Unable to start web server
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:165) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:618) ~[spring-context-6.1.8.jar:6.1.8]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352) ~[spring-boot-3.3.0.jar:3.3.0]
at com.example.demo.DemoApplication.main(DemoApplication.java:12) ~[classes/:na]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:147) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:107) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:516) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:222) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:188) ~[spring-boot-3.3.0.jar:3.3.0]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:162) ~[spring-boot-3.3.0.jar:3.3.0]
... 8 common frames omitted
Caused by: org.apache.catalina.LifecycleException: Failed to start component [org.apache.catalina.core.StandardThreadExecutor@6c931d35]
at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:419) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:186) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:419) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:874) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.startup.Tomcat.start(Tomcat.java:437) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:128) ~[spring-boot-3.3.0.jar:3.3.0]
... 13 common frames omitted
Caused by: java.lang.IllegalArgumentException: null
at org.apache.tomcat.util.threads.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1357) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.tomcat.util.threads.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1279) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.core.StandardThreadExecutor.startInternal(StandardThreadExecutor.java:117) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
... 19 common frames omitted
Comment From: wilkinsona
Thanks for the report. The change in behavior is due to an unanticipated side-effect of https://github.com/spring-projects/spring-boot/issues/36087.
The failure's occurring because the maximum number of threads is less than the minimum number of spare threads. You can avoid it by setting server.tomcat.threads.min-spare=9.
This is necessary with Spring Boot 3.3.0 as Tomcat's no longer creating its executor. When it does create the executor, it calls org.apache.tomcat.util.net.AbstractEndpoint.getMinSpareThreads() which does Math.min(getMinSpareThreadsInternal(), getMaxThreads()).
I'm not sure if we should add similar logic to our creation of the executor. Arguably, having the max threads set to something that's less than min-spare threads is a configuration error and I'm not 100% sure that silently adapting to it is the right thing to do. The counter-argument is that it used to work and it's what Tomcat does by default so we should probably reinstate the old behavior.
Comment From: fabian-froehlich
@wilkinsona Thanks, your suggestion works. Our application starts again.
I would prefer a solution that either - makes the application startable after upgrading the dependency right away or - add a paragraph in the releasenotes / migration guide
Comment From: philwebb
We're going to align with Tomcat default behavior.
Comment From: mhalbritter
I've opened https://bz.apache.org/bugzilla/show_bug.cgi?id=69133 - if this is resolved, we can switch back to the default executor and set the queue size for #36087 via a normal setter.
Comment From: mhalbritter
Tomcat has added a setter to set the maximum queue size. When using this setter, this issue should go away. I've opened https://github.com/spring-projects/spring-boot/issues/41093 for that.