With Spring Boot 3.2 and enabled virtual threads, RabbitMQ fails to start if using simple listener container type with concurrency more than hosts CPU logical processors, direct listener container type is working OK.
For e.g., at my machine (Intel i9 with 16 logical processors) I can use RabbitMQ with spring.threads.virtual.enabled: true with spring.rabbitmq.concurrency <= 16. With concurrency above 16 I get:
13:04:28.086 INFO [ main] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#4e93dcb9:0/SimpleConnection@2924f1d8 [delegate=amqp://guest@192.168.10.2:5672/, localPort=59765]
13:05:28.119 ERROR [ main] o.s.a.r.l.SimpleMessageListenerContainer : Consumer failed to start in 60000 milliseconds; does the task executor have enough threads to support the container concurrency?
With disabled virtual threads or when using direct listener type all is working as expected.
Minimal sample to reproduce is here: https://github.com/ztomic/rmq-issue-38563
(RabbitMQ should be running somewhere)
Comment From: wilkinsona
An AmqpIllegalStateException should have been thrown with the underlying cause of the startup failure. Can you please share the details of that exception or, ideally, a minimal example that reproduces it?
Comment From: ztomic
Example is linked in issue description already, https://github.com/ztomic/rmq-issue-38563 Exception is not thrown, at least not logged.
Only consumer failed to start errors are logged.
13:29:53.620 ERROR [ main] o.s.a.r.l.SimpleMessageListenerContainer : Consumer failed to start in 60000 milliseconds; does the task executor have enough threads to support the container concurrency?
13:30:53.630 ERROR [ main] o.s.a.r.l.SimpleMessageListenerContainer : Consumer failed to start in 60000 milliseconds; does the task executor have enough threads to support the container concurrency?
13:31:53.641 ERROR [ main] o.s.a.r.l.SimpleMessageListenerContainer : Consumer failed to start in 60000 milliseconds; does the task executor have enough threads to support the container concurrency?
Comment From: wilkinsona
Sorry, not sure how I missed the sample earlier.
I think this will have to be addressed in Spring AMQP. There's no exception logged as there isn't a failure but there is a threading problem.
It appears that there's a race between the virtual threads that start the consumers and the main thread that waits for the consumers to have started. In the debugger, I have observed the main thread waiting for the consumers to start and holding consumersMonitor. Each of the virtual consumer threads is blocked in SimpleMessageListenerContainer.isActive(BlockingQueueConsumer) trying to lock consumersMonitor. This continues until the main thread gives up on waiting for the consumers to start and logs the error message.
@garyrussell, can you please take a look?
Comment From: garyrussell
We have an open issue for this https://github.com/spring-projects/spring-amqp/issues/2481
Unfortunately, it was not addressed in Spring AMQP 3.1.0 due to a medical problem that had me sidelined for several weeks before the GA. We will try to get it resolved in the 3.1.1 release.
Comment From: wilkinsona
Thanks, @garyrussell.
I'll close this one in favor of https://github.com/spring-projects/spring-amqp/issues/2481.