Affects: 5.1.2, 5.3.9
Background: We experienced an issue due to DefaultMessageListenerContainer no longer receiving messages. This was because of a misunderstanding on our part how ThreadPoolExecutorFactoryBean works. We used following settings for the executor:
corePoolSize=5 maximumPoolSize=10 queueCapacity=1
which was injected into a DefaultMessageListenerContainer with these parameters:
concurrentConsumers=1 maxConcurrentConsumers=10 maxMessagesPerTask=1
The goal was to have a dynamic scaling of threads aligned with the scaling of consumers; however we shouldn't have set neither a queue capacity nor a maximum pool size and rather used something like Executors.newCachedThreadPool(); after all, the container will take care of not scheduling more than maxConcurrentConsumers * 2 threads in the worst case (since each task reschedules itself if all are busy).
Our problem was reproducible given this test, the log is available here (I killed the process after no more messages were being received).
The issue: There are some rejected task debug level log entries due to the executor misconfiguration, but no warn or error entries. I would expect at least the error "All scheduled consumers have been paused, probably due to tasks having been rejected. Check your thread pool configuration! Manual recovery necessary through a start() call." at DefaultMessageListenerContainer.java#L1180-L1183 to appear. I don't understand why it didn't, since everything happens inside locks held on the lifecycleMonitor.
On a side note, why is the rejection of tasks logged only at debug level (not even info)? After all, dropping below concurrentConsumers raises a warning (this works in the test if I set concurrentConsumers to 2). Is there any use case of configuring an executor that occasionally rejects tasks?
Comment From: jhoeller
Not sure why the error did not get logged, but in any case, each individual rejection should get logged at warn level at least. While executor implementations may temporarily reject tasks for any reason, it really isn't the norm, in particular not with the common executor implementations out there. I'll therefore backport the log level change to 5.2.18 as well.