Affects: \
We are using Spring-jms:5.3.6 to receive messages from Amazon SQS, our relevant configuration is:
@Bean
public DefaultJmsListenerContainerFactory listenerContainerFactory(SQSConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("10-50");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return factory;
}
We receive the message using the following code:
@JmsListener(destination = "${amazon.config.sqs-name: }"
, containerFactory = "listenerContainerFactory")
public void consumer(SQSTextMessage sqsTestMessage) throws Exception {
//...
}
However, after a period of time, we found that JMS stopped working, so we used arthas to view the thread information, and arthas indicated that there was a deadlock. The relevant thread information is as follows:
[arthas@12275]$ thread 517
"org.springframework.jms.JmsListenerEndpointContainer#0-6" Id=517 BLOCKED on java.lang.Object@f8dca owned by "org.springframework.jms.JmsListenerEndpointContainer#0-23" Id=664
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1198)
- blocked on java.lang.Object@f8dca
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
at java.lang.Thread.run(Thread.java:748)
[arthas@12275]$ thread 664
"org.springframework.jms.JmsListenerEndpointContainer#0-23" Id=664 BLOCKED on java.lang.Object@f8dca owned by "org.springframework.jms.JmsListenerEndpointContainer#0-6" Id=517
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1198)
- blocked on java.lang.Object@f8dca
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
at java.lang.Thread.run(Thread.java:748)
However, the thread information we got through the java command did not find deadlock information. jstack shows that only one thread is working, but this thread does not receive the message
"org.springframework.jms.JmsListenerEndpointContainer#0-24" #687 prio=5 os_prio=0 tid=0x00007f32a0045000 nid=0x3348 runnable [0x00007f31efab3000]
java.lang.Thread.State: RUNNABLE
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1198)
- locked <0x0000000681c1c0b0> (a java.lang.Object)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
Full Jstack info here: amzzt_stack.log
Comment From: nagarsogesn
Any luck here?
Facing same issue:
stackTrace:
java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(Unknown Source)
- waiting to lock <0x0000000614b699c8> (a java.lang.Object)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(Unknown Source)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
Thanks!
Comment From: bclozel
Sorry this got overlooked.
From what I've seen in this report, both threads would be blocked on the following line: https://github.com/spring-projects/spring-framework/blob/v5.3.6/spring-jms/src/main/java/org/springframework/jms/listener/DefaultMessageListenerContainer.java#L1198
Both would be blocked on a simple synchronized block - I don't understand how that would be possible in this case. A lock management issue in Spring Framework would be actionable if the DefaultMessageListenerContainer
would use several locks or not release a lock properly.
I'm closing this issue right now but we'll reopen if we get more reliable thread information or a reproducible example. Thanks!