Affects: Spring Framework 5.2.9.RELEASE, Hibernate 5.4.21.Final

Suspected connection leak in Spring Hibernation 5 integration with using JTA transaction manager. Consider the following sample application context XML:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
    http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd
    http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd">

    <context:component-scan
        base-package="com.github.sammyhk.test" />

    <tx:annotation-driven
        transaction-manager="transactionManager" mode="proxy"
        proxy-target-class="false" />

    <!-- Wildfly managed data source -->
    <jee:jndi-lookup id="dataSource"
        jndi-name="java:jboss/datasources/testDataSource"
        expected-type="javax.sql.DataSource" resource-ref="true" />

    <bean id="sessionFactory"
        class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="annotatedClasses">
            <list>
                <value><![CDATA[com.github.sammyhk.test.MyEntity]]></value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect"><![CDATA[org.hibernate.dialect.Oracle10gDialect]]></prop>
            </props>
        </property>
    </bean>

    <bean id="transactionManager"
        class="org.springframework.transaction.jta.JtaTransactionManager" />


    <!-- Using Hibernate Template to CRUD MyEntity -->
    <bean id="myEntityDao" class="com.github.sammyhk.test.MyEntityDaoImpl">
        <property name="sessionFactory" ref="sessionFactory" />
    </bean>

    <!-- Annotated with @org.springframework.transaction.annotation.Transactional, operate myEntityDao in a transaction -->
    <bean id="myService" class="com.github.sammyhk.test.MyServiceImpl">
        <property name="myEntityDao" ref="myEntityDao" />
    </bean>
</beans>

Calling myService will be suspected connection leak due to Hibernate session has NOT been close. Debugging found org.springframework.orm.hibernate5.SpringSessionSynchronization has NOT been registered due to org.springframework.orm.hibernate5.SpringSessionContext returned at line 122 while SpringSessionSynchronization registration is in line 136.

In this example we tried to use HibernateTransactionManager and it is working correctly, but using JtaTransactionManager encountered such issue.

Comment From: jhoeller

You might need to explicitly set "hibernate.transaction.jta.platform" (see Hibernate's documentation for JTA integration). In such a scenario, Hibernate itself is responsible for auto-closing the Session through its JTASessionContext (here active through its SpringJtaSessionContext subclass), depending on its JtaPlatform configuration. This is the same mechanism as with a non-Spring Hibernate setup in a JTA environment, with Spring just adding read-only support and flush synchronization here.

SpringSessionSynchronization is only registered in a non-JTA setup, specifically if no Hibernate JtaPlatform has been configured. In this case Spring manages the entire transaction synchronization including the Session lifecycle, however, without Hibernate itself being aware of JTA participation or any other transaction specifics. Only in such a setup, Spring is responsible for closing the transactional Hibernate Session itself. This is effectively what you get when using HibernateTransactionManager.

Comment From: sammyhk

I checked the server log, Hibernate already picked the hibernate.transaction.jta.platform to org.hibernate.engine.transaction.jta.platform.internal.WildFlyStandAloneJtaPlatform

15:38:59,536 INFO  [org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator] (ServerService Thread Pool -- 99) {} HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.WildFlyStandAloneJtaPlatform]

As Spring changed the "hibernate.connection.handling_mode" to DELAYED_ACQUISITION_AND_HOLD (or "hibernate.connection.release_mode" to ON_CLOSE in early versions) but no SpringSessionSynchronization is registered to close session after transaction, who is responsible to do so?

FYI: I also tried to add <property name="jtaTransactionManager" ref="jtaTransactionManager" /> to the sessionFactory configuration and it is working (as it changed "hibernate.connection.handling_mode" to DELAYED_ACQUISITION_AND_RELEASE_AFTER_STATEMENT), but if I also changed "hibernate.connection.handling_mode" back to DELAYED_ACQUISITION_AND_HOLD, same issue occurred.

Comment From: jhoeller

Hibernate's JTASessionContext.buildOrObtainSession sets autoClose to true by default, internally registering a Hibernate-specific transaction completion callback that eventually closes the Session. We don't register a synchronization of our own here in order to not interfere with Hibernate's internal resource management. This should implicitly release the JDBC Connection once the Session is auto-closed by Hibernate itself... I'm not sure why this is not happening in your scenario.

The connection release mode is indeed relevant here, and I'd recommend setting it to DELAYED_ACQUISITION_AND_RELEASE_AFTER_STATEMENT for the JTA case... whereas DELAYED_ACQUISITION_AND_HOLD is absolutely recommendable for the non-JTA case. We automatically infer this in our local setJtaTransactionManager method indeed but have no proper place to do it with Hibernate's own JtaPlatform resolution, so the only way to set it there is explicitly in your Hibernate properties. However, this should just be an optimization, JTA support should work with the hold mode as well.

Since Hibernate contains some specific detection code for the standalone WildFly JTA client, I wonder whether this effect could be specific to that setup. Do you see Session close messages in your log? I wonder whether the Sessions are not being closed at all there, or whether they are closed but somehow not triggering the auto-close mechanism for a long-held JDBC Connection.

Comment From: sammyhk

Yes, the case is no one closing the Hibernate Session. I enabled TRACE log and set breakpoint on org.hibernate.internal.SessionImpl close() and closeWithoutOpenChecks(), nothing get called.

Comment From: jhoeller

Hmm this means that Hibernate's own auto-close mechanism for JTA-scoped sessions does not seem to work in this scenario. This not just affects connection releasing, there are other callbacks attached to Session closing as well which will be missing here.

Auto-closing is effectively handled by Hibernate's SessionImpl.afterTransactionCompletion where the shouldAutoClose check is expected to trigger a managedClose step eventually. You could set a breakpoint there to see what's actually happening,

Comment From: sammyhk

It never called SessionImpl.afterTransactionCompletion and I suspected JdbcResourceLocalTransactionCoordinatorImpl has been used instead of JtaTransactionCoordinatorBuilderImpl. Checked TransactionCoordinatorBuilderInitiator will always use JdbcResourceLocalTransactionCoordinatorImpl if "hibernate.transaction.coordinator_class" (or "hibernate.transaction.factory_class" in old versions) has not been set. Added "hibernate.transaction.factory_class" = org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl (or just short name jta) works correctly. Check the document:

If a non-JPA application does not provide a setting for hibernate.transaction.coordinator_class, Hibernate will use jdbc as the default. This default will cause problems if the application actually uses JTA-based transactions. A non-JPA application that uses JTA-based transactions should explicitly set hibernate.transaction.coordinator_class=jta or provide a custom org.hibernate.resource.transaction.TransactionCoordinatorBuilder that builds a org.hibernate.resource.transaction.TransactionCoordinator that properly coordinates with JTA-based transactions.

May I ask if Spring can have some help on this? Maybe add a side note on document that the default setting only work with HibernateTransactionManager, using JtaTransactionManager need to change this hibernate property.

Thank you very much for you help.

Comment From: jhoeller

Thanks a lot for the thorough analysis and the quick turnaround there! So it looks like we can not only document that this property needs to be configured but also implicitly set it behind our setJtaTransactionManager arrangement. Even if the latter uses DELAYED_ACQUISITION_AND_RELEASE_AFTER_STATEMENT, it should still enforce proper Session auto-closing as well.

I'm not sure why Hibernate doesn't set it automatically when it discovered an active JpaPlatform. Also, I get the impression that most Hibernate JTA usage is with JPA-driven setup (also within Spring applications), not with native Hibernate bootstrapping. In any case, we'll do what we can do document this on our end.

Comment From: jhoeller

@sammyhk This is available in the current 5.2.10.BUILD-SNAPSHOT now. Please give it a try with <property name="jtaTransactionManager" ref="jtaTransactionManager" /> style configuration on LocalSessionFactoryBean, I'd like to double-check that the now-exposed hibernate.transaction.coordinator_class=jta property actually makes a difference for such a scenario.

I've also added some further examples and hints to the reference documentation.

Comment From: sammyhk

5.2.10.BUILD-SNAPSHOT is working correctly without connection leak. I also set breakpoint on SessionImpl.afterTransactionCompletion and confirmed that it is get called. I also tried to change hibernate.connection.handling_mode=DELAYED_ACQUISITION_AND_HOLD and it is also working correctly.

Once again, thank you very much.

For completeness, this is the application context XML that I tested:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
    http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd
    http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd">

    <context:component-scan
        base-package="com.github.sammyhk.test" />

    <tx:annotation-driven
        transaction-manager="transactionManager" mode="proxy"
        proxy-target-class="false" />

    <!-- Wildfly managed data source -->
    <jee:jndi-lookup id="dataSource"
        jndi-name="java:jboss/datasources/testDataSource"
        expected-type="javax.sql.DataSource" resource-ref="true" />

    <bean id="sessionFactory"
        class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="jtaTransactionManager" ref="transactionManager" />
        <property name="annotatedClasses">
            <list>
                <value><![CDATA[com.github.sammyhk.test.MyEntity]]></value>
            </list>
        </property>
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.dialect"><![CDATA[org.hibernate.dialect.Oracle10gDialect]]></prop>
                <!-- <prop key="hibernate.connection.handling_mode"><![CDATA[DELAYED_ACQUISITION_AND_HOLD]]></prop> --> <!-- also working when hibernate.connection.handling_mode = DELAYED_ACQUISITION_AND_HOLD -->
            </props>
        </property>
    </bean>

    <bean id="transactionManager"
        class="org.springframework.transaction.jta.JtaTransactionManager" />


    <!-- Using Hibernate Template to CRUD MyEntity -->
    <bean id="myEntityDao" class="com.github.sammyhk.test.MyEntityDaoImpl">
        <property name="sessionFactory" ref="sessionFactory" />
    </bean>

    <!-- Annotated with @org.springframework.transaction.annotation.Transactional, operate myEntityDao in a transaction -->
    <bean id="myService" class="com.github.sammyhk.test.MyServiceImpl">
        <property name="myEntityDao" ref="myEntityDao" />
    </bean>
</beans>

Comment From: jhoeller

That's great to hear! Thanks for the quick turnaround.