Since upgrading to Spring Boot 3.1.0 from 3.0.6 we are experiencing NPEs when calling .deleteAll() on Spring Data JPA repositories, but not for each entity. I suspect it is related to entities which contain lazy collections, but could not narrow it down further yet.
I have extracted a minimal sample application which shows the issue: https://github.com/pwandl/spring-hibernate-bug-demo
It seems to only occur if the transaction calling .deleteAll has not previously modified the to-be-deleted entities. I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7
Comment From: wilkinsona
Thanks for the sample. While I haven't managed to reproduce the problem without any usage of Spring, I have reproduced it without Spring Boot and Spring Data JPA. I've pushed a commit to my fork of your sample that just uses Spring Framework and Hibernate. It should fail with the same NPE.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
May 24, 2023 5:25:06 PM org.hibernate.jpa.internal.util.LogHelper logPersistenceUnitInformation
INFO: HHH000204: Processing PersistenceUnitInfo [name: default]
May 24, 2023 5:25:06 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 6.2.2.Final
May 24, 2023 5:25:06 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
May 24, 2023 5:25:06 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl logSelectedDialect
INFO: HHH035001: Using dialect: org.hibernate.dialect.H2Dialect, version: 2.1.214
May 24, 2023 5:25:07 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 8.0.0.Final
May 24, 2023 5:25:08 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Hibernate: create table application_user (id bigint not null, createdBy_id bigint, primary key (id))
Hibernate: create table User_userRoles (User_id bigint not null, userRoles varchar(255) check (userRoles in ('USER','ADMIN','OPERATOR')))
Hibernate: create sequence application_user_SEQ start with 1 increment by 50
Hibernate: alter table if exists application_user add constraint FKiag4pd92c0slpoao47m5i793b foreign key (createdBy_id) references application_user
Hibernate: alter table if exists User_userRoles add constraint FKbuid0qjp6u03wg7qj2fnpf4e8 foreign key (User_id) references application_user
Hibernate: select next value for application_user_SEQ
Hibernate: select next value for application_user_SEQ
1
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: select u1_0.id,u1_0.createdBy_id from application_user u1_0
Hibernate: delete from User_userRoles where User_id=?
Hibernate: delete from User_userRoles where User_id=?
Exception in thread "main" org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:570)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152)
at com.example.demo.DemoApplication.deleteUsers(DemoApplication.java:47)
at com.example.demo.DemoApplication.main(DemoApplication.java:41)
Caused by: jakarta.persistence.RollbackException: Error while committing the transaction
at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:65)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:561)
... 5 more
Caused by: jakarta.validation.ValidationException: HV000028: Unexpected exception during isValid call.
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:186)
at org.hibernate.validator.internal.engine.constraintvalidation.SimpleConstraintTree.validateConstraints(SimpleConstraintTree.java:66)
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateConstraints(ConstraintTree.java:75)
at org.hibernate.validator.internal.metadata.core.MetaConstraint.doValidateConstraint(MetaConstraint.java:130)
at org.hibernate.validator.internal.metadata.core.MetaConstraint.validateConstraint(MetaConstraint.java:123)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateMetaConstraint(ValidatorImpl.java:555)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForSingleDefaultGroupElement(ValidatorImpl.java:518)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForDefaultGroup(ValidatorImpl.java:488)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForCurrentGroup(ValidatorImpl.java:450)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateInContext(ValidatorImpl.java:400)
at org.hibernate.validator.internal.engine.ValidatorImpl.validate(ValidatorImpl.java:172)
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:128)
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.onPreUpdate(BeanValidationEventListener.java:92)
at org.hibernate.action.internal.EntityUpdateAction.preUpdate(EntityUpdateAction.java:329)
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:489)
at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:486)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:358)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2296)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1961)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
... 6 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.hibernate.persister.collection.CollectionPersister.isExtraLazy()" because "persister" is null
at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$readSize$0(AbstractPersistentCollection.java:155)
at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
at org.hibernate.collection.spi.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:148)
at org.hibernate.collection.spi.PersistentSet.size(PersistentSet.java:151)
at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:37)
at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:22)
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:180)
... 35 more
Given that we can't (yet?) reproduce the problem without Spring Framework, I think we should get the Framework team to take a look. To that end, please open a Spring Framework issue referencing this issue and my updated version of your sample.
Comment From: quaff
I think it's bug of hibernate 6.2, It works fine when downgrade to 6.1.7, you should report to hibernate team not spring team.
Comment From: wilkinsona
@pwandl already said that in the opening comment:
I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7
However, thus far we've been unable to reproduce the problem without Spring. If you have managed to do so, @quaff, please do share the code.
Comment From: quaff
I would take a look if @pwandl share the code that works fine using hibernate directly without any spring components.
Comment From: jhoeller
As far as I was able to reproduce this, neither Spring transaction management nor HibernateJpaVendorAdapter have any impact here since the problem remains even with plain EntityManagerFactory.createEntityManager / EntityManager.getTransaction().begin() etc and with HibernatePersistenceProvider configuration on our LocalContainerEntityManagerFactoryBean, coming close to plain Hibernate usage in practice.
The one thing that makes it pass is removing the Jakarta EL provider org.apache.tomcat.embed:tomcat-embed-el:10.1.8 from the classpath. Replacing it with org.glassfish:jakarta.el:4.0.2 breaks as well, whereas it still passes with the presence of the plain EL API in the form of jakarta.el:jakarta.el-api:4.0.0.
So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection. I have a suspicion that this is related to Hibernate Validator but it might be in Hibernate ORM 6.2 itself since a plain downgrade to ORM 6.1.7 makes the problem go away.
In any case, this is definitely a Hibernate interaction problem. If someone has the cycles for it, please try to reproduce it with plain Hibernate (bringing a Jakarta EL provider onto the classpath) and report it to the Hibernate project.
Comment From: wilkinsona
So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection
That matches what I saw too. It's size-related validation that calls a method on AbstractPersistentCollection which breaks. Without an EL provider, Bean Validation backs off so the problem doesn't occur.
Comment From: jhoeller
I originally could not reproduce this in the form of a test in spring-orm. It took a while to find the reason: namely that there was no EL provider on the classpath there, whereas it was declared in the standalone repro build. This difference unfortunately was not obvious at all.
You got a point there that without an EL provider, Hibernate ORM seems to back off from applying Hibernate Validator completely. A standalone Hibernate Validator setup fails with a proper exception if there is no EL provider on the classpath; I suppose Hibernate ORM silently swallows that and simply does not validate then.
In any case, I expect the AbstractPersistentCollection problem to be reproducible with plain Hibernate as long as there is a complete Hibernate Validator setup with an EL provider on the classpath. The CollectionPersister is meant to be present during flush but is somehow missing in action here, with nothing that we can do about it from the outside.
Comment From: jhoeller
P.S.: The early failure for an incomplete Hibernate Validator setup can be enforced via setValidationMode(ValidationMode.CALLBACK) on the EMF setup. I wonder whether we should make that the default for Spring-driven setups? On the other hand, what I really want is AUTO with different rules: back off if no validation provider is present but fail early in case of an incomplete validation setup.
Comment From: wilkinsona
My attempt to reproduce the problem without Framework is here. The debugger shows org.hibernate.boot.beanvalidation.BeanValidationEventListener being called but no failure occurs. Perhaps someone who's more familiar with how Framework configures Hibernate can spot what's missing.
Comment From: jhoeller
That almost hits the spot, it just shouldn't reuse the EntityManager instance between the transactions but rather create a fresh EntityManager per transaction. The following change to the pure JPA version makes the same exception appear for me:
public class DemoApplication {
private final EntityManagerFactory entityManagerFactory;
public DemoApplication() {
entityManagerFactory = Persistence.createEntityManagerFactory("test");
}
public static void main(String[] args) {
DemoApplication demoApplication = new DemoApplication();
demoApplication.createUsers();
demoApplication.deleteUsers();
}
private void deleteUsers() {
EntityManager entityManager = entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
for (User user : entityManager.createQuery("Select u from User u", User.class).getResultList()) {
User toDelete = entityManager.find(User.class, user.getId());
entityManager.remove(entityManager.contains(toDelete) ? toDelete : entityManager.merge(toDelete));
}
entityManager.getTransaction().commit();
entityManager.close();
}
private void createUsers() {
EntityManager entityManager = entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
User a = new User();
a.setUserRoles(EnumSet.of(UserRole.USER));
entityManager.persist(a);
User user = new User();
user.setUserRoles(EnumSet.of(UserRole.USER));
user.setCreatedBy(a);
entityManager.persist(user);
entityManager.getTransaction().commit();
entityManager.close();
System.out.println(user.getCreatedBy().getId());
}
}
Comment From: wilkinsona
Thanks, @jhoeller.
@pwandl I've updated my fork of your example and it now reproduces the problem with pure JPA/Hibernate in the pure-jpa branch:
May 25, 2023 5:47:45 PM org.hibernate.jpa.internal.util.LogHelper logPersistenceUnitInformation
INFO: HHH000204: Processing PersistenceUnitInfo [name: test]
May 25, 2023 5:47:46 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 6.2.2.Final
May 25, 2023 5:47:46 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
May 25, 2023 5:47:46 PM org.hibernate.engine.jdbc.connections.internal.ConnectionProviderInitiator initiateService
INFO: HHH000130: Instantiating explicit connection provider: org.hibernate.hikaricp.internal.HikariCPConnectionProvider
[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
[main] INFO com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection conn0: url=jdbc:h2:mem:test user=ROOT
[main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
May 25, 2023 5:47:46 PM org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl logSelectedDialect
INFO: HHH035001: Using dialect: org.hibernate.dialect.H2Dialect, version: 2.1.214
May 25, 2023 5:47:46 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 25, 2023 5:47:46 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 8.0.0.Final
May 25, 2023 5:47:47 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Exception in thread "main" jakarta.persistence.RollbackException: Error while committing the transaction
at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:65)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at com.example.demo.DemoApplication.deleteUsers(DemoApplication.java:30)
at com.example.demo.DemoApplication.main(DemoApplication.java:20)
Caused by: jakarta.validation.ValidationException: HV000028: Unexpected exception during isValid call.
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:186)
at org.hibernate.validator.internal.engine.constraintvalidation.SimpleConstraintTree.validateConstraints(SimpleConstraintTree.java:66)
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateConstraints(ConstraintTree.java:75)
at org.hibernate.validator.internal.metadata.core.MetaConstraint.doValidateConstraint(MetaConstraint.java:130)
at org.hibernate.validator.internal.metadata.core.MetaConstraint.validateConstraint(MetaConstraint.java:123)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateMetaConstraint(ValidatorImpl.java:555)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForSingleDefaultGroupElement(ValidatorImpl.java:518)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForDefaultGroup(ValidatorImpl.java:488)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForCurrentGroup(ValidatorImpl.java:450)
at org.hibernate.validator.internal.engine.ValidatorImpl.validateInContext(ValidatorImpl.java:400)
at org.hibernate.validator.internal.engine.ValidatorImpl.validate(ValidatorImpl.java:172)
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:128)
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.onPreUpdate(BeanValidationEventListener.java:92)
at org.hibernate.action.internal.EntityUpdateAction.preUpdate(EntityUpdateAction.java:329)
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:489)
at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:486)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:358)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2296)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1961)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
... 2 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.hibernate.persister.collection.CollectionPersister.isExtraLazy()" because "persister" is null
at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$readSize$0(AbstractPersistentCollection.java:155)
at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
at org.hibernate.collection.spi.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:148)
at org.hibernate.collection.spi.PersistentSet.size(PersistentSet.java:151)
at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:37)
at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:22)
at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:180)
... 31 more
Please report this to the Hibernate team.
Comment From: pwandl
Thanks for the detailled analysis! Hibernate issue can be found here: https://hibernate.atlassian.net/browse/HHH-16701