I'm experiencing a problem when upgrading from Spring Boot 2.7.15 to Spring Boot 3.2. The error is NoClassDefFoundError and the exception is thrown randomly. Examples:
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/LongArrayList$SubList
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ObjectOpenHashMap$EntrySpliterator
Sometimes the NoClassDefFoundError is thrown with an inner class of my business class. So the error is very unlikely due to conflicting dependencies.
So far it seems like all NoClassDefFoundError occur only with inner classes and the error seems to be very random.
After the spring boot upgrade 1000 k8s pods were created but the error occurred only on 4 pods. Hard to replicate and the error is non-deterministic.
Upon looking into the JVM, it was discovered that although classes were loaded successfully on the problematic pods, their associated inner classes were not loaded.
Prior to the error being thrown I don't see any initialization exceptions, the logs look normal.
I'm using gradle 8.5 to build the jar.
Comment From: wilkinsona
Unfortunately, I don't think we'll be able to help you without some more information. The full stack trace of the failure would be useful. We'll also need to know exactly how you're launching the application is this may affect the class loader that's being used.
Comment From: philwebb
@lazarole Are you using 3.2.3? We fixed some classloader race conditions in earlier 3.2 versions.
Comment From: lazarole
Thanks for the response, we are using 3.2.0.
Here is the stacktrace, sorry I had to obfuscate some business package and class names
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
XXX.v2.servlet.ZZZZ.lambda$getResponse$1(ZZZZ.java:137) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.getBBBViewCandidates(NNNNNN.java:260) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.doIIIII(NNNNNN.java:387) ~[!/:2.22.0]
XXX.YYYYYYl.rank(CandidateYYl.java:218) ~[!/:2.22.0]
XXX.YYYYYYl.getTopVVVV(CandidateYYl.java:292) ~[!/:2.22.0]
XXX.common.service.preranking.PPP.getTopVVVV(PPP.java:178) ~[!/:2.22.0]
java.base/java.util.Collection.stream(Collection.java:743) ~[?:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:927) ~[fastutil-8.5.11.jar!/:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:940) ~[fastutil-8.5.11.jar!/:?]
Caused by: java.lang.ClassNotFoundException: it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$EntrySpliterator
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
XXX.v2.servlet.ZZZZ.lambda$getResponse$1(ZZZZ.java:137) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.getBBBViewCandidates(NNNNNN.java:260) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.doIIIII(NNNNNN.java:387) ~[!/:2.22.0]
XXX.YYYYYYl.rank(CandidateYYl.java:218) ~[!/:2.22.0]
XXX.YYYYYYl.getTopVVVV(CandidateYYl.java:292) ~[!/:2.22.0]
XXX.common.service.preranking.PPP.getTopVVVV(PPP.java:178) ~[!/:2.22.0]
java.base/java.util.Collection.stream(Collection.java:743) ~[?:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:927) ~[fastutil-8.5.11.jar!/:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:940) ~[fastutil-8.5.11.jar!/:?]
Caused by: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ObjectOpenHashMap$EntrySpliterator
java.base/java.lang.Thread.run(Thread.java:840) [?:?]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1151) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1196) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:971) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:410) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:471) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:649) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Server.handle(Server.java:179) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:96) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:720) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:809) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:693) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:571) [jetty-security-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:463) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:491) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:886) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1542) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:208) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.EndpointMetricsFilter.doFilter(EndpointMetricsFilter.java:37) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.RequestScopeFilter.doFilter(RequestScopeFilter.java:33) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:195) [jetty-ee10-websocket-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1609) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:547) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
XXX.v2.servlet.ZZZZ.doPost(ZZZZ.java:107) [!/:2.22.0]
XXX.v2.servlet.ZZZZ.getResponse(ZZZZ.java:150) [!/:2.22.0]
XXX.v2.servlet.ZZZZ.getSVCandidatesResult(ZZZZ.java:176) [!/:2.22.0]
XXX.common.service.Concurrent.invokeAllAndWaitAndReturnResults(Concurrent.java:155) ~[!/:2.22.0]
java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205) ~[?:?]
java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ObjectOpenHashMap$EntrySpliterator
2024-02-27 21:03:04,576 [http-93] ERROR XXX.v2.servlet.ZZZZ - Could not get Response. Error occurred during execution.
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
XXX.v2.servlet.ZZZZ.lambda$getResponse$1(ZZZZ.java:137) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.getBBBViewCandidates(NNNNNN.java:260) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.doIIIII(NNNNNN.java:387) ~[!/:2.22.0]
XXX.YYYYYYl.rank(CandidateYYl.java:218) ~[!/:2.22.0]
XXX.YYYYYYl.getTopVVVV(CandidateYYl.java:292) ~[!/:2.22.0]
XXX.common.service.preranking.PPP.getTopVVVV(PPP.java:178) ~[!/:2.22.0]
java.base/java.util.Collection.stream(Collection.java:743) ~[?:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:927) ~[fastutil-8.5.11.jar!/:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:940) ~[fastutil-8.5.11.jar!/:?]
Caused by: java.lang.ClassNotFoundException: it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$EntrySpliterator
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
XXX.v2.servlet.ZZZZ.lambda$getResponse$1(ZZZZ.java:137) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.getBBBViewCandidates(NNNNNN.java:260) ~[!/:2.22.0]
XXX.YYYY.generator.OOO.NNNNNN.doIIIII(NNNNNN.java:387) ~[!/:2.22.0]
XXX.YYYYYYl.rank(CandidateYYl.java:218) ~[!/:2.22.0]
XXX.YYYYYYl.getTopVVVV(CandidateYYl.java:292) ~[!/:2.22.0]
XXX.common.service.preranking.PPP.getTopVVVV(PPP.java:178) ~[!/:2.22.0]
java.base/java.util.Collection.stream(Collection.java:743) ~[?:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:927) ~[fastutil-8.5.11.jar!/:?]
it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap$MapEntrySet.spliterator(Long2ObjectOpenHashMap.java:940) ~[fastutil-8.5.11.jar!/:?]
Caused by: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ObjectOpenHashMap$EntrySpliterator
java.base/java.lang.Thread.run(Thread.java:840) [?:?]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1151) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1196) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:971) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:410) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:471) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:649) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Server.handle(Server.java:179) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:96) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:720) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:809) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:693) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:571) [jetty-security-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:463) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:491) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:886) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1542) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:208) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.EndpointMetricsFilter.doFilter(EndpointMetricsFilter.java:37) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.RequestScopeFilter.doFilter(RequestScopeFilter.java:33) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:195) [jetty-ee10-websocket-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1609) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:547) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
XXX.v2.servlet.ZZZZ.doPost(ZZZZ.java:107) [!/:2.22.0]
XXX.v2.servlet.ZZZZ.getResponse(ZZZZ.java:150) [!/:2.22.0]
XXX.v2.servlet.ZZZZ.getSVCandidatesResult(ZZZZ.java:176) [!/:2.22.0]
XXX.common.service.Concurrent.invokeAllAndWaitAndReturnResults(Concurrent.java:155) ~[!/:2.22.0]
java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205) ~[?:?]
java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ObjectOpenHashMap$EntrySpliterator
2024-02-27 21:03:04,096 [http-91] ERROR XXX.v2.servlet.ZZZZ - Could not get Response. Error occurred during execution.
java.base/java.lang.Thread.run(Thread.java:840) [?:?]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1151) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1196) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:971) [jetty-util-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) [jetty-io-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:471) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:649) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Server.handle(Server.java:179) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:96) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:720) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:809) [jetty-server-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:693) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:571) [jetty-security-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:463) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:491) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:886) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1542) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) [spring-web-6.1.1.jar!/:6.1.1]
org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) [spring-web-6.1.1.jar!/:6.1.1]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:208) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.EndpointMetricsFilter.doFilter(EndpointMetricsFilter.java:37) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
XXX.extensions.servlet.filter.RequestScopeFilter.doFilter(RequestScopeFilter.java:33) [!/:2.22.0]
org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1581) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:195) [jetty-ee10-websocket-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1609) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379) [jetty-ee10-servlet-12.0.3.jar!/:12.0.3]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
jakarta.servlet.http.HttpServlet.service(HttpServlet.java:547) [jakarta.servlet-api-6.0.0.jar!/:6.0.0]
XXX.v2.servlet.BulkCandidateServletV2.doPost(BulkCandidateServletV2.java:148) [!/:2.22.0]
XXX.v2.servlet.BulkCandidateServletV2.getDynamicAllocationResponse(BulkCandidateServletV2.java:232) [!/:2.22.0]
XXX.v2.servlet.BulkCandidateServletV2.getBulkCandidatesResult(BulkCandidateServletV2.java:292) [!/:2.22.0]
XXX.common.service.Concurrent.invokeAllAndWaitAndReturnResults(Concurrent.java:155) ~[!/:2.22.0]
java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205) ~[?:?]
java.base/java.util.concurrent.FutureTask.report(FutureTask.java:121) ~[?:?]
java.util.concurrent.CancellationException: null
But sometimes a similar error would be thrown but that time it would be an inner class of our business code where we have only a bunch of fields.
Comment From: lazarole
We start the jar with a docker image
ENTRYPOINT ["java", "-Dfile.encoding=UTF-8", "-jar", "$jarFilename"]
And the jar is created with the spring boot plugin of gradle
Comment From: lazarole
I think it's worth trying 3.2.3 if there was a race condition in 3.2.0
Thank you
Comment From: wilkinsona
Yes, please do try 3.2.3. It's always worth trying to latest maintenance release before opening a new issue as, even if the problem hasn't been solved, any fix that we can provide will be based off that version.
Comment From: lazarole
Thank you, I will try the new deployment with spring boot 3.2.3 tomorrow.
Comment From: mhalbritter
Could this be the class loader issue when using parallel streams / common pool? I think this was https://github.com/spring-projects/spring-boot/issues/19427.
Comment From: lazarole
The error has disappeared after using Spring Boot 3.2.3 instead of 3.2.0.
Thanks everyone.