Archie Cobbs opened SPR-9512 and commented
I have a bean factory with this:
<task:annotation-driven executor="taskExecutor" scheduler="taskScheduler"/>
<task:executor id="taskExecutor" pool-size="2-10" queue-capacity="100" keep-alive="10"/>
<task:scheduler id="taskScheduler" pool-size="1"/>
and I'm occasionally seeing this exception on bean factory shutdown:
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'transactionManager':
Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean
from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:212)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.transaction.interceptor.TransactionAspectSupport.determineTransactionManager(TransactionAspectSupport.java:248)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:272)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$before$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:63)
at com.foobar.DefaultQueueProcessor.runQueue(DefaultQueueProcessor.java:50)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:64)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
A scheduled task is starting and then dying because the bean factory is shutting down.
It seems like an improvement would be: before shutting down a bean factory, find all beans that perform scheduled tasks and stop them, so they don't generate new activity during bean factory destruction.
Affects: 3.1.1
9 votes, 12 watchers
Comment From: spring-projects-issues
Thorsten Vogel commented
The same happens in a web environment (with web flow) when shutting down tomcat, running a web application context that has scheduled tasks (same configuration as OP).
Spring 3.2.4
WARN : org.springframework.web.context.support.GenericWebApplicationContext - Exception thrown from ApplicationListener handling ContextClosedEvent org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'org.springframework.scheduling.config.ContextLifecycleScheduledTaskRegistrar#0': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:210) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:198) at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:155) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:86) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:334) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:336) at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1049) at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1010) at org.springframework.webflow.engine.Flow.destroy(Flow.java:597) at org.springframework.webflow.engine.builder.DefaultFlowHolder.destroy(DefaultFlowHolder.java:100) at org.springframework.webflow.definition.registry.FlowDefinitionRegistryImpl.destroy(FlowDefinitionRegistryImpl.java:114) at org.springframework.webflow.config.FlowRegistryFactoryBean.destroy(FlowRegistryFactoryBean.java:161) at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:238) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:510) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:486) at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:742) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:455) at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1090) at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1064) at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1010) at org.springframework.web.servlet.FrameworkServlet.destroy(FrameworkServlet.java:794) at org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1482) at org.apache.catalina.core.StandardWrapper.stopInternal(StandardWrapper.java:1843) at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5614) at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:232) at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1575) at org.apache.catalina.core.ContainerBase$StopChild.call(ContainerBase.java:1564) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)
Comment From: spring-projects-issues
Arne Hildebrand commented
We're still facing the issue in the Spring 4.1.4. Through we do have a small fixed delay (1000ms) to trigger our task ...
Comment From: spring-projects-issues
Botez Bianca-Elena commented
This issue is still reproducing in Spring 4.2.3
Comment From: spring-projects-issues
Juergen Hoeller commented
Could you post your specific stacktrace please? Or is it identical to one of the above?
I suppose the "waitForTasksToCompleteOnShutdown" and "awaitTerminationSeconds" settings would help a bit here, but indeed, some beans might get destroyed before the executor/scheduler beans depending the order of bean registration... and if exactly such beans get dynamically obtained from scheduler tasks, the above warning would get logged.
If the registration/shutdown order of beans is indeed the root of the issue, we can try to do something about it within the 4.3 timeframe still.
Comment From: spring-projects-issues
Ike Chen commented
I am facing the same issue.
sequence: 1. Tomcat server shutdown; 2. Task in ThreadPoolTaskExecutor continue to run with "waitForTasksToCompleteOnShutdown" and "awaitTerminationSeconds" set; 3. BeanCreationNotAllowedException thrown because task instance need to be injected with a scoped bean (in my case: scope="step", from spring batch);
I think "shutdown order" might help, with which, I may set my ThreadPoolTaskExecutor to be on the top of the shutdown sequence. Looking forward for this improvement in 4.3 timeframe. Thx.
Comment From: spring-projects-issues
Fabien MARSAUD commented
Same thing with SpringBoot 1.3.3. Though we noticed the issue is systematic on our test environment, and isn't present on our production environment.
TEST: SUSE Linux Enterprise Server 11 (x86_64) patch 1 PROD: Ubuntu 14.04.1 LTS
So, whenever we try to stop a SpringBoot on our TEST environment using SIGTERM (kill -15), the process remains active. Then if we try to call the application (it exposes rest services), we get that output in the browser:
HTTP Status 500 - Request processing failed; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'departureResource': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
type Exception report
message Request processing failed; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'departureResource': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
description The server encountered an internal error that prevented it from fulfilling this request.
exception
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'departureResource': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:980)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:859)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:844)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:237)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:112)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
root cause
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'departureResource': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:216)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
org.springframework.web.method.HandlerMethod.createWithResolvedBean(HandlerMethod.java:245)
org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:314)
org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:60)
org.springframework.web.servlet.handler.AbstractHandlerMapping.getHandler(AbstractHandlerMapping.java:351)
org.springframework.web.servlet.DispatcherServlet.getHandler(DispatcherServlet.java:1120)
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:932)
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:968)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:859)
javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:844)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:237)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:112)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
note The full stack trace of the root cause is available in the Apache Tomcat/8.0.32 logs.
Apache Tomcat/8.0.32
On the logs, we have:
2016-04-26 11:55:38.722 INFO 24059 --- [Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@2b458293: startup date [Wed Apr 20 12:06:02 GMT 2016]; root of context hierarchy
2016-04-26 11:55:38.725 INFO 24059 --- [Thread-2] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@2635bf84: startup date [Wed Apr 20 12:06:06 GMT 2016]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@2b458293
2016-04-26 11:55:38.728 INFO 24059 --- [Thread-2] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 0
2016-04-26 11:55:38.729 INFO 24059 --- [Thread-2] c.n.e.EurekaDiscoveryClientConfiguration : Unregistering application departure-query with eureka with status DOWN
2016-04-26 11:55:38.742 INFO 24059 --- [Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
2016-04-26 11:55:38.742 INFO 24059 --- [Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans
2016-04-26 11:55:38.743 INFO 24059 --- [Thread-2] o.s.c.n.servo.ServoMetricCollector : Stopping Servo PollScheduler
2016-04-26 11:55:38.746 INFO 24059 --- [Thread-2] c.n.e.EurekaDiscoveryClientConfiguration : Closing DiscoveryClient.jerseyClient
2016-04-26 11:55:38.747 INFO 24059 --- [Thread-2] c.n.e.EurekaDiscoveryClientConfiguration : Removing application departure-query from eureka
2016-04-26 11:55:38.751 WARN 24059 --- [Thread-2] com.netflix.discovery.DiscoveryClient : Action: Cancel => returned status of 404 from http://sagq.deolan.biz:8761/eureka/apps/DEPARTURE-QUERY/sagq
2016-04-26 11:55:38.756 ERROR 24059 --- [Thread-2] com.netflix.discovery.DiscoveryClient : Can't get a response from http://sagq.deolan.biz:8761/eureka/apps/DEPARTURE-QUERY/sagq
Can't contact any eureka nodes - possibly a security group issue?
java.lang.RuntimeException: Bad status: 404
at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1155) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1060) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.unregister(DiscoveryClient.java:694) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.shutdown(DiscoveryClient.java:684) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryManager.shutdownComponent(DiscoveryManager.java:94) [eureka-client-1.1.147.jar!/:1.1.147]
at org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration.close(EurekaDiscoveryClientConfiguration.java:93) [spring-cloud-netflix-core-1.0.7.RELEASE.jar!/:1.0.7.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_73]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_73]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:354) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:319) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:153) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:243) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:972) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:979) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1006) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:982) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:934) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.boot.builder.ParentContextCloserApplicationListener$ContextCloserListener.onApplicationEvent(ParentContextCloserApplicationListener.java:102) [spring-boot-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.builder.ParentContextCloserApplicationListener$ContextCloserListener.onApplicationEvent(ParentContextCloserApplicationListener.java:86) [spring-boot-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:163) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:136) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:381) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:335) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:967) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:901) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
2016-04-26 11:55:38.757 ERROR 24059 --- [Thread-2] com.netflix.discovery.DiscoveryClient : DiscoveryClient_DEPARTURE-QUERY/sagq - de-registration failedBad status: 404
java.lang.RuntimeException: Bad status: 404
at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1155) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1060) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.unregister(DiscoveryClient.java:694) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryClient.shutdown(DiscoveryClient.java:684) [eureka-client-1.1.147.jar!/:1.1.147]
at com.netflix.discovery.DiscoveryManager.shutdownComponent(DiscoveryManager.java:94) [eureka-client-1.1.147.jar!/:1.1.147]
at org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration.close(EurekaDiscoveryClientConfiguration.java:93) [spring-cloud-netflix-core-1.0.7.RELEASE.jar!/:1.0.7.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_73]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_73]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:354) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:319) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:153) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:243) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:972) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:979) [spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1006) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:982) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:934) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.boot.builder.ParentContextCloserApplicationListener$ContextCloserListener.onApplicationEvent(ParentContextCloserApplicationListener.java:102) [spring-boot-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.builder.ParentContextCloserApplicationListener$ContextCloserListener.onApplicationEvent(ParentContextCloserApplicationListener.java:86) [spring-boot-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:163) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:136) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:381) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:335) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:967) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:901) [spring-context-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
2016-04-26 11:55:46.280 ERROR 24059 --- [http-nio-8096-exec-9] s.c.c.d.h.DiscoveryClientHealthIndicator : Error
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'scopedTarget.eurekaDiscoveryClient': Singleton bean creation not allowed while the singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:216) ~[spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) ~[spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35) ~[spring-aop-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.getTarget(CglibAopProxy.java:687) ~[spring-aop-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:637) ~[spring-aop-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at com.netflix.discovery.DiscoveryClient$$EnhancerBySpringCGLIB$$caefaf57.getApplications(<generated>) ~[eureka-client-1.1.147.jar!/:1.1.147]
at org.springframework.cloud.netflix.eureka.EurekaDiscoveryClient.getServices(EurekaDiscoveryClient.java:133) ~[spring-cloud-netflix-core-1.0.7.RELEASE.jar!/:1.0.7.RELEASE]
at org.springframework.cloud.client.discovery.health.DiscoveryClientHealthIndicator.health(DiscoveryClientHealthIndicator.java:61) ~[spring-cloud-commons-1.0.5.RELEASE.jar!/:1.0.5.RELEASE]
at org.springframework.cloud.client.discovery.health.DiscoveryCompositeHealthIndicator$Holder.health(DiscoveryCompositeHealthIndicator.java:52) [spring-cloud-commons-1.0.5.RELEASE.jar!/:1.0.5.RELEASE]
at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.actuate.endpoint.HealthEndpoint.invoke(HealthEndpoint.java:81) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.getHealth(HealthMvcEndpoint.java:162) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(HealthMvcEndpoint.java:136) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:817) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:731) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:968) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:859) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:844) [spring-webmvc-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:237) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:112) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) [spring-boot-actuator-1.3.3.RELEASE.jar!/:1.3.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.5.RELEASE.jar!/:4.2.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.0.32.jar!/:8.0.32]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]
We're using Eureka in the app, so it complains too, but basically, this is the same issue as above.
Comment From: spring-projects-issues
Juergen Hoeller commented
As of 4.3 RC2, ScheduledAnnotationBeanPostProcessor features a revised registration mechanism capable of tracking individual bean instances, both on creation for beans of any scope and on destruction of each individual bean. This affects the implicit shutdown order since it is is bound to each bean now, not to ScheduledAnnotationBeanPostProcessor's own lifecycle.
So please revisit this against 4.3 RC2 and see whether there are any further steps to take.
Comment From: spring-projects-issues
Eduard Wirch commented
Spring 5.0.9.RELEASE:
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'myController': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:208)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.web.method.HandlerMethod.createWithResolvedBean(HandlerMethod.java:313)
at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:325)
at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:62)
at org.springframework.web.servlet.handler.AbstractHandlerMapping.getHandler(AbstractHandlerMapping.java:350)
at org.springframework.web.servlet.DispatcherServlet.getHandler(DispatcherServlet.java:1188)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:964)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.base/java.lang.Thread.run(Thread.java:844)
Comment From: OleksiiBondar
Is there any update on this?
Comment From: limhs25
Not Yet? or Newer? Any body here?
Comment From: Sineaggi
We're seeing this too. Any update?
Comment From: rustyx
Same here with Spring Boot 2.2.4.RELEASE (Spring 5.2.3.RELEASE).
From the looks of it, if a new web request comes in at the exact moment the server is shutting down, the dispatcher tries to re-create the already destroyed controller.
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'mainController': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:208)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.web.method.HandlerMethod.createWithResolvedBean(HandlerMethod.java:336)
at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.getHandlerInternal(AbstractHandlerMethodMapping.java:368)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.getHandlerInternal(RequestMappingHandlerMapping.java:449)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.getHandlerInternal(RequestMappingHandlerMapping.java:67)
at org.springframework.web.servlet.handler.AbstractHandlerMapping.getHandler(AbstractHandlerMapping.java:395)
at org.springframework.web.servlet.DispatcherServlet.getHandler(DispatcherServlet.java:1234)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1016)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:399)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Comment From: acsipak
Any suggested workarounds?
Comment From: stefanocke
As far as I see, there are at least two issues here in the comments:
- BeanCreationNotAllowedException in scheduled tasks
- BeanCreationNotAllowedException in web requests
Regarding scheduled tasks:
I also experienced BeanCreationNotAllowedExceptions but sometimes also "EntityManager is closed", if the task did some JPA work. When I looked deeper into this, I recognized the main cause was, that my task was not interruptible.
So, the following happened: 1. TaskScheduler is stopped and tries to interrupt its scheduled tasks 2. Tasks continue to run, since they are not interruptible. 3. Spring shuts down other beans, like EntityManagerFactory 4. Task try to get beans / resources, that no longer exist -> Exceptions are thrown.
By "not interruptible" I mean, they are not running any oprations mentionend in java.lang.Thread.interrupt(). This is for instance even true if the task does lots of DB work and uses Postgres JDBC driver, since it is pre-NIO. (This explanation is probably not perfect, but I am quite sure my long running DB tasks were definitely not interruptible...)
Thus, a possible solution for scheduled tasks is to make them interruptible b checking for Thread.current().isInterrupted() frequently. See for instance https://www.ibm.com/developerworks/java/library/j-jtp05236/index.html?ca=drs-#2.2 (Section: "Implementing cancelable tasks")
Additionally, you should set spring.task.execution.shutdown.await-termination-period and spring.task.execution.shutdown.await-termination-period. So you compensate for the milliseconds or seconds you scheduled task needs until it checks for Thread.current().isInterrupted() the next time.
Update 2021-02-03 While this appraoach helps me in most cases, it is unfortunately not complete: If my tasks publish Application-Events, I get a similar Exception as mentioned by Thorsten Vogel. Please see my next comment for further analysis.
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'refreshEventListener': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:220)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:207)
at org.springframework.context.event.AbstractApplicationEventMulticaster.retrieveApplicationListeners(AbstractApplicationEventMulticaster.java:247)
at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:204)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:134)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:404)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:374)
Regarding web requests:
I have not too much experience with this case but hope that https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.3-Release-Notes#graceful-shutdown would help.
- Update 2021-02-05: I did some tests with graceful shutdown and think it works as expected. Especially, the grace period happens before any bean is destroyed. So as long as your request is not longer than grace period (default: 30s), everything should be fine.
Comment From: stefanocke
Some more debugging... I looked at the disposable bean names and their order by debugging into destroySingletons() .
For better overview I only show some relevant beans here:
[... dataSource,
..., entityManagerFactory,
..., taskScheduler,
... , applicationTaskExecutor,
..., refreshEventListener,
..., springApplicationAdminRegistrar, ... ]
The destruction oder is from last to first element. As you can see, dataSource and entityManagerFactory are already in good positions. If my tasks depend on them, I can rely on them being destroyed after taskScheduler and applicationTaskExecutor.
However, I got my exception for the bean "refreshEventListener" and also for "springApplicationAdminRegistrar" (which I do not use directly). As the stacktrace in my previous comment shows, they are just requested because the EventMulticaster asks for all potential EventListener beans. (AFAIU. Excuse me, if incorrect...) So, my tasks have a very "indirect" dependency on those beans, just because I publish events. I tend to belive it is not possible to completely detect all those kinds of dependencies.
How could Spring or Spring Boot help with this?
- Maybe, the original title of this issue from 2012 is still valid (as the only possible solution) : "Shut down task executor beans first during factory destruction" . But I guess, the one who creates the TaskScheduler should do this then. So, it would be a Spring Boot issue.
- When looking for other places with Schedulers, I have seen that in
org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer, the consumer threads are stopped withinstop()method, which IMHO ist called bySmartLifecycle.stop()which is IMHO before destroy. When applying the same forScheduledAnnotationBeanPostProcessorit would mean to stop the scheduled tasks also already inSmartLifecycle.stop()or similar. Same forAsyncAnnotationBeanPostProcessorwhich currently stops nothing at all.
Workaround
Besides the already mentioned
- await_termination_period and
- interruptible tasks,
my workaround for this part of the issue is to shutdown the TaskExecuter and TaskScheduler early by myself. I guess, this is not completely surprising and there are similar solutions out there on SO:
@Bean
public ApplicationListener<ContextClosedEvent> earlyTaskExecutorShutdown(ThreadPoolTaskExecutor applicationTaskExecutor) {
return new ApplicationListener<ContextClosedEvent>() {
@Override
public void onApplicationEvent(ContextClosedEvent event) {
applicationTaskExecutor.shutdown();
}
};
}
@Bean
public ApplicationListener<ContextClosedEvent> earlyTaskSchedulerShutdown(ThreadPoolTaskScheduler taskScheduler) {
return new ApplicationListener<ContextClosedEvent>() {
@Override
public void onApplicationEvent(ContextClosedEvent event) {
taskScheduler.shutdown();
}
};
}
Comment From: jhoeller
Superseded by #27090 now, being released today in 6.1 M2.
We're effectively doing what's shown above, triggering an early shutdown on ContextClosedEvent for both ThreadPoolTaskExecutor and ThreadPoolTaskScheduler, as well as for ScheduledAnnotationBeanPostProcessor.