Peter H opened SPR-11125 and commented
When Quartz Scheduler is configured to utilize the CommonJ WorkManager, worker (WorkManager) threads have a different ContextClassLoader (CCL) than their caller. This happens at WebSphere AS at least, not sure about WebLogic.
The reason is that Spring creates the main Quartz thread during scheduler startup (startScheduler method in SchedulerFactoryBean) on its own and that means that this thread is unmanaged. This unmanaged thread is then used to control the all the Quartz activity and also to send jobs to the WorkManager. While CCL is propagated properly from the any managed thread to an unmanaged thread, it is not propagated from any unmanaged thread to a managed thread.
This means that while the unmanaged Quartz main thread will have correct CCL, managed WorkManager threads will not. They will get SystemClassLoader instead.
This should be corrected so that jobs can execute with correct CCL and thus especially (hardly modifiable) 3rd party code that may utilizes this class loader won't fail.
Most likely, the run method in the DelegatingWork class should be modified to explicitly propagate the CCL. Also a cleanup action that resets the CCL back at the end of the run would be essential.
Affects: 3.0.7, 3.2.5
Reference URL: https://www.ibm.com/developerworks/community/forums/html/topic?id=7fe9a868-128c-41c2-9cf6-bcc87d9889ab
2 votes, 4 watchers
Comment From: spring-projects-issues
Peter H commented
Note that the same issue will probably affect also ConcurrentTaskExecutor / ManagedConcurrentTaskExecutor.
Comment From: spring-projects-issues
Juergen Hoeller commented
Revisiting this one, it's actually a good old Quartz problem since it's Quartz internally starting that scheduler thread, with Spring historically not being able to do much about it. Granted, we could explicitly pass the CCL to worker threads, but wouldn't it be preferable for the Quartz scheduler thread to be managed to begin with? I'll double-check Quartz 2.2 whether there's any way to override the scheduler thread now.
Note that these problems should not exist when using Spring's own scheduling support (@Scheduled etc). It's a Quartz-specific issue.
Juergen
Comment From: spring-projects-issues
Peter H commented
This unmanaged threads problem causes also another issues (for example with JNDI lookups) so I took another look at it and there's actually a possibility to run all the Quartz controller threads as managed. What has to be done in addition to providing the taskExecutor instance (in this case the WorkManagerTaskExecutor implementation) to the SchedulerFactoryBean instance:
Give (to Quartz) the ThreadExecutor, in this case via Quartz properties org.quartz.threadExecutor.class=org.quartz.commonj.WorkManagerThreadExecutor and org.quartz.threadExecutor.workManagerName=
After this, it will run even its own controller threads on that work manager and things will be much better (CCL and Java EE context aren't lost) .
There are few culprits however - the current (as of Quartz up to 2.2.1) WorkManagerThreadExecutor implementation needs one little fix in the DelegatingWork.isDaemon() method to return true (true is for long running work, false for short) to avoid hung thread notifications such as here https://jira.terracotta.org/jira/browse/QTZ-433 - so for now you should use your own implementation that extends this one and overrides at least that method. (See also your DelegatingWork implementation) - the configuration is quite cumbersome now, you need to provide taskExecutor to the SchedulerFactoryBean (that's fine) and then the appropriate threadExecutor to the Quartz itself. Can't this be done right away by the SchedulerFactoryBean itself, similarly to setting the LocalDataSourceJobStore as the Quartz job store implementation if the dataSource is provided? - there's currently no implementation of JSR-236 ManagedExecutorService for this, it would be nice to have one at least here (so that we won't have to depend on what Quartz provides as things seem to proceed very slowly over there). And then SchedulerFactoryBean would pick the right one depending on the provided taskExecutor.
Note that I've found also this article http://gybas.com/2014/quartz-websphere-spring/ where the use of fully qualified JNDI name (indirect lookup) such as java:comp/env/wm/default is discouraged and the short one (direct lookup) like wm/default is recommended. Well, I did everything with the full name and didn't have any problem and don't see any good reason why there should be any (if you've got a resource reference for that work manager). You could probably make use of the resourceRef property (JndiLocatorSupport) here.
Note about the Quartz control threads (for those curious about how many of them may exist and how to determine the thread pool sizing): - Each Quartz Scheduler instance creates one main scheduler thread - If JDBC job store is configured, an additional MisfireHandler thread is created - if isClustered is set (in addition to the JDBC job store of course), an additional ClusterManager thread is created
So there could be 1-3 such threads depending on the scheduler configuration.
All the above is possible with Quartz 1.8.6 and 2.1.0 or later: https://jira.terracotta.org/jira/browse/QTZ-113 https://jira.terracotta.org/jira/browse/QTZ-194 https://jira.terracotta.org/jira/browse/QTZ-233
Comment From: spring-projects-issues
Bulk closing outdated, unresolved issues. Please, reopen if still relevant.
Comment From: mag01
Hello, I'm original reporter of the issue and yes, it's definitely still relevant. I don't seem to be able to directly reopen this issue here though.
Comment From: snicoll
@mag01 sorry but with the deprecation of CommonJWorkManager in Spring 5.1, and Quartz itself being dormant, we can't really justify spending more time on this.