Andy Wilkinson opened SPR-16977 and commented
It appears the classpath scanning doesn't work when Surefire launches the JVM configured to use the module path. target/classes is placed on the module path and target/test-classes is patched into this module but classpath scanning only finds classes in target/test-classes.
I have attached a minimal sample that should reproduce the problem when built (mvn test) with Java 10. The sysout from the test should show that only the test class has been found:
[INFO] Running com.example.ScanningTest
[file [/Users/awilkinson/dev/temp/module-path-scanning/target/test-classes/com/example/ScanningTest.class]]
To my rather untrained eye, building with -X and examining the arguments that Surefire uses to launch the forked JVM (in target/surefire) suggests that Surefire's configuration of the JVM is correct.
When the sample is modified to work with Java 8 (remove module-info.java and change the compiler plugin configuration to remove <release>10</release>) the class in target/classes is also found:
[INFO] Running com.example.ScanningTest
[file [/Users/awilkinson/dev/temp/module-path-scanning/target/test-classes/com/example/ScanningTest.class], file [/Users/awilkinson/dev/temp/module-path-scanning/target/classes/com/example/One.class]]
Affects: 5.0.7
Reference URL: https://github.com/spring-projects/spring-boot/issues/13581
Attachments: - module-path-scanning.zip (3.66 kB)
Issue Links: - #20937 Compatibility with JDK 11
2 votes, 7 watchers
Comment From: spring-projects-issues
Juergen Hoeller commented
This all depends on ClassLoader.getResources results for the given base package. The module system possibly only exposes the root URL for the patched part there? In any case, debugging ClassLoader.getResources results in both scenarios would help here...
Comment From: spring-projects-issues
Andy Wilkinson commented
On closer inspection, this appears to be a bug in the JDK. With the test changed to the following:
@Test
public void scanningTest() throws Exception {
Enumeration<URL> resourceUrls = getClass().getClassLoader().getResources("com/example");
while (resourceUrls.hasMoreElements()) {
System.out.println(resourceUrls.nextElement());
}
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
System.out.println(Arrays.toString(resolver.getResources("classpath*:com/example/**/*.class")));
}
It produces output similar to this:
[INFO] Running com.example.ScanningTest
file:/Users/awilkinson/dev/temp/module-path-scanning/target/test-classes/com/example/
file:/Users/awilkinson/dev/temp/module-path-scanning/target/test-classes/com/example
[file [/Users/awilkinson/dev/temp/module-path-scanning/target/test-classes/com/example/ScanningTest.class]]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 s - in com.example.ScanningTest
Note the two (slightly different) URLs that are both pointing to target/test-classes. This explains why no classes in target/classes are found.
Comment From: spring-projects-issues
Juergen Hoeller commented
Looks like a bug in the ClassLoader implementation indeed. They might not have fully tested this in module patch scenarios...
Probably worth reporting to OpenJDK. Aside from a proper fix, maybe there's a workaround to discover in time for our GA still.
Comment From: spring-projects-issues
Andy Wilkinson commented
I've reported a bug. It's in internal review at the moment (ID 9055803). I'll comment again here if it makes it out the other end and becomes public. For reference, this is the test case that I provided with the bug report. Unlike the sample attached to this issue, it does not use Spring Framework.
Comment From: spring-projects-issues
Andy Wilkinson commented
The bug is now public but has been closed as not an issue. The situation's now being discussed in this thread on the core libs dev OpenJDK mailing list.
Comment From: spring-projects-issues
Andy Wilkinson commented
The thread on the mailing list seems to have reached a conclusion. The following recommendation came from Alan Bateman:
With modules then it should be looking at the modules in the boot layer and using ModuleReader to get the contents. It can use the value of java.class.path to scan the class path.
Comment From: spring-projects-issues
Thomas Kratz commented
Are there any plans to resolve this like recommended?
Would doFindAllClassPathResources(String path) be the point to put that? I don't clearly understand that suggestion, but maybe I could invest some time over the holidays.
I would still think this is a JDK issue.
Comment From: wimdeblauwe
Is there a workaround for this? I have a JavaFx project that I added Spring Boot to which works really great, but I am unable to run my @JsonTest.
As a workaround, I need to manually specify the application class + the json deserializers I want to test:
@JsonTest
class MyJsonDeserializerTest {
...
}
needs to become:
@JsonTest
@ContextConfiguration(classes = {MyApplication.class, MyJsonDeserializerTest.TestConfig.class})
class MyJsonDeserializerTest {
...
@TestConfiguration
static class TestConfig {
@Bean
public MyJsonDeserializer deserializer() {
return new MyJsonDeserializer();
}
}
}
(Note: It seems I only need to do this for Maven, IntelliJ seems to run the test fine, but I have to assume that IntelliJ is somehow "cheating" given this issue and the related https://github.com/spring-projects/spring-boot/issues/13581 )
Comment From: eiswind
Another year has passed. To me this still is a showstopper.
Comment From: tgolden-andplus
I have the opposite problem currently. Failsafe works just fine, apparently because they point to the packaged JAR instead of the exploded target/classes directory. However, IntelliJ wants to use the exploded directory when running integration tests in the IDE, so this fails unless I uncheck the module-path option (which I have to do for every run configuration separately).
Additionally, I can make Failsafe break the same way as the IDE by forcing it to use the exploded directory (via <classesDirectory>${project.build.outputDirectory}</classesDirectory>), so there's something magic about being in a JAR versus hanging around on the filesystem -- but I'm not clear on what.
Is this truly a JDK issue or a Spring issue? It doesn't seem that the core JDK team plans on changing behavior, thus it seems incumbent on the Spring team to work around this somehow (or at the very least, document that component scanning will break unexpectedly when using exploded classes versus packaged in JAR classes....)
Comment From: tomdw
@jhoeller @wilkinsona could this be fixed inside spring by making the component scanning code be more module path aware? Especially for running spring tests on module path using the surefire plugin or in intellij this is a problem.
Comment From: PeterMylemans
@tgolden-andplus I'm having the same issue.
For future reference; the only work around I found was to uncheck "use module path" in Templates > JUnit in your Run/Debug Generations to make sure it is unchecked for every new junit run config you create.
Comment From: manosbatsis
For me the workaround was using a org.springframework.core.io.ClassPathResource instead of ClassLoader.getResource
Comment From: tomdw
@manosbatsis and how dit you make sure that spring uses this ClassPathResource for its component scanning?
@tgolden-andplus problem with that workaround is that you then no longer have the module checking which is important. Also stops working when using e.g. serviceloader to load services from other modules in your code under test
Probably the only fix is a fix in the code of spring itself, solving the way it does component scanning to handle modules better.
Comment From: manosbatsis
@tomdw to be clear, i had trouble when trying to "manually" load a classpath resource from within tests using ClassLoader.getResource. After a search landed me here, i figured out a workaround was needed to bypass the bug and ClassPathResource did the trick for me, so i thought it might be useful.
Comment From: retheesh-mr
I am facing similar issue when writing junit5 cases with spring boot and java 11. As the jdk bug raised is in resolved status, what is further action on this? How can this be resolved? It seems module-info cannot be used together with junit and spring due to this issue. As it is blocking for me in my project I have no other way than to fall back to java 8 version.
Comment From: kopper
I had the same (or very similar) issue running tests with maven-failsafe-plugin 3.0.0-M5 on AdoptOpen JDK 11. The work around is to disable using module path in plugin's configuration
<useModulePath>false</useModulePath>
Comment From: oscarhaglund
I should mention that I am also suffering from this issue and it would really be appreciated if this could be solved by Spring if it is not going to be solved by Java itself.
Comment From: prakash0409
We upgraded to a new version of spring boot 2.3.4 and all our integration tests started failing. All our integration test cases were earlier working on AdoptOpen JDK 11 with the older version. I guess this should be an issue at Spring end. As mentioned above the classpath scanning is only finding only "target/test-classes".
I was able to use the workaround as suggested by @kopper
Comment From: msche
FYI, I had similar classpath issues when I introduced java modules in my project and executed the Spring Boot unit tests.
07:28:35.820 [main] ERROR - o.s.test.context.TestContextManager - prepareTestInstance - Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener@63e5b8aa] to prepare test instance [...]
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name '...': Unsatisfied dependency expressed through field '...'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type '...' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
I eventually got the test executed successfully by including within the configuration of SureFire version 2.22.x <forkCount>0</forkCount>.
Comment From: barbarosalp
Is there any proper solution, we are also stuck with this?
Comment From: OleksandrGavryliukTR
FYI, I had similar classpath issues when I introduced java modules in my project and executed the Spring Boot unit tests.
07:28:35.820 [main] ERROR - o.s.test.context.TestContextManager - prepareTestInstance - Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener@63e5b8aa] to prepare test instance [...] org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name '...': Unsatisfied dependency expressed through field '...'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type '...' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}I eventually got the test executed successfully by including within the configuration of SureFire version 2.22.x
<forkCount>0</forkCount>.
This has fixed it for me. With a warning:
[WARNING] useSystemClassloader setting has no effect when not forking
Which must be removing the cause of this problem.
Comment From: sbrannen
Disclaimer: This is in no way production ready code. This is 100% "proof of concept" code.
Now, having said that... with an updated version of the original example application from @wilkinsona (now using Java 17, Maven 3.8.5, Maven Surefire 3.0.0-M6, Spring Framework 5.3.20, and JUnit Jupiter 5.8.2), I have come up with the following.
class ScanningTest {
@Test
void scanningTest() throws Exception {
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
System.err.println(Arrays.toString(resolver.getResources("classpath*:com/example/**/*.class")));
System.err.println("------------------------------------------------");
String moduleName = getClass().getModule().getName();
List<String> resourceNames = streamResolvedModules(Predicate.isEqual(moduleName))//
.map(ResolvedModule::reference)//
.map(moduleReference -> scanForNames(moduleReference, "^com/example/.+\\.class$"))//
.flatMap(List::stream)//
.toList();
resourceNames.forEach(System.err::println);
System.err.println("------------------------------------------------");
resourceNames.forEach(resouceName -> {
String className = ClassUtils.convertResourcePathToClassName(resouceName);
className = className.substring(0, className.length() - ".class".length());
try {
Class<?> clazz = ClassUtils.forName(className, getClass().getClassLoader());
System.err.println(clazz);
}
catch (Exception ex) {
throw new RuntimeException("Failed to load class %s".formatted(className), ex);
// ex.printStackTrace(System.err);
}
});
System.err.println("------------------------------------------------");
List<URI> resources = streamResolvedModules(Predicate.isEqual(moduleName))//
.map(ResolvedModule::reference)//
.map(moduleReference -> scanForResources(moduleReference, "^com/example/.+\\.class$"))//
.flatMap(List::stream)//
.toList();
resources.forEach(resource -> {
try {
System.err.println(resource.toURL());
}
catch (MalformedURLException ex) {
ex.printStackTrace();
}
});
}
private Stream<ResolvedModule> streamResolvedModules(Predicate<String> moduleNamePredicate) {
ModuleLayer layer = getClass().getModule().getLayer();
if (layer == null) {
layer = ModuleLayer.boot();
}
return layer.configuration().modules().stream()//
.filter(module -> moduleNamePredicate.test(module.name()));
}
private List<String> scanForNames(ModuleReference reference, String regex) {
try (ModuleReader reader = reference.open()) {
try (Stream<String> names = reader.list()) {
return names.filter(name -> name.matches(regex)).toList();
}
}
catch (IOException ex) {
throw new UncheckedIOException("Failed to read contents of " + reference, ex);
}
}
private List<URI> scanForResources(ModuleReference reference, String regex) {
try (ModuleReader reader = reference.open()) {
try (Stream<String> names = reader.list()) {
return names.filter(name -> name.matches(regex))//
.map(name -> {
try {
return reader.find(name);
}
catch (IOException ex) {
ex.printStackTrace();
}
return Optional.<URI> empty();
})//
.filter(Optional::isPresent)//
.map(Optional::get).toList();
}
}
catch (IOException ex) {
throw new UncheckedIOException("Failed to read contents of " + reference, ex);
}
}
}
The output of running that is:
[file [/Users/sbrannen/source/spring-issues/module-path-scanning/target/test-classes/com/example/ScanningTest.class]]
------------------------------------------------
com/example/One.class
com/example/ScanningTest.class
------------------------------------------------
class com.example.One
class com.example.ScanningTest
------------------------------------------------
file:/Users/sbrannen/source/spring-issues/module-path-scanning/target/classes/com/example/One.class
file:/Users/sbrannen/source/spring-issues/module-path-scanning/target/test-classes/com/example/ScanningTest.class
The first output demonstrates the issue: PathMatchingResourcePatternResolver only finds ScanningTest.class and not One.class.
The final output demonstrates that we can find resources (as instances of URI) using the Module APIs and successfully find both ScanningTest.class and One.class.
Kudos to @sormuras for providing inspirational use of the java.lang.module APIs in JUnit 5!
Where to go from here?
We may introduce a new ModulePathResource as a companion to ClassPathResource, where the logic in ModulePathResource is similar to the logic in the above proof of concept.
We may then introduce additional support in PathMatchingResourcePatternResolver that resolves ModulePathResource instances when appropriate.
Comment From: sbrannen
Current work on this issue can be viewed in the following feature branch.
https://github.com/sbrannen/spring-framework/commits/module-path-scanning
Comment From: sbrannen
Hi everybody,
Thanks for all of the feedback over the years!
I am now closing this issue since it has been superseded by the following issues.
-
28506
-
28507
Please follow those issues for additional updates on module path scanning and resource support.
Comment From: sbrannen
If you've been following this issue, I'm happy to let you know that #28506 has been resolved for inclusion in 6.0 M5.
In the interim, feel free to try out 6.0 snapshots with modular testing using Maven.
I've created a new repository for demonstrating the use of the Spring Framework with the Java Module System: https://github.com/sbrannen/spring-module-system
That repository currently contains a maven-surefire-patched-module project which demonstrates support for @ComponentScan in a patched module using Maven Surefire.