Maël OUAHI opened SPR-15989 and commented
PathMatchingResourcePatternResolver provides duplicate resources when using "classpath*" prefix combined with Ant-style patterns.
I'll attach an application that reproduces the problem.The application uses the location pattern "classpath:/-perm.json" to resolve Resources. If you package and run it
mvn clean package && java -jar target/resources-0.0.1-SNAPSHOT.jar
You should see the following output:
file:/D:/resources/target/resources-0.0.1-SNAPSHOT.jar!/
file:D:\resources\target\resources-0.0.1-SNAPSHOT.jar!/
file:/C:/Program%20Files/Java/jdk1.8.0_40/jre/lib/ext/dnsns.jar!/
...
file:/D:/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
file:D:\resources\target\resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
Affects: 4.3.8, 4.3.9, 4.3.10, 4.3.11
Attachments: - resources.zip (1.04 MB)
Issue Links: - #18260 PathMatchingResourcePatternResolver does not consider manifest based classpaths - #20665 PathMatchingResourcePatternResolver returns duplicate resources when using classpath* prefix
Referenced from: commits https://github.com/spring-projects/spring-framework/commit/9d8e3d4185b9f1158de31c4f9a53f2a036a041df, https://github.com/spring-projects/spring-framework/commit/a839477d4a4e2b52464381c69ba198a7c703fe9f
Backported to: 4.3.12
Comment From: spring-projects-issues
Juergen Hoeller commented
Running the attached app, all I can see are regular entries along the lines of D:\resources\target in your output - no duplicates here, I'm afraid. There have to be some specific conditions involved in your setup, so please try to debug PathMatchingResourcePatternResolver accordingly: Somewhere in doFindAllClassPathResources, possibly in addAllClassLoaderJarRoots / addClassPathManifestEntries, the same jar might be seen twice but with different URL format.
I'm pretty sure this only happens when searching for such patterns in the root of your jar files... since the root of such files is unfortunately not easily discoverable with a standard ClassLoader (since ClassLoader.getResources("") is not guaranteed to deliver a complete result). As a consequence, it is usually recommendable to put such files into a specific subdirectory (even META-INF) in each jar file which can be more reliably searched through a standard call (e.g. ClassLoader.getResources("META-INF")).
Comment From: spring-projects-issues
Christophe Blanchard commented
I do reproduce the problem as well :
"C:\Program Files\Java\jdk1.8\bin\java.exe" -jar target/resources-0.0.1-SNAPSHOT.jar
file:/D:/tmp/resources/resources/target/resources-0.0.1-SNAPSHOT.jar!/ file:D:\tmp\resources\resources\target\resources-0.0.1-SNAPSHOT.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/access-bridge-64.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/cldrdata.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/dnsns.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/jaccess.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/jfxrt.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/localedata.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/nashorn.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunec.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunjce_provider.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunmscapi.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/sunpkcs11.jar!/ file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/zipfs.jar!/ file:/D:/tmp/resources/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json file:D:\tmp\resources\resources\target\resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
The problem does not appear when running with spring-boot-maven-plugin instead of maven-shade-plugin. weird...
Comment From: spring-projects-issues
Maël OUAHI commented
thanks for your reply, i confirm that the problem does not appear when i put the file into subdirectory, but i reproduce the problem when i run the app on windows 7 OS and centOs 7.
Comment From: spring-projects-issues
Juergen Hoeller commented
I'm afraid I still can't reproduce the problem on Windows 10 here... but it indeed seems to be related to the maven-shade-plugin.
Any chance you could attach a debugger to this? Which Resource type are those duplicated entries? Which step in PathMatchingResourcePatternResolver creates them (I suppose addAllClassLoaderJarRoots / addClassPathManifestEntries)?
Comment From: spring-projects-issues
Maël OUAHI commented
I reproduce the problem when using maven-assembly-plugin to create the fat jar instead of maven-shade-plugin.
Comment From: spring-projects-issues
Juergen Hoeller commented
I can reproduce the problem as well now: on JDK 8 but not on JDK 9 (which I was accidentally always running on before - sorry). On JDK 9, the system class loader never extends URLClassLoader but on JDK 8, it apparently may do so... even in combination with the "java.class.path" system property (which we're not taking into account in our code).
I'll try to sanitize the paths in our code accordingly before adding them to the result set, properly identifying the same jar there and only including it once.
Comment From: spring-projects-issues
Juergen Hoeller commented
We're explicitly checking for duplicates in addClassPathManifestEntries now, with or without extra leading slash in the existing result (typically from URLClassLoader.getURLs()).
This is in master now for the 5.0 GA release. I'll also backport it to 4.3.12 later today. It'd be great if you could give 4.3.12.BUILD-SNAPSHOT a try once it's in; I'll let you know.
Comment From: spring-projects-issues
Maël OUAHI commented
thanks for you responsiveness, let me know when backporting was done and i will try it.
Comment From: spring-projects-issues
Maël OUAHI commented
It's OK. I tried it and this is the output
file:/D:/dev/workspaces/oxygen/hyperia/resources/target/resources-0.0.1-SNAPSHOT.jar!/
file:/C:/Program%20Files/Java/jdk1.8/jre/lib/ext/access-bridge-64.jar!/
...
file:/D:/dev/workspaces/oxygen/hyperia/resources/target/resources-0.0.1-SNAPSHOT.jar!/domaine-perm.json
Comment From: spring-projects-issues
Udayakumar commented
I am facing exactly the same issue even with 5.0.1.RELEASE.
The pattern i am using is - classpath*:/**/*.rdrl to fetch all the .rdrl files inside jars in the class path. Basically we have multiple jars in the class path containing .rdrl file. Each file is returned two times one with / and another with \ in the path.
Comment From: spring-projects-issues
Juergen Hoeller commented
Udayakumar, could you please create a new JIRA issue for your scenario? Please dump the full URLs that you're seeing, like above... Plain slash differences shouldn't matter since UrlResource.equals considers them equal, so there's probably something else involved. In any case, let's try to fix this for 5.0.2 / 4.3.13.
Comment From: spring-projects-issues
Udayakumar commented
Thanks. opened a new issue.