Affects: Spring Framework 5.3.10
Prior discussion: https://gitter.im/spring-projects/spring-boot?at=615ffdb17db1e3753e2ba6f9 Repository showcasing the issue: https://github.com/schosin/ResourceUrlEncodingFilterPerformance
When using ResourceUrlEncodingFilter with its default configuration, invocations of HttpServletResponse#encodeURL will be wrapped by ResourceUrlEncodingResponseWrapper, which resolves URLs according to a ResourceUrlProvider.
With WebJarAssetLocator present, this will catch all URLs (/**) and test against five different locations whether the URL passed is a static resource:
[class path resource [META-INF/resources/], class path resource [resources/], class path resource [static/], class path resource [public/], ServletContext resource [/]]
For class path resources, this is done by resolving the path against the resource folder and checking calling Resource#isReadable, which ultimately throws a FileNotFoundException if the resource is not present. See PathResourceResolver#getResource(String, HttpServletRequest, List<? extends Resource>).
Since simple URLs to any handler mappings cause these checks, pages with many links will have poor performance, especialy on machines with weaker CPUs.
A workaround is to disable Spring's default resource handling (spring.web.resources.add-mappings) and registering custom resource handlers mapping handlers directly (e.g. "/css/** -> classpath:/static/css/").
Comment From: rstoyanchev
This is sub-optimal indeed, but by default Spring Boot maps static resources on "/**", which means there is no way to rule out a given link as a non-static resource. All we can do is to pass it through the ResourceResolver chain which tries to match it to a resources under the configured locations.
This isn't too much of an issue when handling incoming requests since controllers are checked before static resources, but for ResourceUrlEncodingFilter it means it has to try to resolve every link. Ideally it should be used with static resource URLs that have some prefix that differentiates them. So one possible action would be to document this in Boot and recommend to switch to prefix based static resource URL paths.
Perhaps, PathResourceResolver could check if all of the locations given to it exist and if not remove them from the list. That could significantly reduce the checks, if you don't have static/, public/, or resources/. Not sure what else we could do on the Spring Framework side.
Comment From: schosin
Would it be an option to reimplement AbstractFileResolvingResource#exists and AbstractFileResolvingResource#isReadable based on AbstractResource#resolveURL?
That would have to be done in sub classes of AbstractFileResolvingResource as not to break compatibility with existing user code, but would help with ClassPathResource, ServletContextResource and UrlResource.
Comment From: jhoeller
@schosin It looks like this is related to #27541 - which I currently have in the works for 5.3.11 - and originally #21372 which led to a reimplementation of AbstractFileResolvingResource.isReadable() in 5.1, implying an existence check in isReadable() itself.
While ServletContextResource overrides both exists() and isReadable() for a more efficient check, ClassPathResource only overrides exists() - whereas as of 5.1, PathResourceResolver only calls isReadable(). It's worth noting that PathResourceResolver used to call resource.exists() && resource.isReadable() but as of 5.1, that's not semantically necessary anymore. Unfortunately it misses out on the efficient existence check that way.
As far as I see right now, overriding isReadable() in ClassPathResource to start with a resolveURL() check (just like its exists() implementation) would address the efficiency problem that you're pointing out, bringing it to the same performance level that ServletContextResource is at. I'm going to try that along with my AbstractFileResolvingResource revision in #27541, making it available in a 5.3.11 snapshot which you could give an early try later today.
Comment From: rstoyanchev
I've made a change to filter out non-existing locations. In the given scenario, with default Boot settings and use of WebJars, it filters out 4 and leaves 1 location (ServletContextResource '/'). Using the sample with 10K and 100K links, I see a significant reduction of processing time, by more than a third. That should complement the other change to improve isReadable.
Comment From: jhoeller
This is available in the latest 5.3.11 snapshot now and will be available in the upcoming 5.2.18 snapshot as well. Feel free to give it an early try...
Comment From: silver-mx
@rstoyanchev and @jhoeller we recently upgraded to spring-boot 2.5.6 which I think includes these changes, and static content (included in a JAR file inside the executable spring-boot JAR) is not found anymore. This static content is an angular application and in spring-boot 2.5.5 it is found and works OK but not in 2.5.6.
Could it be that the behavior we see is caused by these changes? Is it required to update any application property or do something extra to allow spring to find static content inside JAR files (included in the spring-boot executable JAR)?
Comment From: bclozel
@silver-mx it could be, but it's hard to tell from your comment. Could you share a sample application (something we can download or git clone) so that we can reproduce this problem?
Comment From: silver-mx
@bclozel I cannot share the application as is but I will try to come up with a simpler version that hopefully reproduces the issue. Thanks!
Comment From: silver-mx
Unfortunately, I have not been able to reproduce the issue with a simpler application, but I enabled the debug logs in our application and I can see the following log, which lists the paths where resources are found:
When static resources are found (spring-boot 2.5.5)
o.s.w.s.h.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler [Classpath [META-INF/resources/], Classpath [resources/], Classpath [static/], Classpath [public/], ServletContext [/]]
When static resources are NOT found (spring-boot 2.5.6)
o.s.w.s.h.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler [classpath [META-INF/resources/], ServletContext [/]]
NOTE: Our resources are in this case placed in static/ and as you can see it is not on the list for spring-boot 2.5.6.
The logs were obtainer sending a simple request to http://localhost:8080/app. A bit more detail can be seen in the attached file. It is not much, but hopefully, it can give an idea to determine whether this is related to the fix discussed here.
log_spring_static_resources_not_found.txt
Comment From: bclozel
@silver-mx without a project reproducing the error, it's going to be hard to help you.
When your application starts up, does the classpath:static/ location exist? Are those files within a fat jar? Are those resources generated at runtime?
Comment From: silver-mx
@bclozel The resources are placed in a JAR contained in the BOOT-INF/lib folder of the fat jar, so they exist when the application starts, nothing is generated at runtime.
I have not given up trying to create an application that reproduces the issue, but based on the traces I have shared we can see that spring-boot 2.5.6 does not seem to consider Classpath [resources/], Classpath [static/], Classpath [public/] for the existence of static resources any more right? Is this configurable from now on?
Comment From: bclozel
It might be that the location is considered as missing and filtered out. I'll work on a sample if you can confirm the following: can you add an src/main/resources/static/bogus.txt file in your app and confirm that this works with Spring Boot 2.5.6?
Comment From: wilkinsona
I wonder if this could be caused by the jar not containing entries for its directories? For example, if static/resource.txt exists but static/ does not, I think the check for the existence of classpath:static/ would fail and the location will be filtered out.
Comment From: silver-mx
@bclozel and @wilkinsona I can confirm that adding src/main/resources/static/bogus.txt causes our static resources to be found in classpath:static/ (inside the JAR file) in spring-boot 2.5.6.
In summary:
WITHOUT src/main/resources/static/bogus.txt (spring-boot 2.5.6)
o.s.w.s.h.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler [classpath [META-INF/resources/], ServletContext [/]]
WITH src/main/resources/static/bogus.txt (spring-boot 2.5.6)
o.s.w.s.h.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler [classpath [META-INF/resources/], classpath [static/], ServletContext [/]]
So it looks like spring filters out classpath:static/ when src/main/resources/static does not exist. Does that help you to understand the issue?
log_spring_static_resources_not_found.txt
Comment From: bclozel
@silver-mx adding a src/main/resources/static folder in your application was merely a way to better understand the problem. I think @wilkinsona is right - your JAR is probably built in a way that it's missing an entry for the static/ folder inside it.
I've tested this very scenario with an application that depends on a JAR that's built with Gradle and only contains repackaged static resources (generated by webpack). With Spring Boot 2.5.6, static resources are still found and I can't reproduce this issue.
Looking at the jar, I can see an entry for the static/ folder:
$ jar -tvf frontend/build/libs/frontend.jar | grep "static/$"
0 Wed Oct 27 10:13:24 CEST 2021 static/
Could you check that the jar containing static resources is missing that entry? Could you elaborate on how it's being built?
Comment From: silver-mx
@bclozel I see that the JAR has the static folder:
$ jar -tvf frontend.jar
207430 Mon Oct 25 12:05:34 CEST 2021 static/10.f4cf5f1dfc9c0e0c6425.js
153499 Mon Oct 25 12:05:34 CEST 2021 static/4.28289e819892b2ce955a.js
38108 Mon Oct 25 12:05:34 CEST 2021 static/5.135095cc05a9a9480f20.js
24529 Mon Oct 25 12:05:34 CEST 2021 static/6.a8131e0b9b7fd3c92f35.js
1003151 Mon Oct 25 12:05:34 CEST 2021 static/7.5b66e14d22a21098572a.js
38058 Mon Oct 25 12:05:34 CEST 2021 static/8.ef40ae98e6f9ef049bac.js
18329 Mon Oct 25 12:05:34 CEST 2021 static/9.f6ab6c6381925372b929.js
0 Mon Oct 25 12:05:34 CEST 2021 static/app/
0 Mon Oct 25 12:05:34 CEST 2021 static/app/assets/
0 Mon Oct 25 12:05:34 CEST 2021 static/app/assets/config/
Not completely sure how it is being built as it comes as a maven dependency. Do you suspect that the format could be wrong? Created like a ZIP instead of a JAR of something?
Comment From: bclozel
@silver-mx this confirms that the static/ folder entry is missing. I don't know how this jar is created, I'd be interested to know if you can get that information. As far as I understand, this is done by default by Maven and Gradle since Java would rely on class.getResource() to get the folder - which is exactly the case here.
Comment From: silver-mx
@bclozel I am confused, the JAR contains a static folder and inside it are all the static resources (e.g. static/10.f4cf5f1dfc9c0e0c6425.js). How should the structure look like then?
Comment From: bclozel
@silver-mx it's missing an entry for the static/ directory; there's one for static/app/ and child directories but not for the static/ one.
Comment From: silver-mx
@bclozel and @wilkinsona thanks, I was not understanding what you meant as directory entry but google just enlightened me :). Thanks a lot, I will check why the /static entry is missing in our frontend JAR file.
Comment From: sdeleuze
FYI this change broke static resource handling on Spring Native side since even with native configuration only the file resources are available, not the directories. Short term, I will try to craft a substitution to disable this optimization on Spring Native side. I am wondering if we should disable this optimization on native in next Spring Framework release, or if we want to refine this mechanism one way or another. We can maybe raise a bug on GraalVM side if we think this is the right thing to do but it will take several months to be fixed.
Comment From: sdeleuze
New update: this issue only happens with GraalVM 21.2.0 and seems to not be present with GraalVM 21.3.0. So if confirmed, I will just provide a substitution for Spring Native 0.10.x based on GraalVM 21.2.0 and Spring Native 0.11.x based on GraalVM 21.3.0 should be fine.
Comment From: wilkinsona
It's valid for a jar file to be created without entries for its directories. IMO, the optimisation needs to be refined to check for the existence of anything matching static/** rather than just static/. If that would be too costly then I wonder if it would be better to remove it.
Comment From: rstoyanchev
I've created #27624 to follow up on this.
Comment From: jhoeller
See #27624 for an update: This existence check is behind an optimizeLocations flag on ResourceHttpRequestHandler now, potentially to be enabled by default in Boot and other scenarios where the jar layout consistently includes directory entries.