Affects: 5.1.6.RELEASE

Hello,

I am working on a web application that serves some files stored in AWS S3, acting as a proxy. S3 provides ETags for these files. I would like to reuse those ETags but ShallowEtagHeaderFilter does not seem to allow it.

When I let the ShallowEtagHeaderFilter to generate the ETag, it consumes too much memory for big files.

When I set the ETag manually in the controller, the If-None-Match header does not seem to be considered. I would need to copy some pieces of code from ShallowEtagHeaderFilter in order to have a full working feature.

So I ended up not using ETags for this endpoint for now: ShallowEtagHeaderFilter.disableContentCaching(request);

I would expect ShallowEtagHeaderFilter to be able to consider the existing ETag instead of regenerating it.

Thanks

Comment From: rstoyanchev

When I set the ETag manually in the controller, the If-None-Match header does not seem to be considered

How are you doing that? Can you show a snippet of controller code?

I would expect ShallowEtagHeaderFilter to be able to consider the existing ETag instead of regenerating it.

ShallowEtagHeaderFilter wraps the response and buffers output content, so it can then calculate a hash based on the buffered content. I'd think the filter should not be used at all for requests where the ETag is already determined.

Comment From: mickaeltr

I was hoping that thanks to the following snippet, the filter could just skip the ETag generation but still handle the If-None-Match behavior for further requests:

response.setHeader(HttpHeaders.ETAG, s3.document.getObjectMetadata().getETag());

Comment From: rstoyanchev

No it doesn't work like that. Please check the documentation on that.

Comment From: mickaeltr

I see. My controller method is special because it is a proxy to an AWS S3 document:

response.setContentType(document.getObjectMetadata().getContentType());
response.setIntHeader(HttpHeaders.CONTENT_LENGTH, (int) document.getObjectMetadata().getContentLength());
IOUtils.copy(document.getObjectContent(), response.getOutputStream());

Tomorrow, I'll see if I can convert it to an InputStreamResource and make the ETag work that way.

Thannos @rstoyanchev

Comment From: mickaeltr

Hello, I was finally able to use ResponseEntity and get 304 Not Modified responses:

return ResponseEntity.ok()
  .contentType(MediaType.parseMediaType(document.getObjectMetadata().getContentType()))
  .eTag(document.getObjectMetadata().getETag())
  .contentLength(document.getObjectMetadata().getContentLength())
  .body(new InputStreamResource(document.getObjectContent()));

However I still need to turn off the ShallowEtagHeaderFilter, otherwise my ETag and Content-Length are ignored and recalculated, which uses a lot of memory for big documents:

ShallowEtagHeaderFilter.disableContentCaching(request);

Should I rename or create a new issue for skipping the ShallowEtagHeaderFilter when an ETag was already set?

Comment From: rstoyanchev

No need, I re-purposed this ticket.

Comment From: mickaeltr

Hi @rstoyanchev,

I upgraded my application to Spring 5.1.9 and unfortunately, I still need to explicitly disableContentCatching if I don't want ETag and Content-Length to be recalculated…

  @GetMapping
  public ResponseEntity<InputStreamResource> downloadDocument(HttpServletRequest request) {
    // ShallowEtagHeaderFilter.disableContentCaching(request);

    S3Object documentFile = …

    return ResponseEntity.ok()
      .contentType(MediaType.parseMediaType(documentFile.getObjectMetadata().getContentType()))
      .eTag(documentFile.getObjectMetadata().getETag())
      .contentLength(documentFile.getObjectMetadata().getContentLength())
      .header(HttpHeaders.CONTENT_DISPOSITION, "inline; filename=\"" + FilenameUtils.getName(documentFile.getKey()) + "\"")
      .cacheControl(CacheControl.maxAge(1, TimeUnit.HOURS).cachePrivate())
      .body(new InputStreamResource(documentFile.getObjectContent()));
  }

I will try to dig more into it…

Comment From: rstoyanchev

If you see the commit reference above, maybe you can put a breakpoint and step through to see what happens.

Comment From: mickaeltr

Hi @rstoyanchev,

I had a look at it. I saw 2 places in https://github.com/spring-projects/spring-framework/commit/1a97a26eb7ed21f5c12f0d89aa67ad518d213e89 were the ShallowEtagHeaderFilter is turned off.

  1. HttpEntityMethodProcessor: does not apply to my request because isResourceNotModified(inputMessage, outputMessage) returns false
  2. ServletInvocableHandlerMethod: does not apply to my request because returnValue is not null (it is a ResponseEntity<InputStreamResource>)

Thanks

Comment From: mickaeltr

Hello,

Despite #23775 bug fix, I am still having an issue with ETag and Content-Length being overwritten by the ShallowETagHeaderFilter (using Spring Boot 2.2.4 / Spring 5.2.3), when not disabled.

This is related to what I already documented here https://github.com/spring-projects/spring-framework/issues/22797#issuecomment-484496140 and there https://github.com/spring-projects/spring-framework/issues/22797#issuecomment-520486211

Please let me know if / how I can help…

Comment From: mickaeltr

I created a project with unit tests that demonstrate the issue I am facing with ShallowEtagHeaderFilter: https://github.com/mickaeltr/Spring-ShallowEtagHeaderFilter-issue/blob/master/src/test/java/com/example/demo/DemoControllerTest.java