(What follows is how I created my application from scratch, and how you can reproduce it, in order to get the behaviour I'm questioning. The question itself will be at the end. I felt the question would be easier to make this way.)

I've created an empty WebMVC application with the following files:

MainApplication.java
-----------------------------------------------------------
@SpringBootApplication
public class MainApplication {
    public static void main(String[] args) {
        SpringApplication.run(MainApplication.class, args);
    }
}

application.yaml
----------------
server:
  port: 8080

Then I ran the following request (note that the HTTP verb is OPTIONS):

curl -i -X OPTIONS http://localhost:8080/path

and got 404, as expected, since the application has no endpoints.

HTTP/1.1 404
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Content-Type: application/json
Transfer-Encoding: chunked
Date: Wed, 28 Oct 2020 09:37:28 GMT

{"timestamp":"2020-10-28T09:37:28.433+00:00","status":404,"error":"Not Found","message":"","path":"/path"}

Since I don't want any body to be returned for 404s I changed the application:

MainApplication.java
-----------------------------------------------------------------
@SpringBootApplication(exclude = ErrorMvcAutoConfiguration.class)
public class MainApplication {
    public static void main(String[] args) {
        SpringApplication.run(MainApplication.class, args);
    }
}

application.yaml
---------------------------------------------
server:
  port: 8080

spring:
  mvc:
    throw-exception-if-no-handler-found: true
  resources:
    add-mappings: false

I disabled ErrorMvcAutoConfiguration (visible effect: body is now in XML) and resources.add-mappings (visible effect: removes Vary headers) since the application will have its own custom global error handler. I've also enabled mvc.throw-exception-if-no-handler-found (no visible effect yet).

The request from before now returns:

HTTP/1.1 404
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 431
Date: Wed, 28 Oct 2020 09:47:08 GMT

<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1></body></html>

I still get a body because I'm not catching the NoHandlerFoundException and processing it myself, so it goes via the default way.

Now I add a global error handler with ResponseEntityExceptionHandler and @RestControllerAdvice and handle NoHandlerFoundException so I don't return any body for 404s (the class is very bare-bones since I'm only focusing on custom output for 404s in this example):

ApplicationResponseEntityExceptionHandler.java
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
@RestControllerAdvice
public class ApplicationResponseEntityExceptionHandler extends ResponseEntityExceptionHandler {
    @Override
    protected ResponseEntity<Object> handleNoHandlerFoundException(NoHandlerFoundException exception, HttpHeaders headers, HttpStatus status, WebRequest request) {
        return new ResponseEntity<>(status);
    }
}

The request from before now returns:

HTTP/1.1 404 
Allow: GET, HEAD, POST, PUT, DELETE, OPTIONS, PATCH
Content-Length: 0
Date: Wed, 28 Oct 2020 10:10:54 GMT

I now have 404s without bodies via a custom global error handler. However, where did the Allow header come from? It wasn't here before. Why are we saying that an endpoint that doesn't exist accepts all HTTP verbs? Is this as intended?

If I call OPTIONS on an endpoint that exists I will get the right content for Allow but when the endpoint doesn't exist I always get all verbs. This only happens when I'm using a custom global error handler as in the example above.

Comment From: wilkinsona

This is standard behaviour for an HttpServlet where it determines the methods that are allowed based on the methods that the HttpServlet sub-class has overridden. Spring Framework makes a small addition to include PATCH and only set the Allow header if not already set.

When you remove you custom global error handler, it's Tomcat that serves the response, sending an error page back to the client. As the response is not coming from an HttpServlet, the Allow header is not set.

If, given the explanation above, you believe that there's further room for improvement here, please open a Spring Framework enhancement request.