I'm trying to test my Controller using MockMvc. The service used by Controller throws RuntimeException if something is wrong. The spring-boot's default exception resolver catches this exception and responds with status 500. But when I use MockMvc in my test, the test method just throws the exception instead of catch status 500. If I add explicit exception resolver using @ExceptionHandler, then test method starts to work.

The simple example. My Controller just throws new RuntimeException() for simplicity.

@RunWith(SpringRunner.class)
@WebMvcTest(MyController.class)
public class ControllerTests {

    @Autowired
    private MockMvc mvc;

    @Test
    public void testInternalServerError() throws Exception {
        mvc.perform(get("/api/data"))
                .andExpect(status().isInternalServerError());
    }
}

Expected result: Test catches status 500 and passes.

Actual result: The exception is thrown and test fails.

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException

Maybe I'm wrong assuming that spring-boot installs some default mvc exception handler. But anyway the real application returns status 500 with properly formatted JSON of error details. So I think the MockMvc should do the same.

I can add the reproducible example if someone confirms that this is really not intended behaviour.

Comment From: xak2000

Also, even if I add explicit @ExceptionHandler like this:

    @ExceptionHandler(RuntimeException.class)
    public void hadnle(HttpServletResponse res) throws IOException {
        res.sendError(404, "bla bla");
    }

Then real response and MockMvc response is also different.

The real response contains JSON with error details, like this:

{
  "timestamp" : "2016-11-06T03:16:02.440+0000",
  "status" : 404,
  "error" : "Not Found",
  "exception" : "java.lang.RuntimeException",
  "message" : "bla bla",
  "path" : "/api/data"
}

But the MockMvc response body is empty:

MockHttpServletResponse:
           Status = 404
    Error message = bla bla
          Headers = {}
     Content type = null
             Body = 
    Forwarded URL = null
   Redirected URL = null
          Cookies = []

So we can't check, for example, the message field.

Comment From: bclozel

This might be related to Spring Framework and it seems that MockMvc will follow redirects and forwards in the 5.0 version (see SPR-14342 and SPR-14755).

Could you share a sample repro project we could take a look at? I'm not sure MockMvc is supposed to dispatch error requests back to the container, since we're not firing up a container for those tests.

Comment From: xak2000

demo-mock-mvc-gh-7321.zip - Try: gradle test and see the report. - Try: gradle bootRun, then curl -i localhost:8080/unhandled and curl -i localhost:8080/handled and see the good JSON responses.

There are 2 tests here. 1. testUnhandledException(). It tests the case when no explicit @ExceptionHandler provided by user code.

This test doesn't work. It throws: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException: Hello, unhandled Exception! while real request to the app (gradle bootRun, then curl -i localhost:8080/unhandled) shows good JSON response. 2. testHandledException(). It test the case when explicit @ExceptionHandler (which just does res.sendError(400, "Explicit exception handler works!");) is provided by user code.

This test also doesn't work. It not throws any exceptions. But checked response body is empty.

java.lang.AssertionError: Response content Expected: not an empty string but: was ""

But real request shows good JSON response:

``` HTTP/1.1 400 Content-Type: application/json;charset=UTF-8 Transfer-Encoding: chunked Date: Thu, 17 Nov 2016 15:26:03 GMT Connection: close

{ "timestamp" : "2016-11-17T15:26:03.043+0000", "status" : 400, "error" : "Bad Request", "exception" : "java.lang.IllegalStateException", "message" : "Explicit exception handler works!", "path" : "/handled" } ```

I don't know the internals. Not sure how spring-boot provides this json response. But if you think it works as expected, then we can't rely on this behaviour in tests? It is inconsistent. :( How can we write tests for error conditions using default spring-boot JSON responses, then?

Comment From: philwebb

/cc @rstoyanchev

Comment From: rstoyanchev

Spring Boot's error handling is based on Servlet container error mappings that result in an ERROR dispatch to an ErrorController. MockMvc however is container-less testing so with no Servlet container the exception simply bubbles up with nothing to stop it.

So MockMvc tests simply aren't enough to test error responses generated through Spring Boot. I would argue that you shouldn't be testing Spring Boot's error handling. If you're customizing it in any way you can write Spring Boot integration tests (with an actual container) to verify error responses. And then for MockMvc tests focus on fully testing the web layer while expecting exceptions to bubble up.

This is a typical unit vs integration tests trade off. You do unit tests even if they don't test everything because they give you more control and run faster.

Comment From: wilkinsona

Thanks, @rstoyanchev.

How can we write tests for error conditions using default spring-boot JSON responses, then?

@xak2000 Rossen's already covered this, but I wanted to give you a direct answer. If you really want to test the precise format of the error response then you can use an integration test using @SpringBootTest configured with a DEFINED_PORT or RANDOM_PORT web environment and TestRestTemplate.

Comment From: mrcosta

And just to give visibility in terms of implementation, that's how I'm doing:

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment  = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SizesRestControllerIT {

    @LocalServerPort
    int port;

    @Before
    public void setUp() {
        RestAssured.port = port;
    }

    @Test
    public void test2() throws InterruptedException {
        given().basePath("/clothes").get("").then().statusCode(200);
    }
}

Comment From: tunix

Unfortunately TestRestTemplate doesn't work well with Spring REST Docs. 😞

Comment From: rstoyanchev

Spring Boot 2 supports the WebTestClient with @SpringBootTest and so does Spring REST Docs 2.0. I haven't tried the combination but you actually need access to the WebTestClient.Builder in order to apply Spring REST Docs. You might be able to do something like this with the injected client:

webTestClient.mutate()
        .filter(documentationConfiguration(restDocumentation))
        .build()

@snicoll, @wilkinsona do you have a better idea?

Comment From: wilkinsona

We have a pull request lined up for RC1 that'll auto-configure a WebTestClient with REST Docs: https://github.com/spring-projects/spring-boot/pull/10969

Comment From: tunix

My project uses Spring Boot 1.5.8 -- unfortunately these changes don't apply for my project.

Comment From: Jan-Dosedel

I had the same problem with using implementation of HandlerExceptionResolver interface for global error handling in the server. Where was also writing of the response based on throwed exception during request processing (implementation of the method resolveException ). I got the same exception from test using mockMvc : org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException (although server returns correctly written response with appropriate http status, etc.).

My problem was fixed by returning new (empty) instance of ModelAndView class in resolveException method instead of null value.

Comment From: cogit2

I could achieve this with WebTestClient: https://stackoverflow.com/a/55731539/3125006

Comment From: rburgst

the problem with using TestRestTemplate is that you loose the ability to setup your test data in a @Transactional integration test inside the test method, therefore, keeping the database clean for other tests to run. TestRestTemplate causes the request to be processed in a different thread and therefore, you have to persistently modify the database which is a pain to clean up. I would be ok if the response is not the EXACT same for MockMvcTests, but at least the HTTP status returned should be honoured.

Comment From: satishpatro44

Same. Mockmvc is not giving the option to handle custom service exception even with controller advice and RestTemplate is not allowing us to rollback. What option do we have to check at controller level?

Comment From: RubenGamarrarodriguez-tomtom

maybe this helps with a MockMvc approach: 

//... fooMockMvc
        .andReturn()
        .getResolvedException()
        .getMessage();

source: https://github.com/spring-projects/spring-framework/issues/17290#issuecomment-453422142

Comment From: rubensa

maybe this helps with a MockMvc approach:

java //... fooMockMvc .andReturn() .getResolvedException() .getMessage();

source: spring-projects/spring-framework#17290 (comment)

This only works if your Exception is annotated with @ResponseStatus as, then, the ResponseStatusExceptionResolver handles the exception. If the Exception is not annotated, then, it is thrown encapsulated inside a NestedServletException.

In that case you can test it with something like:

package org.eu.rubensa.springboot.error;

import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * Using this annotation will disable full auto-configuration and instead apply
 * only configuration relevant to MVC tests
 * (i.e. @Controller, @ControllerAdvice, @JsonComponent,
 * Converter/GenericConverter, Filter, WebMvcConfigurer and
 * HandlerMethodArgumentResolver beans but not @Component, @Service
 * or @Repository beans).
 * <p>
 * By default, tests annotated with @WebMvcTest will also auto-configure
 * MockMvc.
 * <p>
 * For more fine-grained control of MockMVC the @AutoConfigureMockMvc annotation
 * can be used.
 * <p>
 * By default MockMVC printOnlyOnFailure = true so information is printed only
 * if the test fails.
 */
@WebMvcTest()
public class MockMvcNestedServletExceptionTest {
  /**
   * MockMvc is not a real servlet environment, therefore it does not redirect
   * error responses to ErrorController, which produces error response.
   * <p>
   * See: https://github.com/spring-projects/spring-boot/issues/5574
   */
  @Autowired
  private MockMvc mockMvc;

  @Test
  public void testRuntimeException() throws Exception {
    Assertions
        .assertThatThrownBy(
            () -> mockMvc.perform(MockMvcRequestBuilders.get("/exception").contentType(MediaType.APPLICATION_JSON)))
        .hasCauseInstanceOf(RuntimeException.class).hasMessageContaining("The exception message");
  }

  /**
   * A nested @Configuration class wild be used instead of the application’s
   * primary configuration.
   * <p>
   * Unlike a nested @Configuration class, which would be used instead of your
   * application’s primary configuration, a nested @TestConfiguration class is
   * used in addition to your application’s primary configuration.
   */
  @Configuration
  /**
   * Tells Spring Boot to start adding beans based on classpath settings, other
   * beans, and various property settings.
   */
  @EnableAutoConfiguration
  /**
   * The @ComponentScan tells Spring to look for other components, configurations,
   * and services in the the TestWebConfig package, letting it find the
   * TestController class.
   * <p>
   * We only want to test the classes defined inside this test configuration so
   * not using it.
   */
  static class TestConfig {
    @RestController
    public class TestController {
      @GetMapping("/exception")
      public void getException() {
        throw new RuntimeException("The exception message");
      }
    }
  }
}