Sam Brannen opened SPR-15366 and commented

Status Quo

Spring's support for JUnit Jupiter already supports detection of test configuration (e.g., (@ContextConfiguration) on @Nested classes.

However, if a @Nested class does not declare its own test configuration, Spring will not find the configuration from the enclosing class.

See also this discussion on Stack Overflow regarding nested test classes and @Transactional.

Proposal

Inspired by issue 8 from the spring-test-junit5 project, it would perhaps be desirable if the Spring TestContext Framework would discover test configuration on an enclosing class for a @Nested test class.

Deliverables

  • [x] Introduce @NestedTestConfiguration with EnclosingConfiguration enum supporting INHERITED and OVERRIDE modes.
  • [x] Make the default EnclosingConfiguration mode globally configurable via SpringProperties
  • [x] Switch the default EnclosingConfiguration mode to INHERITED
  • [x] @ContextConfiguration / @ContextHierarchy
  • [x] @ActiveProfiles
  • [x] @TestPropertySource / @TestPropertySources
  • [x] @WebAppConfiguration
  • [x] @TestConstructor
  • [x] @BootstrapWith
  • [x] @TestExecutionListeners
  • [x] @DirtiesContext
  • [x] @Transactional
  • [x] @Rollback / @Commit
  • ❌ @Sql / @SqlConfig / @SqlGroup
  • see #25913
  • ❌ Document @NestedTestConfiguration support in the reference manual
  • see #25912
  • ❌ Document @NestedTestConfiguration support and the switching of the default mode to INHERITED in the upgrade notes in the wiki
  • see #25912

Affects: 5.0

Issue Links: - #18722 Support nested test classes with SpringClassRule & SpringMethodRule - #21136 JUnit Jupiter @Nested class cannot share enclosing class's ApplicationContext if nested class is deemed to be a configuration candidate

6 votes, 10 watchers

Comment From: spring-projects-issues

Sam Brannen commented

Current work is being performed in the following branch:

https://github.com/sbrannen/spring-framework/tree/SPR-15366

Comment From: spring-projects-issues

Andy Wilkinson commented

Sam asked me to comment on this with reference to this Spring Boot issue. In looking at that issue I observed that the following test would start two separate application contexts:

@ExtendWith(SpringExtension.class)
@SpringBootTest
public class TwoDifferentContextsTests {

    @Autowired
    ApplicationContext context;

    @Nested
    class NestedTests {

        @Test
        public void test() {
        }

    }

}

One context is started for TwoDifferentContextsTests. Due to @SpringBootTest, it finds the @SpringBootApplication-annotated class and uses that as the basis for the context's configuration. A second context is started for NestedTests as it hasn't inherited @SpringBootTest from the enclosing class and, therefore, has different configuration. In fact, NestedTests doesn't really have any configuration. It's only able to have a context created for it due to the context customiser factories declared by spring-boot-test in spring.factories. It enters the if block rather than throwing in this code in AbstractDelegatingSmartContextLoader:

// If neither of the candidates supports the mergedConfig based on resources but
// ACIs or customizers were declared, then delegate to the annotation config
// loader.
if (!mergedConfig.getContextInitializerClasses().isEmpty() || !mergedConfig.getContextCustomizers().isEmpty()) {
    return delegateLoading(getAnnotationConfigLoader(), mergedConfig);
}

// else...
throw new IllegalStateException(String.format(
        "Neither %s nor %s was able to load an ApplicationContext from %s.", name(getXmlLoader()),
        name(getAnnotationConfigLoader()), mergedConfig));

Comment From: spring-projects-issues

Sam Brannen commented

I actually assumed it was due to an auto-registered feature of Spring Boot Test (i.e., something like a ContextCustomizer).

So thank you for confirming my suspicions!

Comment From: spring-projects-issues

Stefan Ludwig commented

FYI: I've extended the original bug example from the (GitHub issue) with "Boot - less" implementations of @Nested Spring REST Docs tests: https://github.com/slu-it/bug-spring-restdocs-nested-tests/tree/master/src/test/java/withoutboot

The following test fails because the WebApplicationContext could not be provided by the Spring Extension (not found in context):

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Stacktrace;

org.junit.jupiter.api.extension.ParameterResolutionException: Failed to resolve parameter [org.springframework.web.context.WebApplicationContext arg0] in executable [void withoutboot.FailingTest.setup(org.springframework.web.context.WebApplicationContext,org.springframework.restdocs.RestDocumentationContextProvider)]

    at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:221)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:174)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:135)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:116)
    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:302)
    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$12(ClassTestDescriptor.java:290)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:135)
    at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:155)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:134)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:109)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
    at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
    at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
    at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
    at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:65)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.web.context.WebApplicationContext' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1509)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1104)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1065)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveDependency(AbstractAutowireCapableBeanFactory.java:344)
    at org.springframework.test.context.junit.jupiter.ParameterAutowireUtils.resolveDependency(ParameterAutowireUtils.java:98)
    at org.springframework.test.context.junit.jupiter.SpringExtension.resolveParameter(SpringExtension.java:177)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:207)
    ... 69 more

If the context configuration is duplicated by adding @SpringJUnitWebConfig(classes = Application.class) on the @Nested class it works (as previously mentioned by Andy Wilkinson). But this doesn't feel very intuitive.

(comment moved from SPR-16595)

Comment From: spring-projects-issues

Sam Brannen commented

Stefan Ludwig,

I think the behavior you've described might actually be a bug in JUnit Jupiter instead of Spring.

Can you please tell me what happens if you move the setup for MockMvc into a constructor as follows?

@SpringJUnitWebConfig(Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    FailingTest(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Thanks in advance for feedback!

Comment From: spring-projects-issues

Stefan Ludwig commented

If I do that, it works! I added that code as an working example to the repository.

Comment From: spring-projects-issues

Sam Brannen commented

If I do that, it works!

Awesome... and depressing... at the same time. ;-)

Awesome that it works for you!

Depressing (in a facetious way) for me because that means it is in fact an issue in JUnit Jupiter's handling of @BeforeEach methods with regard to the ExtensionContext supplied to a ParameterResolver registered for an enclosing test class when executing a test method within a @Nested test class.

And if that's too much of a mouthful for you, don't worry: I'll address the issue you've described within JUnit Jupiter.

Cheers,

Sam

Comment From: spring-projects-issues

Stefan Ludwig commented

Glad to have helped! And sorry to depress you ;)

But are you sure it's not Springs management / caching of application contexts?

I thought it might be the same issue as, or at least very closely related to, #21136 because when I debug the following example:

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Test
    void testAndDocumentation() throws Exception {
        mockMvc.perform(get("/hello"))
            .andExpect(status().is2xxSuccessful())
            .andExpect(jsonPath("message", equalTo("Hello World!")))
            .andDo(document("hello-get-200"));
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext). I get an GenericWebApplicationContext with all the expected beans for the main test class. But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

That sounds to me like what was described in #21136. Especially because adding @SpringJUnitWebConfig(classes = Application.class) to the @Nested class actually makes the test work and for both contexts (nested and enclosing class) the same application context is used in the parameter resolver.

Comment From: spring-projects-issues

Sam Brannen commented

FYI: I have opened the following issue for JUnit Jupiter.

https://github.com/junit-team/junit5/issues/1332

Comment From: spring-projects-issues

Sam Brannen commented

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext). I get an GenericWebApplicationContext with all the expected beans for the main test class. But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

Well, you have spring-boot-test in the classpath -- right?

Otherwise, Spring will not load any ApplicationContext for the @Nested class (if the nested class does not provide its own annotations).

Comment From: spring-projects-issues

Sam Brannen commented

That sounds to me like what was described in #21136.

No. #21136 covers use cases where @Import is used on a test class.

Comment From: spring-projects-issues

Stefan Ludwig commented

Oh, I did not think about the spring-boot-test dependency existing influencing the test. Thanks for clarifying!

Comment From: spring-projects-issues

Sam Brannen commented

You're welcome.

And... yeah... I agree that having spring-boot-test change things just by being present in the classpath can be a bit confusing at times. ;-)

Comment From: spring-projects-issues

Sam Brannen commented

I added that code as an working example to the repository.

Speaking of which, thanks for creating that!

It's a nice collection of examples.

Comment From: spring-projects-issues

Andy Wilkinson commented

having spring-boot-test change things just by being present in the classpath

It doesn’t. You have to using SpringRunner or SpringExtension too. At that point we’re at the mercy of the registration mechanism for ContextCustomizerFactory and the like. Is there a way to opt those back out again?

Comment From: spring-projects-issues

Sam Brannen commented

I of course meant "change things for tests executing with the Spring TestContext Framework".

Sorry if that was not apparent.

Comment From: spring-projects-issues

Sam Brannen commented

Is there a way to opt those back out again?

No, there is currently no built-in mechanism for disabling a ContextCustomizer that was registered automatically.

Maybe we should introduce something analogous to Boot's @EnableAutoConfiguration(exclude = ...) support.

Comment From: spring-projects-issues

Sam Brannen commented

Another question on Stack Overflow: https://stackoverflow.com/questions/53236807/spring-boot-testing-run-script-in-a-nested-test-sql-script-sql/53240875

Comment From: sdeleuze

Hey @sbrannen, I got several developers (including the awesome @jnizet) asking for a fix for https://github.com/spring-projects/spring-boot/issues/12470 which depend on that issues, any chance you could fix it for 5.2 in order to allow Boot team to support correctly nested classes in 2.2?

Comment From: sbrannen

@sdeleuze, I'll do my best to prioritize this one.

Comment From: sbrannen

Now tentatively slated for 5.2 M1.

Comment From: sbrannen

Current work on this issue is now being performed in the following branch:

https://github.com/spring-projects/spring-framework/compare/issues/gh-19930

Comment From: ShahBinoy

I need this functionality for our Spring-boot projects. Considering it as an acceptable risk, what would be the best strategy to adopt this functionality in my project? @sbrannen. We do host our own nexus repository.

Comment From: cheese10yun

@ActiveProfiles(profiles = "test")
@ExtendWith({RestDocumentationExtension.class, SpringExtension.class})
@AutoConfigureMockMvc
@Import(RestDocsConfiguration.class)
@Transactional
@SpringBootTest
public class TestSupport {

    @Autowired
    protected MockMvc mockMvc;

    @BeforeEach
    public void setUp(final WebApplicationContext context, final RestDocumentationContextProvider provider) {
        this.mockMvc = MockMvcBuilders.webAppContextSetup(context)
            .apply(documentationConfiguration(provider))
            .alwaysDo(print())
            .alwaysDo(MockMvcRestDocumentation.document(
                "{method-name}",
                preprocessRequest(prettyPrint()),
                preprocessResponse(prettyPrint())))
            .build();
    }
}

class Tset {

    @DisplayName("get")
    @Nested
    class Get extends TestSupport {

        @DisplayName("test..")
        @Test
        void test() throws Exception {
            mockMvc.perform(get("/test"))
                .andExpect(status().isBadRequest());
        }
    }

}

The above code works. But I don't know if it's a good way.

Comment From: ShahBinoy

Any progress on this ?

Comment From: sbrannen

Any progress on this ?

As can be seen on the right-hand side of this page, this issue is currently slated for 5.3 M1.

Comment From: gscokart

It would indeed be nice to have it in the next release to be able to correctly use Nested tests.

I din't looked completely at the code on the branch, but I saw you wanted to use a @TestConfiguration annotation. If not too late, it might be a good idea to give it an other name because springboot already has a @TestConfiguration. Maybe @NestedTestConfiguration ?

Comment From: sbrannen

The TestInstances API introduced in conjunction with https://github.com/junit-team/junit5/issues/1618 may be of interest regarding the implementation of this feature.

Comment From: sbrannen

I saw you wanted to use a @TestConfiguration annotation. If not too late, it might be a good idea to give it an other name because Spring Boot already has a @TestConfiguration. Maybe @NestedTestConfiguration ?

Good point!

We should definitely aim to avoid naming conflicts with Spring Boot's testing support.

Comment From: sbrannen

Update

As can be seen in the updated description for this issue (see the check boxes), I am making progress in my feature branch.

So, for anyone willing to live on the bleeding edge, feel free to build spring-test from that branch and give it a try.

At this point in time, we don't have snapshot builds for that since this work is not yet in master, but that may change in the coming days.

Comment From: sbrannen

Reopening to address additional deliverables.