Spring TestExecutionListener does not get applied for a new context for @Nested test class

When setting up @SpringBootTest as follows:

@SpringBootTest(classes = [MyTestComponent::class])
@TestExecutionListeners(value = [MyTestExecutionListener::class], mergeMode = MERGE_WITH_DEFAULTS)
@TestConstructor(autowireMode = ALL)
class TestExecutionListenersNestedApplicationTests(private val testComponent: MyTestComponent) {
  // ...
  @TestComponent
  class MyTestComponent(var enabled: Boolean = false)

  class MyTestExecutionListener : TestExecutionListener {

    override fun beforeTestMethod(testContext: TestContext) {
      testContext.applicationContext.getBean(MyTestComponent::class.java).enabled = true
    }

    override fun afterTestMethod(testContext: TestContext) {
      testContext.applicationContext.getBean(MyTestComponent::class.java).enabled = false
    }
  }
  // ...
}  

I would expect both of these @Nested test classes to behave the same way:

  @Nested
  inner class ClassWithSameContext {

    @Test
    fun noProblemo() {
      assertThat(testComponent.enabled).isTrue
    }
  }

  @Nested
  @TestPropertySource(properties = ["test=true"])
  inner class ClassWithDifferentContext {

    @Test
    fun problemo() {
      assertThat(testComponent.enabled).isTrue
    }
  }

However, the test in the first one passes and the test in the second one fails.

This looks like a bug to me, or am I missing something?

Stepping through the test with a debugger shows that a context gets processed in the beforeTestMethod() method, but that context does not seem to be used in the test.

See here for a runnable sample project.

Comment From: wilkinsona

Thanks for the report and sample. This problem doesn't have anything to do with Spring Boot itself as it can be reproduced using a pure Spring Framework based test:

package io.github.vootelerotov.testexecutionlistenersnested

import io.github.vootelerotov.testexecutionlistenersnested.TestExecutionListenersNestedApplicationTests.MyTestComponent
import io.github.vootelerotov.testexecutionlistenersnested.TestExecutionListenersNestedApplicationTests.MyTestExecutionListener
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Nested
import org.junit.jupiter.api.Test
import org.springframework.stereotype.Component
import org.springframework.test.context.TestConstructor
import org.springframework.test.context.TestConstructor.AutowireMode.ALL
import org.springframework.test.context.TestContext
import org.springframework.test.context.TestExecutionListener
import org.springframework.test.context.TestExecutionListeners
import org.springframework.test.context.TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS
import org.springframework.test.context.TestPropertySource
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig

@SpringJUnitConfig(classes = [MyTestComponent::class])
@TestExecutionListeners(value = [MyTestExecutionListener::class], mergeMode = MERGE_WITH_DEFAULTS)
@TestConstructor(autowireMode = ALL)
class TestExecutionListenersNestedApplicationTests(private val testComponent: MyTestComponent) {

  @Test
  fun worksWithoutNestedClassWithNewContext() {
    assertThat(testComponent.enabled).isTrue
  }

  @Nested
  inner class ClassWithSameContext {

    @Test
    fun noProblemo() {
      assertThat(testComponent.enabled).isTrue
    }
  }

  @Nested
  @TestPropertySource(properties = ["test=true"])
  inner class ClassWithDifferentContext {

    @Test
    fun problemo() {
      assertThat(testComponent.enabled).isTrue
    }
  }

  @Component
  class MyTestComponent(var enabled: Boolean = false)

  class MyTestExecutionListener : TestExecutionListener {

    override fun beforeTestMethod(testContext: TestContext) {
      testContext.applicationContext.getBean(MyTestComponent::class.java).enabled = true
    }

    override fun afterTestMethod(testContext: TestContext) {
      testContext.applicationContext.getBean(MyTestComponent::class.java).enabled = false
    }
  }

}

We'll transfer the issue so that the Framework team can take a look.

Comment From: sbrannen

This looks like a bug to me, or am I missing something?

This is not a bug. It is the expected behavior.

The enclosing test class instance is injected with beans from its own ApplicationContext, not with beans from the ApplicationContext of a nested test class or subclass. The same holds true for any test class.

Similarly, a TestExecutionListener is provided access to the ApplicationContext for the test class with which the listener is associated.

Thus, when the MyTestExecutionListener is invoked for ClassWithDifferentContext, testContext.getApplicationContext().getBean(MyTestComponent.class) accesses the MyTestComponent bean in the ApplicationContext for ClassWithDifferentContext.

But... the testComponent field/property is injected from the ApplicationContext for TestExecutionListenersNestedApplicationTests.

So your test method and your listener are interacting with different instances of MyTestComponent from different application contexts.

The following modified (and converted to Java) version of your example demonstrates this.

@SpringJUnitConfig(MyTestComponent.class)
@TestExecutionListeners(listeners = MyTestExecutionListener.class, mergeMode = MERGE_WITH_DEFAULTS)
@TestConstructor(autowireMode = ALL)
class TestExecutionListenersNestedApplicationTests {

    private final MyTestComponent testComponent;

    TestExecutionListenersNestedApplicationTests(MyTestComponent testComponent) {
        this.testComponent = testComponent;
    }

    @Test
    void worksWithoutNestedClassWithNewContext() {
        assertThat(testComponent.enabled).isTrue();
    }

    @Nested
    class ClassWithSameContext {
        @Test
        void noProblemo() {
            assertThat(testComponent.enabled).isTrue();
        }
    }

    @Nested
    @TestPropertySource(properties = "test=true")
    class ClassWithDifferentContext {

        private final MyTestComponent localTestComponent;

        ClassWithDifferentContext(MyTestComponent testComponent) {
            this.localTestComponent = testComponent;
        }

        @Test
        void problemo() {
            assertThat(testComponent.enabled).isFalse();
            assertThat(localTestComponent.enabled).isTrue();
        }
    }

    @Component
    static class MyTestComponent {

        private boolean enabled;

        boolean isEnabled() {
            return enabled;
        }

        void setEnabled(boolean enabled) {
            this.enabled = enabled;
        }
    }

    static class MyTestExecutionListener implements TestExecutionListener {
        @Override
        public void beforeTestMethod(TestContext testContext) {
            testContext.getApplicationContext().getBean(MyTestComponent.class).setEnabled(true);
        }

        @Override
        public void afterTestMethod(TestContext testContext) {
            testContext.getApplicationContext().getBean(MyTestComponent.class).setEnabled(false);
        }
    }

}

Specifically, the problemo() test method asserts that the local MyTestComponent did in fact have its enabled flag set to true by the TestExecutionListener.

---

The only way to access the ApplicationContext of a different test class from a Spring TestExecutionListener is by creating a new TestContextManager for the specific class. The following demonstrates how you could interact with the context for the enclosing class, which would make your original test pass.

@Override
public void beforeTestMethod(TestContext testContext) {
    new TestContextManager(TestExecutionListenersNestedApplicationTests.class)
        .getTestContext().getApplicationContext().getBean(MyTestComponent.class).setEnabled(true);
}

However, if you choose to implement a JUnit Jupiter extension instead of a Spring TestExecutionListener you can come up with a cleaner (though admittedly more involved) solution. If you completely remove your custom TestExecutionListener and add the following BeforeEachCallback and AfterEachCallback extensions to TestExecutionListenersNestedApplicationTests, you'll see that your original tests pass.

@RegisterExtension
BeforeEachCallback enableComponentExtension = context ->
    getApplicationContextForTopLevelTestClass(context).getBean(MyTestComponent.class).setEnabled(true);

@RegisterExtension
AfterEachCallback disableComponentExtension = context ->
    getApplicationContextForTopLevelTestClass(context).getBean(MyTestComponent.class).setEnabled(false);

There's no built-in support for getApplicationContextForTopLevelTestClass(), but the following proof of concept (which should be simplified) demonstrates that it's possible.

private static ApplicationContext getApplicationContextForTopLevelTestClass(ExtensionContext context) {
    List<Object> enclosingInstances = context.getRequiredTestInstances().getEnclosingInstances();
    if (enclosingInstances.isEmpty()) {
        return SpringExtension.getApplicationContext(context);
    }
    else {
        Class<?> topLevelTestClass = enclosingInstances.get(0).getClass();
        ExtensionContext parent = context.getParent().orElse(null);
        while (parent != null) {
            if (parent.getRequiredTestClass().equals(topLevelTestClass)) {
                break;
            }
            parent = parent.getParent().orElse(null);
        }
        return SpringExtension.getApplicationContext(parent);
    }
}

In light of the above, I am closing this issue.

Though if you have any further questions, feel free to ask.

Comment From: vootelerotov

Thus, when the MyTestExecutionListener is invoked for ClassWithDifferentContext, testContext.getApplicationContext().getBean(MyTestComponent.class) accesses the MyTestComponent bean in the ApplicationContext for ClassWithDifferentContext.

But... the testComponent field/property is injected from the ApplicationContext for TestExecutionListenersNestedApplicationTests.

This seems to be what I missed in my mental model. Makes sense.

@sbrannen, thanks for taking the time to give a very thorough answer.

Comment From: sbrannen

@sbrannen, thanks for taking the time to give a very thorough answer.

You're welcome.

The reason it's so "thorough" is that your use case sparked some thought experiments in my head, and I wanted to verify what options actually exist to support such use cases.