Given dependency on org.springframework.boot:spring-boot-starter-test (tested 2.5.6) with mockito in classpath

When using mockk's spies to wrap Spring-managed beans And those beans happen to be AOP'ed (e.g. @Cacheable etc.)

An unexpected error featuring mockito SpringBootMockResolver is observed upon the tests teardown (full stacktrace further down below):

java.lang.IllegalStateException: Failed to unwrap proxied object
    at org.springframework.test.util.AopTestUtils.getUltimateTargetObject(AopTestUtils.java:97) ~[spring-test-5.3.12.jar:5.3.12]
    at org.springframework.boot.test.mock.mockito.SpringBootMockResolver.resolve(SpringBootMockResolver.java:35) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.mockito.internal.util.MockUtil.resolve(MockUtil.java:118) ~[mockito-core-3.9.0.jar:na]
..
    at org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.afterTestMethod(ResetMocksTestExecutionListener.java:64) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.springframework.test.context.TestContextManager.afterTestMethod(TestContextManager.java:445) ~[spring-test-5.3.12.jar:5.3.12]
...

Steps to reproduce

(demo project at https://github.com/mgurov/mockk-v-cached/tree/spring-boot/issues%2F28604 )

Given gradle dependencies 

testImplementation("org.springframework.boot:spring-boot-starter-test") {
   //exclude(module = "mockito-core") // workaround 0: no problem with mockito excluded
}
testImplementation("io.mockk:mockk:1.12.0")

and a bean: 

@Component
class Service {

    @Cacheable("response_cache")
    fun respondCached(input: String) = input + "_" + Instant.now().toEpochMilli()

}

When the bean is wrapped by a mockk spy and injected back to the Spring context: 

import io.mockk.spyk

@Configuration
class InjectSpiesConfiguration{
    @Bean
    @Primary
    fun spiedService(service: Service): Service {
        return spyk(service)
    }
}

The following test crashes ResetMocksTestExecutionListener: 

@SpringBootTest
class ServiceCachedSpiedTest(
    @Autowired private val service: Service
) {
    @Test
    fun `this fails because cached`() {

        every { service.respondCached("empty") } returns ""

        assertThat(service.respondCached("empty")).isNotNull()
    }
}

with the stacktrace: 

java.lang.IllegalStateException: Failed to unwrap proxied object
    at org.springframework.test.util.AopTestUtils.getUltimateTargetObject(AopTestUtils.java:97) ~[spring-test-5.3.12.jar:5.3.12]
    at org.springframework.boot.test.mock.mockito.SpringBootMockResolver.resolve(SpringBootMockResolver.java:35) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.mockito.internal.util.MockUtil.resolve(MockUtil.java:118) ~[mockito-core-3.9.0.jar:na]
    at org.mockito.internal.util.MockUtil.isMock(MockUtil.java:108) ~[mockito-core-3.9.0.jar:na]
    at org.mockito.internal.util.DefaultMockingDetails.isMock(DefaultMockingDetails.java:32) ~[mockito-core-3.9.0.jar:na]
    at org.springframework.boot.test.mock.mockito.MockReset.get(MockReset.java:106) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.resetMocks(ResetMocksTestExecutionListener.java:82) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.resetMocks(ResetMocksTestExecutionListener.java:70) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener.afterTestMethod(ResetMocksTestExecutionListener.java:64) ~[spring-boot-test-2.5.6.jar:2.5.6]
    at org.springframework.test.context.TestContextManager.afterTestMethod(TestContextManager.java:445) ~[spring-test-5.3.12.jar:5.3.12]
    at org.springframework.test.context.junit.jupiter.SpringExtension.afterEach(SpringExtension.java:206) ~[spring-test-5.3.12.jar:5.3.12]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAfterEachCallbacks$11(TestMethodTestDescriptor.java:253) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$12(TestMethodTestDescriptor.java:269) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$13(TestMethodTestDescriptor.java:269) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) ~[na:na]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeAllAfterMethodsOrCallbacks(TestMethodTestDescriptor.java:268) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeAfterEachCallbacks(TestMethodTestDescriptor.java:252) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65) ~[junit-jupiter-engine-5.7.2.jar:5.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) ~[na:na]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) ~[na:na]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) ~[junit-platform-engine-1.7.2.jar:1.7.2]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108) ~[na:na]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) ~[na:na]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) ~[na:na]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) ~[na:na]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) ~[na:na]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96) ~[na:na]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75) ~[na:na]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99) ~[na:na]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79) ~[na:na]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75) ~[na:na]
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) ~[na:na]
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) ~[na:na]
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) ~[na:na]
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) ~[na:na]
    at com.sun.proxy.$Proxy2.stop(Unknown Source) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) ~[na:na]
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) ~[na:na]
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) ~[na:na]
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) ~[na:na]
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) ~[gradle-worker.jar:na]
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74) ~[gradle-worker.jar:na]
Caused by: java.lang.IllegalStateException: Failed to unwrap proxied object
    at org.springframework.test.util.AopTestUtils.getUltimateTargetObject(AopTestUtils.java:97) ~[spring-test-5.3.12.jar:5.3.12]
    at org.springframework.test.util.AopTestUtils.getUltimateTargetObject(AopTestUtils.java:92) ~[spring-test-5.3.12.jar:5.3.12]
    ... 79 common frames omitted
Caused by: io.mockk.MockKException: no answer found for: TargetSource(child of #1#2).getTarget()
    at io.mockk.impl.stub.MockKStub.defaultAnswer(MockKStub.kt:93) ~[mockk-1.12.0.jar:na]
    at io.mockk.impl.stub.MockKStub.answer(MockKStub.kt:42) ~[mockk-1.12.0.jar:na]
    at io.mockk.impl.recording.states.AnsweringState.call(AnsweringState.kt:16) ~[mockk-1.12.0.jar:na]
    at io.mockk.impl.recording.CommonCallRecorder.call(CommonCallRecorder.kt:53) ~[mockk-1.12.0.jar:na]
    at io.mockk.impl.stub.MockKStub.handleInvocation(MockKStub.kt:266) ~[mockk-1.12.0.jar:na]
    at io.mockk.impl.instantiation.JvmMockFactoryHelper$mockHandler$1.invocation(JvmMockFactoryHelper.kt:23) ~[mockk-1.12.0.jar:na]
    at io.mockk.proxy.jvm.advice.Interceptor.call(Interceptor.kt:21) ~[mockk-agent-jvm-1.12.0.jar:na]
    at io.mockk.proxy.jvm.advice.BaseAdvice.handle(BaseAdvice.kt:42) ~[mockk-agent-jvm-1.12.0.jar:na]
    at io.mockk.proxy.jvm.advice.jvm.JvmMockKProxyInterceptor.interceptNoSuper(JvmMockKProxyInterceptor.java:45) ~[mockk-agent-jvm-1.12.0.jar:na]
    at org.springframework.aop.TargetSource$Subclass0.getTarget(Unknown Source) ~[spring-aop-5.3.12.jar:5.3.12]
    at org.springframework.test.util.AopTestUtils.getUltimateTargetObject(AopTestUtils.java:90) ~[spring-test-5.3.12.jar:5.3.12]
    ... 80 common frames omitted

No exception is observed when the @Cacheable annotation is commented out on the target bean.

Expected behavior

No crashes, and definitely not from a mockito-specific test listener

Workarounds

  1. Remove the mockito from the classpath, e.g. in the example project:
testImplementation("org.springframework.boot:spring-boot-starter-test") {
   exclude(module = "mockito-core")
}
  1. Unwrap the target bean before wrapping with a spy: 
@Bean
@Primary
fun spiedUnwrappedService(service: Service): Service {
    val actualService: Service = AopTestUtils.getUltimateTargetObject(service)
    val spyk = spyk(actualService)
    return spyk
}

Comment From: wilkinsona

Thanks for the report.

Given that the root cause of the failure is in Mockk, having read the description above it's not clear to me why you have reported the problem here. We need to get the proxy's underlying target to determine whether or not it's a Mockito mock. It would appear that Mockk is intercepting the getTarget() call such that calling it fails. We could catch the exception and assume that it means that the target isn't a Mockito mock but that would be a broad solution to what appears to be a Mockk-specific problem. It's also not clear to me that it would help as I don't know what state the failed interception of the getTarget() call will have left things in.

Comment From: mgurov

Thanks for checking the report.

I see the point that it's probably Mockk who misbehaves in the root of the exception by not handling the call to getTarget() properly - that instance was supposed to be a spy after all.

It's still not very convenient to have an otherwise working test hindered by the org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener which - in my setup - isn't needed at all, and there seem to be no way to deactivate it (double-checking at https://stackoverflow.com/questions/69933720/how-to-disable-a-spring-factory-registered-via-spring-factories-by-a-dependency ) except for excluding dependencies - which isn't always an easy solution.

I guess, this issue in current form could be closed as not actionable.

Would it make sense for me to create an enhancement proposal to allow suppression (opt-out) of the execution of org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener instead?

Comment From: wilkinsona

except for excluding dependencies - which isn't always an easy solution

If you aren't using Mockito then excluding it shouldn't cause a problem. If you are using Mockito then I'm not aware of a scenario where you wouldn't want ResetMocksTestExecutionListener. Until we learn of such a scenario, I think it's hard to justify the additional complexity of a setting that allows the listener to be disabled.

Comment From: mgurov

If you are using Mockito then I'm not aware of a scenario where you wouldn't want ResetMocksTestExecutionListener.

In my case, we do have a relatively big project with some legacy. Mockito is used for older unit tests where no Spring context is involved, and consequently ResetMocksTestExecutionListener isn't invoked.

The new test code uses Mockk, and all of our integration tests that do involve Spring use Mockk, and not Mockito. Even if we used Mockito with Spring, we wouldn't need ResetMocksTestExecutionListener as our strategy towards higher tier tests is, for various reasons, to minimize the use of mocks by employing fakes or real implementations wrapped with spies instead. We also generate unique ids and references for every test run and use them for assertions instead of (Mokito|Mockk).any().

Under normal circumstances, the unnecessary ResetMocksTestExecutionListener executions don't pose a big problem. The price of scanning the application context after every test is negligible. But when we have something misbehaving in this context - like the issue we discuss, even though this misbehaviour might be rooted elsewhere - it would've been nice to have a simple option to opt-out of the ResetMocksTestExecutionListener while the root cause is being sorted out.

Comment From: wilkinsona

It has occurred to me that it's already possible to do what you want using a org.springframework.boot.test.context.DefaultTestExecutionListenersPostProcessor implementation to remove the unwanted test execution listeners. You can register a post-processor implementation by creating a META-INF/spring.factories file with a key=value entry where key is org.springframework.boot.test.context.DefaultTestExecutionListenersPostProcessor and value is the fully-qualified class name of your implementation. It'll then be loaded and called by getDefaultTestExecutionListenerClasses in SpringBootTestContextBootstrapper.

Comment From: mgurov

Great - the DefaultTestExecutionListenersPostProcessor worked: 

class SuppressMockitoResetMocksTestExecutionListener : DefaultTestExecutionListenersPostProcessor {
    override fun postProcessDefaultTestExecutionListeners(listeners: Set<Class<out TestExecutionListener>>): Set<Class<out TestExecutionListener>> {
        return listeners.filter {it != org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener::class.java}.toSet()
    }
}

Which provides are more structural solution to the potential test listener hiccups problem - and I don't see a reason to keep this issue open.

Thanks a lot for the support, @wilkinsona!

Comment From: wilkinsona

Excellent. Thanks for giving it a try and letting us know it worked.