SpringBoot Support Gradle's configuration cache with Gradle 6.7 and later

When playing around with the new configuration cache in gradle 6.6 I got this error when building our spring boot project with ./gradlew --configuration-cache --configuration-cache-problems=warn build: "Listener registration 'Gradle.buildFinished' by build 'myproject' is unsupported." together with this stack trace:

        org.gradle.api.InvalidUserCodeException: Listener registration 'Gradle.buildFinished' by build 'backend' is unsupported.
            at org.gradle.instantexecution.initialization.DefaultInstantExecutionProblemsListener.onBuildScopeListenerRegistration(InstantExecutionProblemsListener.kt:88)
            at jdk.internal.reflect.GeneratedMethodAccessor276.invoke(Unknown Source)
            at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.base/java.lang.reflect.Method.invoke(Method.java:566)
            at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
            at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
            at org.gradle.internal.event.DefaultListenerManager$ListenerDetails.dispatch(DefaultListenerManager.java:398)
            at org.gradle.internal.event.DefaultListenerManager$ListenerDetails.dispatch(DefaultListenerManager.java:380)
            at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
            at org.gradle.internal.event.DefaultListenerManager$EventBroadcast$ListenerDispatch.dispatch(DefaultListenerManager.java:368)
            at org.gradle.internal.event.DefaultListenerManager$EventBroadcast.dispatch(DefaultListenerManager.java:179)
            at org.gradle.internal.event.DefaultListenerManager$EventBroadcast.dispatch(DefaultListenerManager.java:153)
            at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
            at com.sun.proxy.$Proxy24.onBuildScopeListenerRegistration(Unknown Source)
            at org.gradle.invocation.DefaultGradle.notifyListenerRegistration(DefaultGradle.java:401)
            at org.gradle.invocation.DefaultGradle.buildFinished(DefaultGradle.java:382)
            at org.springframework.boot.gradle.plugin.SpringBootPlugin.unregisterUnresolvedDependenciesAnalyzer(SpringBootPlugin.java:135)
            .....

It would be great to find a solution that's compatible with the configuration cache here: https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-tools/spring-boot-gradle-plugin/src/main/java/org/springframework/boot/gradle/plugin/SpringBootPlugin.java#L141