It uses a build listener to report a summary of all the test failures at the end of the build. The configuration cache does not support build listeners and we need to use a BuildService instead. We can implement AutoCloseable to be called back when the service is no longer needed and implement OperationCompletionListener to ensure that this doesn't happen until the build has completed and all the test tasks have run.