Spring Already failed contexts should not be started again for each test

Before this commit, Spring Test Framework caches contexts shared among different test methods. However, context failures are not cached and failing context is started again and again for each test method the context match.

Starting a context is often a time and space consuming operation. It should be avoided when not necessary. If context fails once, there rather not possible it run second time.

This commit remembers failed context and save resources by not starting them again

Comment From: sbrannen

This appears to be a PR for issue #14182.

Comment From: michaldo

Indeed, that's quite old story.

The interesting point is that the more Spring test suite is optimized, the more it suffers repeated context start.

In suboptimal suite each test has own contexts. Starting many contexts consumes much resources but failure one context does not impact performance.

Optimal suite starts one context for all tests - it saves a lot of time. But if the contest fails, not only all tests fail, but also all tests repeat context start - suite pay back the optimization benefit

Comment From: michaldo

Recently I found a question how to limit number of contexts used in integration test: https://stackoverflow.com/questions/62621248/how-to-count-the-number-of-times-the-spring-boot-loads-on-spring-tests/65286912#65286912

The only answer I know is log number of context used, without automatic break when number exceeds the limit.

I thinking about implementation of the limit. But this PR will be useful for the implementation

Comment From: cru5ader

Recently I found a question how to limit number of contexts used in integration test: https://stackoverflow.com/questions/62621248/how-to-count-the-number-of-times-the-spring-boot-loads-on-spring-tests/65286912#65286912

The only answer I know is log number of context used, without automatic break when number exceeds the limit.

I thinking about implementation of the limit. But this PR will be useful for the implementation

@michaldo How did you solve this issue eventually ? I also want to avoid retry of loading of already failed application context. Thanks

Comment From: michaldo

I had to accept that retrying failed contexts in unavoidable

Comment From: michaldo

feature independently implemented:

Spring Framework 6.1 will finally have "test context failure threshold" support! 🍃 🥳