There are cases where you'd like to not do anything special in case of error. Spring Boot already does that for its TestRestTemplate. ResponseErrorHandler makes it a tad harder as it has two handleError methods depending on the level of information available. This changed recently (https://github.com/spring-projects/spring-framework/pull/28958) which broke them.
Introducing a noop implementation that SB and others can use would be beneficial.