In https://github.com/spring-projects/spring-boot/commit/a170bfcc76cfb47b1c94481ccd35509eaa619b74, auto-configuration of Logback metrics was moved out of MetricsAutoConfiguation into a separate auto-configuration class. MetricsAutoConfigurationWithLog4j2AndLogbackTests wasn't updated at the same time so it is no longer testing anything – when using only MetricsAutoConfiguration there will never be a LogbackMetrics bean irrespective of what's on the classpath