SpringBoot Work around Logback's LoggerContext listener support not being thread-safe

springboot version:  
         'org.springframework.boot:spring-boot-dependencies:2.1.6.RELEASE'
Junit5: 
    testImplementation 'org.junit.jupiter:junit-jupiter:5.4.2'
    testImplementation 'org.junit.platform:junit-platform-runner:1.4.2'
    testImplementation 'org.junit.platform:junit-platform-launcher:1.4.2'

Scenario: we met the same ConcurrentModificationException issue when we manually run multiple SpringApplications in Junit5 parallel mode.

Issue:

below is exception stack:

java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
at java.util.ArrayList$Itr.next(ArrayList.java:859)
at ch.qos.logback.classic.LoggerContext.fireOnLevelChange(LoggerContext.java:317)
at ch.qos.logback.classic.Logger.setLevel(Logger.java:173)
at org.springframework.boot.logging.logback.LogbackConfigurator.root(LogbackConfigurator.java:101)
at org.springframework.boot.logging.logback.DefaultLogbackConfiguration.apply(DefaultLogbackConfiguration.java:95)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadDefaults(LogbackLoggingSystem.java:141)
at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:85)
at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:60)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.initialize(LogbackLoggingSystem.java:117)
at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:293)
at org.springframework.boot.context.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:266)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:229)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:202)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:75)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:54)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:347)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:139)
at org.springframework.cloud.bootstrap.BootstrapApplicationListener.bootstrapServiceContext(BootstrapApplicationListener.java:203)
at org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:114)
at org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:71)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:75)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:54)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:347)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)

Comment From: wilkinsona

Thanks for the report. Unfortunately this is a known issue with Logback which means that concurrently adding or removing a LoggerContextListener and firing an event to the registered listeners will fail.

Spring Boot cannot restrict when listener registration changes or when events are fired to them so I do not believe it is possible for us to work around the problem entirely. That said, we may be able to do something to reduce the likelihood of it happening. I'll leave this issue open so that we can explore that possibility. In the meantime, you may need to use serial test execution. Alternatively you could try switching to Log4j2.

Comment From: wilkinsona

We've discussed this and unfortunately we don't think there's a reliable way in which we can fix this. Your best bet is to use one of the two alternatives I mentioned above.

Comment From: pschichtel

I also just switched to parallel test execution and noticed this issue.

I now overwrite the logback logging system with this one, to initialize logback just once.

private class SynchronizedSingletonLogbackLoggingSystem(private val classLoader: ClassLoader) : LoggingSystem() {
    private val system = lock.withLock {
        cachedLoggingSystem ?: LogbackLoggingSystem(classLoader).apply {
            beforeInitialize()
            cachedLoggingSystem = this
        }
    }

    override fun beforeInitialize() = lock.withLock {
        if (!initialized) {
            initialized = true
            system.beforeInitialize()
        }
    }

    override fun initialize(initializationContext: LoggingInitializationContext?, configLocation: String?, logFile: LogFile?) = lock.withLock {
        if (!initialized) {
            initialized = true
            system.initialize(initializationContext, configLocation, logFile)
        }
    }

    override fun cleanUp() {
    }
    override fun getShutdownHandler(): Runnable? = null
    override fun getSystemProperties(environment: ConfigurableEnvironment?): LoggingSystemProperties = lock.withLock {
        system.getSystemProperties(environment)
    }
    override fun getSupportedLogLevels(): MutableSet<LogLevel> = lock.withLock {
        system.supportedLogLevels
    }
    override fun setLogLevel(loggerName: String?, level: LogLevel?) = lock.withLock {
        system.setLogLevel(loggerName, level)
    }
    override fun getLoggerConfigurations(): MutableList<LoggerConfiguration> = lock.withLock {
        system.loggerConfigurations
    }
    override fun getLoggerConfiguration(loggerName: String?): LoggerConfiguration = lock.withLock {
        system.getLoggerConfiguration(loggerName)
    }

    private companion object {
        @Volatile
        private var initialized = false
        private var cachedLoggingSystem: LogbackLoggingSystem? = null
        private val lock = ReentrantLock()
    }
}