Hi,
I have a memory leak on reactive-stream with Spring boot 2.7.12. All versions (reactor, reactive-stream, mongo driver, netty) is provided by Spring boot 2.7.12, i don't override it.
Here is the stack trace detected by MAT:
AsyncGetter-2-thread-1
at org.apache.logging.log4j.core.appender.OutputStreamManager.write([BIIZ)V (OutputStreamManager.java:220)
at org.apache.logging.log4j.core.appender.OutputStreamManager.write([BZ)V (OutputStreamManager.java:190)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.writeByteArrayToManager(Lorg/apache/logging/log4j/core/LogEvent;)V (AbstractOutputStreamAppender.java:206)
at org.apache.logging.log4j.core.appender.SocketAppender.directEncodeEvent(Lorg/apache/logging/log4j/core/LogEvent;)V (SocketAppender.java:459)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(Lorg/apache/logging/log4j/core/LogEvent;)V (AbstractOutputStreamAppender.java:190)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(Lorg/apache/logging/log4j/core/LogEvent;)V (AbstractOutputStreamAppender.java:181)
at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(Lorg/apache/logging/log4j/core/LogEvent;)V (AppenderControl.java:161)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(Lorg/apache/logging/log4j/core/LogEvent;)V (AppenderControl.java:134)
at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(Lorg/apache/logging/log4j/core/LogEvent;)V (AppenderControl.java:125)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(Lorg/apache/logging/log4j/core/LogEvent;)V (AppenderControl.java:89)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(Lorg/apache/logging/log4j/core/LogEvent;)V (LoggerConfig.java:542)
at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(Lorg/apache/logging/log4j/core/LogEvent;Lorg/apache/logging/log4j/core/config/LoggerConfig$LoggerConfigPredicate;)V (LoggerConfig.java:500)
at org.apache.logging.log4j.core.config.LoggerConfig.log(Lorg/apache/logging/log4j/core/LogEvent;Lorg/apache/logging/log4j/core/config/LoggerConfig$LoggerConfigPredicate;)V (LoggerConfig.java:483)
at org.apache.logging.log4j.core.config.LoggerConfig.log(Ljava/lang/String;Ljava/lang/String;Ljava/lang/StackTraceElement;Lorg/apache/logging/log4j/Marker;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (LoggerConfig.java:417)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(Lorg/apache/logging/log4j/util/Supplier;Ljava/lang/String;Ljava/lang/String;Ljava/lang/StackTraceElement;Lorg/apache/logging/log4j/Marker;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (AwaitCompletionReliabilityStrategy.java:82)
at org.apache.logging.log4j.core.Logger.log(Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Ljava/lang/String;Ljava/lang/StackTraceElement;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (Logger.java:161)
at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(Ljava/lang/String;Ljava/lang/StackTraceElement;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (AbstractLogger.java:2205)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(Ljava/lang/String;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (AbstractLogger.java:2159)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(Ljava/lang/String;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Lorg/apache/logging/log4j/message/Message;Ljava/lang/Throwable;)V (AbstractLogger.java:2142)
at org.apache.logging.log4j.spi.AbstractLogger.logMessage(Ljava/lang/String;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Ljava/lang/String;Ljava/lang/Throwable;)V (AbstractLogger.java:2017)
at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(Ljava/lang/String;Lorg/apache/logging/log4j/Level;Lorg/apache/logging/log4j/Marker;Ljava/lang/String;Ljava/lang/Throwable;)V (AbstractLogger.java:1983)
at org.apache.logging.slf4j.Log4jLogger.error(Ljava/lang/String;Ljava/lang/Throwable;)V (Log4jLogger.java:319)
at reactor.util.Loggers$Slf4JLogger.error(Ljava/lang/String;Ljava/lang/Throwable;)V (Loggers.java:324)
at reactor.core.publisher.Operators.onErrorDropped(Ljava/lang/Throwable;Lreactor/util/context/Context;)V (Operators.java:673)
at reactor.core.publisher.LambdaMonoSubscriber.doError(Ljava/lang/Throwable;)V (LambdaMonoSubscriber.java:158)
at reactor.core.publisher.LambdaMonoSubscriber.onError(Ljava/lang/Throwable;)V (LambdaMonoSubscriber.java:150)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(Ljava/lang/Throwable;)V (MonoPeekTerminal.java:258)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(Ljava/lang/Throwable;)V (MonoPeekTerminal.java:258)
at reactor.core.publisher.FluxPeekFuseable$PeekFuseableConditionalSubscriber.onError(Ljava/lang/Throwable;)V (FluxPeekFuseable.java:553)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(Ljava/lang/Throwable;)V (MonoFlatMap.java:172)
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(Ljava/lang/Throwable;)V (MonoIgnoreThen.java:278)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(Ljava/lang/Throwable;)V (MonoFlatMap.java:192)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(Ljava/lang/Throwable;)V (MonoFlatMap.java:259)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(Ljava/lang/Throwable;)V (MonoPeekTerminal.java:258)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(Ljava/lang/Throwable;)V (MonoPeekTerminal.java:258)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(Ljava/lang/Throwable;)V (MonoFlatMap.java:192)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(Ljava/lang/Throwable;)V (MonoFlatMap.java:259)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(Ljava/lang/Throwable;)V (MonoFlatMap.java:172)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onError(Ljava/lang/Throwable;)V (FluxMapFuseable.java:142)
at reactor.core.publisher.Operators$MonoSubscriber.onError(Ljava/lang/Throwable;)V (Operators.java:1886)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(Ljava/lang/Throwable;)V (FluxOnErrorResume.java:106)
at reactor.core.publisher.Operators.error(Lorg/reactivestreams/Subscriber;Ljava/lang/Throwable;)V (Operators.java:198)
at reactor.core.publisher.MonoError.subscribe(Lreactor/core/CoreSubscriber;)V (MonoError.java:53)
at reactor.core.publisher.Mono.subscribe(Lorg/reactivestreams/Subscriber;)V (Mono.java:4490)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(Ljava/lang/Throwable;)V (FluxOnErrorResume.java:103)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onError(Ljava/lang/Throwable;)V (MonoFlatMapMany.java:255)
at reactor.core.publisher.FluxMap$MapSubscriber.onError(Ljava/lang/Throwable;)V (FluxMap.java:134)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(Ljava/lang/Throwable;)V (FluxOnErrorResume.java:106)
at reactor.core.publisher.Operators.error(Lorg/reactivestreams/Subscriber;Ljava/lang/Throwable;)V (Operators.java:198)
at reactor.core.publisher.MonoError.subscribe(Lreactor/core/CoreSubscriber;)V (MonoError.java:53)
at reactor.core.publisher.Mono.subscribe(Lorg/reactivestreams/Subscriber;)V (Mono.java:4490)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(Ljava/lang/Throwable;)V (FluxOnErrorResume.java:103)
at reactor.core.publisher.MonoNext$NextSubscriber.onError(Ljava/lang/Throwable;)V (MonoNext.java:93)
at reactor.core.publisher.MonoNext$NextSubscriber.onError(Ljava/lang/Throwable;)V (MonoNext.java:93)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(Ljava/lang/Throwable;)V (MonoFlatMap.java:192)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(Ljava/lang/Throwable;)V (MonoFlatMap.java:259)
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(Ljava/lang/Throwable;)V (MonoPeekTerminal.java:258)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(Ljava/lang/Throwable;)V (MonoCreate.java:201)
at com.mongodb.reactivestreams.client.internal.MongoOperationPublisher.lambda$sinkToCallback$30(Lreactor/core/publisher/MonoSink;Ljava/lang/Object;Ljava/lang/Throwable;)V (MongoOperationPublisher.java:545)
at com.mongodb.reactivestreams.client.internal.MongoOperationPublisher$$Lambda$2679.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.reactivestreams.client.internal.OperationExecutorImpl.lambda$execute$9(Lcom/mongodb/internal/binding/AsyncReadWriteBinding;Lreactor/core/publisher/MonoSink;Ljava/lang/Object;Ljava/lang/Throwable;)V (OperationExecutorImpl.java:124)
at com.mongodb.reactivestreams.client.internal.OperationExecutorImpl$$Lambda$2795.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (ErrorHandlingResultCallback.java:48)
at com.mongodb.internal.operation.CommandOperationHelper.lambda$exceptionTransformingCallback$23(Lcom/mongodb/internal/async/SingleResultCallback;Ljava/lang/Object;Ljava/lang/Throwable;)V (CommandOperationHelper.java:632)
at com.mongodb.internal.operation.CommandOperationHelper$$Lambda$2810.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.async.function.AsyncCallbackSupplier.lambda$whenComplete$1(Ljava/lang/Runnable;Lcom/mongodb/internal/async/SingleResultCallback;Ljava/lang/Object;Ljava/lang/Throwable;)V (AsyncCallbackSupplier.java:95)
at com.mongodb.internal.async.function.AsyncCallbackSupplier$$Lambda$2536.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.async.function.RetryingAsyncCallbackSupplier$RetryingCallback.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (RetryingAsyncCallbackSupplier.java:109)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (ErrorHandlingResultCallback.java:48)
at com.mongodb.internal.async.function.AsyncCallbackSupplier.lambda$whenComplete$1(Ljava/lang/Runnable;Lcom/mongodb/internal/async/SingleResultCallback;Ljava/lang/Object;Ljava/lang/Throwable;)V (AsyncCallbackSupplier.java:95)
at com.mongodb.internal.async.function.AsyncCallbackSupplier$$Lambda$2536.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (ErrorHandlingResultCallback.java:48)
at com.mongodb.internal.operation.OperationHelper.lambda$withAsyncSuppliedResource$7(ZLcom/mongodb/internal/async/SingleResultCallback;Lcom/mongodb/internal/async/function/AsyncCallbackFunction;Lcom/mongodb/internal/binding/ReferenceCounted;Ljava/lang/Throwable;)V (OperationHelper.java:649)
at com.mongodb.internal.operation.OperationHelper$$Lambda$2571.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.connection.DefaultServer$1.onResult(Lcom/mongodb/internal/connection/InternalConnection;Ljava/lang/Throwable;)V (DefaultServer.java:130)
at com.mongodb.internal.connection.DefaultServer$1.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (DefaultServer.java:122)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (ErrorHandlingResultCallback.java:48)
at com.mongodb.internal.connection.DefaultConnectionPool.lambda$getAsync$0(Lcom/mongodb/internal/async/SingleResultCallback;Lcom/mongodb/internal/connection/InternalConnection;Ljava/lang/Throwable;)V (DefaultConnectionPool.java:200)
at com.mongodb.internal.connection.DefaultConnectionPool$$Lambda$2643.onResult(Ljava/lang/Object;Ljava/lang/Throwable;)V (Unknown Source)
at com.mongodb.internal.connection.DefaultConnectionPool.lambda$getAsync$1(Lcom/mongodb/internal/async/SingleResultCallback;Lcom/mongodb/internal/Timeout;Ljava/lang/RuntimeException;)V (DefaultConnectionPool.java:211)
at com.mongodb.internal.connection.DefaultConnectionPool$$Lambda$2648.accept(Ljava/lang/Object;)V (Unknown Source)
at com.mongodb.internal.connection.DefaultConnectionPool$Task.doComplete(Ljava/util/function/Supplier;)V (DefaultConnectionPool.java:1385)
at com.mongodb.internal.connection.DefaultConnectionPool$Task.failAsTimedOut()V (DefaultConnectionPool.java:1379)
at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager.workerRun()V (DefaultConnectionPool.java:1309)
at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager$$Lambda$1041.run()V (Unknown Source)
at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager.runAndLogUncaught(Ljava/lang/Runnable;)V (DefaultConnectionPool.java:1342)
at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager.lambda$initUnlessClosed$0()V (DefaultConnectionPool.java:1275)
at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager$$Lambda$1040.run()V (Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object; (Executors.java:515)
at java.util.concurrent.FutureTask.run()V (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:628)
at java.lang.Thread.run()V (Thread.java:829)
Can you help me please?
Thanks
Comment From: wilkinsona
Can you please explain why you have opened a Spring Boot issue for this? Other than it providing the dependency versions, there doesn't appear to be any Spring code involved. It's also not clear to me what is leaking. The stack trace is pointing to this line of code in Log4j2:
if (immediateFlush && byteBuffer.position() == 0) {
It does not appear to be a likely candidate for the cause of a memory leak.
Comment From: spring-projects-issues
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
Comment From: jdelobel
Hi, the issue is on the mongo java async driver side. Please check this https://www.mongodb.com/community/forums/t/out-of-memory-java-heap-space-with-mongodb-driver-core-4-6-1-with-reactivemongotemplate/231496/5.
Comment From: wilkinsona
Thanks for the link. This will have to be addressed in MongoDB. We'll pick up any fix as part of our usual dependency upgrade process.