Spring Boot Version

spring-boot-starter-webflux:2.1.14.RELEASE spring-boot-starter-reactor-netty:2.1.14.RELEASE reactor-netty:0.8.18.RELEASE jdk 1.8.0_162

Problem

Hi, I am using jmeter to test the performance of a microservice, the test case is: + Running the application with -Dio.netty.leakDetectionLevel=paranoid . + Jmeter uses 50 threads to start performance testing. + Force stop jmeter in the testing. + Start the performance testing again.

An exception occurs: io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected. When I remove the subscribeOn in the code, no exception occurs.

Code

MyApplication

@SpringBootApplication
@EnableWebFlux
@Configuration
public class MyApplication {
    public static void main(String[] args) {
        SpringApplication app = new SpringApplication(MyApplication.class);
        app.run(args);
    }

    @Bean
    public Scheduler requestScheduler() {
        ExecutorService executorService =
            new ThreadPoolExecutor(1024, 1024, 60, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
        return Schedulers.fromExecutorService(executorService);
    }
}

MyController

@RestController
public class MyController {

    @Autowired
    @Qualifier("requestScheduler")
    private Scheduler requestScheduler;

    @RequestMapping(value = "/test")
    public Mono<ResponseEntity<String>> process(@RequestBody String reqMsg, @RequestHeader HttpHeaders reqHttpHeaders,
        ServerHttpRequest serverHttpRequest) {
        return Mono.just(reqMsg).subscribeOn(requestScheduler).onErrorResume(e -> {
            return Mono.just(e.getMessage());
        }).flatMap(responseMsg -> {
            return Mono.just(new ResponseEntity<>(responseMsg, HttpStatus.OK));
        });
    }
}

Exception stack

2020-06-09T15:57:00.976+0800|ERROR|pool-2-thread-1021|320|io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
    io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:363)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
    io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
    org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71)
    org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
    org.springframework.core.codec.CharSequenceEncoder.lambda$encode$1(CharSequenceEncoder.java:85)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
    reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94)
    reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1993)
    reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1867)
    reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58)
    reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
    reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
    reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
    reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
    reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
    reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
    reactor.core.publisher.FluxSubscribeOnValue$ScheduledScalar.run(FluxSubscribeOnValue.java:178)
    reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:50)
    reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:27)
    java.util.concurrent.FutureTask.run(FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

Comment From: Pratikjoy

Hello @xuep2006 , Can you again run the code with "-Dio.netty.leakDetectionLevel=PARANOID" ? and let me know if the error still occurs..

Comment From: xuep2006

Hello @Pratikjoy , Thank you for your prompt reply. I use -Dio.netty.leakDetectionLevel=PARANOID to run the code and the error still occurs.

JVM Command line

-Dio.netty.leakDetectionLevel=PARANOID -Dfile.encoding=UTF-8

Log

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::       (v2.1.14.RELEASE)

16:38:04.506|INFO|main|50|o.e.webflux.netty.test.MyApplication:Starting MyApplication on DESKTOP-L2S27F0 with PID 48284 
16:38:04.510|INFO|main|652|o.e.webflux.netty.test.MyApplication:The following profiles are active: dev
16:38:07.655|INFO|main|82|o.s.b.w.e.netty.NettyWebServer:Netty started on port(s): 8090
16:38:07.658|INFO|main|59|o.e.webflux.netty.test.MyApplication:Started MyApplication in 3.709 seconds (JVM running for 4.355)
16:38:45.879|ERROR|reactor-http-nio-1|289|o.s.w.s.a.HttpWebHandlerAdapter:[0ce04e8a] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)
16:38:45.882|ERROR|reactor-http-nio-1|289|o.s.w.s.a.HttpWebHandlerAdapter:[1e2ce5d0] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)
16:38:50.299|ERROR|pool-3-thread-988|320|io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
    io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:363)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
    io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
    org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71)
    org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
    org.springframework.core.codec.CharSequenceEncoder.lambda$encode$1(CharSequenceEncoder.java:85)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
    reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
    reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94)
    reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1993)
    reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1867)
    reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114)
    reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
    reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
    reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
    reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58)
    reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
    reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
    reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
    reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
    reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
    reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:178)
    reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:50)
    reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:27)
    java.util.concurrent.FutureTask.run(FutureTask.java:266)
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)
2020-06-10T16:38:51.650+0800|ERROR|reactor-http-nio-7|289|o.s.w.s.a.HttpWebHandlerAdapter:[de66a20c] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)

Comment From: DarrMirr

@xuep2006 Hi!

Issue easily reproduced at spring-boot-starter-webflux:2.1.14.RELEASE. But I could not reproduce it at spring-boot-starter-webflux:2.4.2. Error message "LEAK: ByteBuf.release() was not called before it's garbage-collected." is not appeared at log output.

Are you able to update webflux version from 2.1.14.RELEASE to 2.4.2 and try to reproduce issue again?

Comment From: rstoyanchev

Yes this might have been addressed with recent fixes related to #26232.

Comment From: spring-projects-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-projects-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Comment From: rchalicham

Hi get same error I changed webflux to 2.45 1 actionable task: 1 executed +--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 | --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE +--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 (n) +--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 | --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE +--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 | --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE +--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 | --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE

the error i get is :

[ntLoopGroup-2-2] io.netty.util.ResourceLeakDetector : LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information. Recent access records:

1:

io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:291) io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

2:

io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670) io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793) io.netty.handler.codec.http.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:592) io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:218) io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

3:

io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670) io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793) io.netty.handler.codec.http.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:572) io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:218) io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

4:

io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670) io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793) io.netty.handler.codec.http.HttpObjectDecoder$LineParser.parse(HttpObjectDecoder.java:842) io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:199) io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

5:

Hint: 'reactor.left.httpCodec' will handle the message from this point. io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

6:

io.netty.buffer.AdvancedLeakAwareByteBuf.internalNioBuffer(AdvancedLeakAwareByteBuf.java:736) io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1485) io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:165) io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:283) io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329) io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224) io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271) io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:748)

7:

io.netty.buffer.AdvancedLeakAwareByteBuf.nioBufferCount(AdvancedLeakAwareByteBuf.java:706)
io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1485)
io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:165)
io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:283)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
Created at:
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:2138)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1324)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748): 12 leak records were discarded because the leak record count is targeted to 4. Use system property io.netty.leakDetection.targetRecords to increase the limit.

'org.springframework.boot' version '2.1.9.RELEASE' ext { set('springCloudVersion', 'Greenwich.SR3') set('springCloudServices', '2.1.4.RELEASE') }

appreciate your help

Comment From: DarrMirr

@rchalicham hello!

org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE

According to comment this issue fixed at 5.3.2. It looks like 5.1.10.RELEASE version is used instead of 5.3.3 one. I recommend to manage all spring modules versions only by 'org.springframework.boot' one. And plug any spring-boot starters without setting version.

I think error disappear if you set 'org.springframework.boot' version to 2.4.2 or higher.

Comment From: AtticusLv

Faced with the same issue, I use Spring Boot with 2.2.10.RELEASE version. What's the latest version of 2.2.x version that can be used to fix LEAK issue? @DarrMirr @rstoyanchev Thank you very much!

Comment From: DarrMirr

Faced with the same issue, I use Spring Boot with 2.2.10.RELEASE version. What's the latest version of 2.2.x version that can be used to fix LEAK issue? @DarrMirr @rstoyanchev Thank you very much!

@AtticusLv according to comment fix was backported to 5.2.x. on December 2020. Spring boot 2.2.13.RELEASE has released on January 2021. Therefore I think fix should be included to 2.2.13.RELEASE.

Spring Spring WebFlux "ResourceLeakDetector:LEAK"

Spring Boot Version

Problem

Code

Exception stack

JVM Command line

Log

1:

2:

3:

4:

5:

6:

7: