When the client request accumulates a certain number of times will appear this error, the general server application will run for about 15-30 days will appear this phenomenon, and the favicon .ico file can not be obtained, this error is basically about the favicon .ico file, restart the springboot application to restore normal, and can also obtain the favicon .ico normally, how to fix this problem?
The logs is Spring-boot 3.1.2 version
2023-09-12 17:12:06.233 [reactor-http-epoll-37] ERROR o.s.web.server.adapter.HttpWebHandlerAdapter - [9096f432-2756942] Error [java.nio.file.FileSystemException: /usr/local/webapps/BOOT-INF/classes/static/favicon.ico: Too many open files] for HTTP GET "/favicon.ico", but ServerHttpResponse already committed (200 OK)
2023-09-12 17:12:06.233 [reactor-http-epoll-37] ERROR reactor.netty.http.server.HttpServerOperations - [9096f432-2, L:/172.17.0.3:443 - R:/116.30.101.28:50220] Error finishing response. Closing connection
java.nio.file.FileSystemException: /usr/local/webapps/BOOT-INF/classes/static/favicon.ico: Too many open files
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ org.springframework.web.cors.reactive.CorsWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ com.ct.salesstatistics.provider.JwtTokenAuthenticationFilter$$SpringCGLIB$$0 [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ com.ct.salesstatistics.provider.JwtTokenAuthenticationFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
*__checkpoint ⇢ HTTP GET "/favicon.ico" [ExceptionHandlingWebHandler]
Original Stack Trace:
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(Unknown Source)
at java.base/java.nio.channels.FileChannel.open(Unknown Source)
at java.base/java.nio.channels.FileChannel.open(Unknown Source)
at reactor.netty.NettyOutbound.lambda$sendFile$0(NettyOutbound.java:169)
at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:75)
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:240)
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:203)
at reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:196)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:52)
at io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:31)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:728)
at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:283)
at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:363)
at io.netty.channel.epoll.AbstractEpollChannel.doWriteBytes(AbstractEpollChannel.java:364)
at io.netty.channel.epoll.AbstractEpollStreamChannel.writeBytes(AbstractEpollStreamChannel.java:260)
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWriteSingle(AbstractEpollStreamChannel.java:471)
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWrite(AbstractEpollStreamChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:931)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.flush0(AbstractEpollChannel.java:557)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:895)
at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1372)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:921)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:907)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:893)
at io.netty.handler.ssl.SslHandler.forceFlush(SslHandler.java:2148)
at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:804)
at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:781)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:925)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:907)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:893)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:531)
at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125)
at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:356)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:923)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:907)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:893)
at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:127)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:923)
at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:941)
at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1247)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:403)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Unknown Source)
Comment From: bclozel
Thanks for getting in touch, but it feels like this is a question that would be better suited to StackOverflow. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements. Feel free to update this issue with a link to the re-posted question (so that other people can find it) or add some more details if you feel this is a genuine bug.
In this case, this merely means that your OS has too many open files. It could be that the OS configuration needs to be changed, or that some process is not properly closing files and leaking file handles (so, not necessarily a Java process). I would suggest looking at the opened files at runtime regularly to find out if some files are not being closed properly.
Comment From: jacksonjim
这是来自QQ邮箱的假期自动回复邮件。 您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。
Comment From: jacksonjim
Thank you for your reply, the configuration of the CentOS operating system has been configured on too manay open files, At present, this situation occurs as long as it is about the file of the favicon .ico, this file is a static file, other HTTP service API interface requests are normal return data, I don't know why this is the case, so you need to provide the full log or there are other ways, currently spring-boot applications run docker containers, the same container of spring-boot applications other http API interfaces can respond to data to the client, only the favicon .ico appears, about whether the built-in netty of spring-boot can be configured to disable the favicon .ico or about connection request timeout control for long connections, closing it may free up related resources
Comment From: jacksonjim
Thanks for getting in touch, but it feels like this is a question that would be better suited to StackOverflow. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements. Feel free to update this issue with a link to the re-posted question (so that other people can find it) or add some more details if you feel this is a genuine bug.
In this case, this merely means that your OS has too many open files. It could be that the OS configuration needs to be changed, or that some process is not properly closing files and leaking file handles (so, not necessarily a Java process). I would suggest looking at the opened files at runtime regularly to find out if some files are not being closed properly.
Abort the docker for spring-boot appliction config OS Too many open files
root@9c0c64a43b0f:/usr/local/webapps# ulimit -a
real-time non-blocking time (microseconds, -R) unlimited
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 254822
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1048576
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
root@9c0c64a43b0f:/usr/local/webapps# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 1.2 2.3 28527044 1563068 pts/0 Ssl+ 08:37 6:27 java org.springframework.boot.loader.JarLauncher
root 875 0.0 0.0 7616 2592 pts/1 Ss 16:51 0:00 /bin/bash
root 888 0.0 0.0 10052 1700 pts/1 R+ 16:55 0:00 ps -aux
root@9c0c64a43b0f:/usr/local/webapps# cat /proc/1/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size unlimited unlimited bytes
Max resident set unlimited unlimited bytes
Max processes unlimited unlimited processes
Max open files 1048576 1048576 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 254822 254822 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Comment From: bclozel
@jacksonjim please don't document your findings here; we assume so far that this is an application or OS-specific problem. We can reopen this issue if it turns out that Spring Boot is responsible for not releasing files properly.