For some reason redis causes a huge server CPU load even though it's idle and I am not sure what is going on.
I have 3 redis instances (1 TCP which is the "master" and 2 slaves that are socket only) on the server. They all have essentially the same config (except slaveof). Even when there is no load at all (= no application using redis at all, bc I turned them all of), the load avg is >100 (!!) on a 4+4 (HT) core CPU
Monitor command shows no activity, despite a few PING
Info: (of 1 slave) redis1-info.log
Screenshot of "top":
Could you please give me a hint on how I could debug this further? Do you perhaps have any idea why I am having this issue?
Comment From: antirez
Hello, this is normal if Redis is persisting on disk (but it is a different process, the one serving the clients is not affected), or when the replicas connect to a master for the first synchronization. This is very likely the case, could you confirm? There are no other known reasons for this. Could you confirm? With INFO you can check if the instances are creating an RDB file for the replicas.
Comment From: kkmuffme
this is normal if Redis is persisting on disk (but it is a different process, the one serving the clients is not affected)
Only redis master is persisting on disk, the slaves are not (appendonly no, save is commented) But the high CPU load is for all 3 instances
or when the replicas connect to a master for the first synchronization. This is very likely the case, could you confirm?
The high CPU load persists even after 1/multiple days.
With INFO you can check if the instances are creating an RDB file for the replicas.
Yes, the master does. Here's the info for the master: (somehow github file upload isn't working today, so pasted it here): https://justpaste.it/2nq0f
Comment From: antirez
Make sure to check "CONFIG GET *" output of the slave that is consuming such CPU.
I saw the original log file you attached in the issue, the one of the slave, and if you check the filed used_cpu_sys_children, it is non zero, so it means that for some reason this instance is also using fork() to persist or write the AOF file or something like that. So this does not look like a Redis bug, please could you repost all this in the Redis mailing list? There there are people that can guide you to find the problem.
Comment From: kkmuffme
@antirez opened an issue there, but it died.
Anyway, I straced it (which didn't help, due to a EAGAIN bug reported on someone else's issue already)
However the solution was to flush redis, then the CPU load goes away for some time. Anything else I can do to investigate this issue?
The rdb file was just 410MB, so not much data in redis either.
Comment From: kkmuffme
Solution: use AOF and not RDB
RDB even when set to "save 900 1" and all other "save" commented will spawn child processes every few mins instead of only once every 15 minutes It's these child processes that cause high CPU load