Is there any way to set cpu affinity to both redis server and bgsave processes? I'd like to have them run on different cpus.

Thanks in advance, Adriano

Comment From: jwilm

I've come here to make this same suggestion. Redis needs to have configuration options for scheduler affinity for both the main process and the bgsave process, and it needs to set the bgsave process affinity before begging the save operation.

This is becoming increasingly important with CPU frequency scaling and growing number of cores within a given system. If the Redis process continually switches cores, the scaling drivers may not detect high load for long enough and keep the CPU frequency at a low value. The result is higher latencies and a lower ops ceiling. This issue is directly addressed by setting the process affinity, but unfortunately bgsave would compete with the main process in the current system.

Thanks for your consideration.

Comment From: cespare

We investigated this issue in early 2018 and it might be useful to describe what we found.

It's of course easy to run redis with whatever affinity settings you want. In our case we found that we got the best performance by pinning each redis process to a single isolated core (using isolcpus). (We also disable irqbalance and spread the interrupt affinity for the NIC's RX queues intentionally as well, taking into account where the redis processes are and the NUMA architecture of the machine.)

The problem you run into with this is that the bgsave process is also pinned to the dedicated redis core. We need it to be on a different, nearby core. The solution we came up with is a small patch which adds a bgsave-cpu-mask configuration option. Then in the config file we can write something like

bgsave-cpu-mask 0000001f00000001

and the bgsave process is run with a different affinity.

This has been working very well for us for the past 20 months. However, as my commit message notes, the patch is a quick'n'dirty workaround that isn't ready to be submitted upstream. It's Linux-specific and has no tests nor documentation. I also don't even know if @antirez wants an option like this.

Comment From: itamarhaber

FWIW - I believe this is a good point (i.e. bgsave binding) as I know the creator has been contemplating on adding some defenses/facilitators for using deploying multiple Redis instances on a multi-core machine.

Comment From: pizhenwei

@cespare @adriano-di-giovanni @jwilm @itamarhaber

Hey, guys, I pulled a commit to supports CPU affinity setting for redis(include IO threads/bio thread/bgsave child process and aof rewrite child process). This issue seems to be resolved, what do you think about this?

Comment From: itamarhaber

Yep, seems like this one can be closed - thanks for the PR btw ;)

Comment From: itamarhaber

Closing as per the previous discussions above.