Redis high cpu usage with multi-io-threads, is there a better way to improve here?

I think it should have something to do with the CAS that the io thread does when the main thread is assigned to the io thread task, with the following code:

// the entry of io threads
void *IOThreadMain(void *myid) {
    .......
    while(1) {
        /* Wait for start */
        for (int j = 0; j < 1000000; j++) {
            if (getIOPendingCount(id) != 0) break;
        }

but there doesn't seem to be a more efficient way to communicate between threads than CAS!

Originally posted by @GuoZhaoran in https://github.com/redis/redis/issues/9461#issuecomment-917850698

Comment From: yoav-steinberg

This is by design. The current io-threading mechanism assume high load, otherwise you wouldn't really need io-theading. In such scenarios high CPU usage is a given anyway so we won't waste much on the above busy loop.

I know these assumptions are both wasteful and limited. One approach would be have both a locking queue implementation and the existing non-locking queue. We'll default to the locking queue but if we detect high load move to the other implementation (and vice versa). If you want you can attempt a PR for this.

I'm closing the issue as this is a known limitation and the best approach would probably be a complete re-write of the io-threading subsystem which is a long term goal anyway..

Comment From: GuoZhaoran

Hi, @yoav-steinberg. Thank you for your answer, it does sound very plausible.

The method you provided works for me, I will try to make a pr for this, my idea is to set a threshold to mark whether it is high load or not, if it is high load then it is to use a lock-free queue, otherwise use a lock queue, the method to determine whether it is high load is similar to the LFU elimination policy.

Very much looking forward to redis7 rewrite io thread system, I should be able to learn more from it!