@grobe0ba @antirez @madolson redis6 enable multi-io-thread,but not worked all still work on main thread,that may be a bug - redis-version

$ ./redis-server  --version
Redis server v=6.0.9 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=c67efd74f306f551
  • redis config
$ cat redis.conf  |grep -v '#'
aof-use-rdb-preamble yes
bind 127.0.0.1
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile ""
databases 16
always-show-logo yes

save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir ./

replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
io-threads-do-reads yes
io-threads 4
oom-score-adj no
oom-score-adj-values 0 200 800
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
  • debug info
Breakpoint 11, readQueryFromClient (conn=0x2aaaabc150c0) at networking.c:1957
1957        client *c = connGetPrivateData(conn);
(gdb) n
1963        if (postponeClientRead(c)) return;
(gdb) 

// we already enable multi-thread,but server.io_threads_actiove is zero always,that may be a bug
Breakpoint 12, postponeClientRead (c=0x2aaaabd4d0c0) at networking.c:3232
3232        if (server.io_threads_active &&
(gdb) p server.io_threads_active
$8 = 0
(gdb) p server.io_threads_active
$9 = 0
(gdb) p server.io_threads_do_reads
$10 = 1
(gdb) p ProcessingEventsWhileBlocked
$11 = 0
(gdb) p c->flags & (CLIENT_MASTER|CLIENT_SLAVE|CLIENT_PENDING_READ)
$12 = 0
(gdb) quit

Comment From: madolson

Hey, just because IO threads are enabled, does not mean they are running and processing events. IO threads become active when there is enough requests to justify the effort of using multiple threads. So under low throughput scenarios, the IO threads won't kick on.

Comment From: perrynzhou

Hey, just because IO threads are enabled, does not mean they are running and processing events. IO threads become active when there is enough requests to justify the effort of using multiple threads. So under low throughput scenarios, the IO threads won't kick on.

@madolson is any way to debug multi-thread with less requests?

Comment From: madolson

IO threads are less performant under low request situations so they aren't used. The best way to test it is to send more requests.

Comment From: jiekun

I assume @perrynzhou is trying to play with I/O thread, not forcing I/O thread to be used under low workload in production env.

so, it's controlled in handleClientsWithPendingWritesUsingThreads()

int handleClientsWithPendingWritesUsingThreads(void) {
    ...
    # HERE, you have to make "stopThreadedIOIfNeeded" return 0.
    if (server.io_threads_num == 1 || stopThreadedIOIfNeeded()) {
        return handleClientsWithPendingWrites();
    }

    /* Start threads if needed. */
    ...

So in stopThreadedIOIfNeeded():

int stopThreadedIOIfNeeded(void) {
    return 0  # HERE, you can try adding this to force a use of Threaded IO

    # origin logics below: 
    ...
}

I tried debugging with my CLion, set a break point first, then send multiple request via different client, then continue. This will use the I/O thread to process read/writes. Not sure if you can have a try or not. :)

https://jiekun.dev/posts/redis_tio_implementation/#limitation

Comment From: madolson

Sure, that is a way to debug it. Since the original post was about about a bug, I'm going to resolve this unless there is any other concern to be raised.

Comment From: perrynzhou

I assume @perrynzhou is trying to play with I/O thread, not forcing I/O thread to be used under low workload in production env.

so, it's controlled in handleClientsWithPendingWritesUsingThreads()

``` int handleClientsWithPendingWritesUsingThreads(void) { ... # HERE, you have to make "stopThreadedIOIfNeeded" return 0. if (server.io_threads_num == 1 || stopThreadedIOIfNeeded()) { return handleClientsWithPendingWrites(); }

/* Start threads if needed. */
...

```

So in stopThreadedIOIfNeeded():

``` int stopThreadedIOIfNeeded(void) { return 0 # HERE, you can try adding this to force a use of Threaded IO

# origin logics below: 
...

} ```

I tried debugging with my CLion, set a break point first, then send multiple request via different client, then continue. This will use the I/O thread to process read/writes. Not sure if you can have a try or not. :)

https://jiekun.dev/posts/redis_tio_implementation/#limitation

@2014BDuck think you,may i join your webchat?my webchat is 13632930825