I found that pipelined WAIT command have performance issue when trying to get better throughput by pipeline.
reproduce:
import redis
import datetime
r = redis.Redis()
with r.pipeline(transaction=False) as pipe:
while True:
for i in range(0, 10):
pipe.set("foo", "bar").wait(1, 0)
res = pipe.execute()
print datetime.datetime.now()
result
$ python sync-bench.py
2019-11-25 14:52:01.980280
2019-11-25 14:52:02.738710
2019-11-25 14:52:03.586276
2019-11-25 14:52:04.289024
2019-11-25 14:52:04.993107
2019-11-25 14:52:05.748331
2019-11-25 14:52:06.602541
2019-11-25 14:52:07.305483
2019-11-25 14:52:08.009707
2019-11-25 14:52:08.762686
2019-11-25 14:52:09.616872
2019-11-25 14:52:10.319952
2019-11-25 14:52:11.022646
2019-11-25 14:52:11.770330
2019-11-25 14:52:12.628073
2019-11-25 14:52:13.332752
2019-11-25 14:52:14.033769
2019-11-25 14:52:14.778328
2019-11-25 14:52:15.640083
2019-11-25 14:52:16.342727
2019-11-25 14:52:17.046337
2019-11-25 14:52:17.789024
root cause:
If client gets blocked again in processUnblockedClients, redis-sever will not send REPLCONF GETACK * to slaves untill next eventloop, so redis will poll for 100ms by default(10hz) if no other file event triggered.
so, pipelined WAIT command would be blocked for 100ms (10hz) in the worst case.
fix:
fix is simple, just move server.get_ack_from_slaves sinppet after processUnblockedClients, so that both the first WAIT command that puts client in CLIENT_BLOCKED context and the following WAIT command that processed in processUnblockedClients will trigger redis-sever to send REPLCONF GETACK * , so that the eventloop will get REPLCONG ACK <reploffset> from slave ASAP.
Comment From: patpatbear
see PR #6623