A serious redesign of the threaded IO is needed. Most importantly in order to support TLS so that the encryption work can be offloaded to the treads. Possibly to assign clients for threads rather than the current fan-out approach. TBD.
Comment From: zuiderkwast
Are there any ideas about this already or is the area free to explore?
Should we have one event loop per IO thread or is it enough to pin each client to a thread? An OpenSSL connection can't read in one thread and write in another at the same time, but it seems fine to move it around between threads when it's idle, with locks.
Comment From: yossigo
@zuiderkwast In the past, we had some discussions and ideas around this, but nothing too solid.
We considered IO threads that have their own event loop and a set of pinned clients (which could move around but would require locking and coordination). The biggest challenge with this approach is accessing clients in a thread-safe and efficient manner. There are probably others as well, like AOF and the reply barrier implementation, to name one.
Comment From: soloestoy
We considered IO threads that have their own event loop and a set of pinned clients (which could move around but would require locking and coordination).
Our redis-like (named tair) project has already implement it many years ago (event before 6.0), and I discussed with antirez about this idea, but he thought it's too complex at that time, seems we can re-discuss the new IO mode now :).
Comment From: zuiderkwast
@soloestoy Is there any chance you will upstream it?
If I were to implement it, it seems easier to keep the existing event loop, pin each client to a thread and then allow TLS to run in the IO threads. This avoids the problems Yossi mentioned.
Comment From: soloestoy
@soloestoy Is there any chance you will upstream it?
If I were to implement it, it seems easier to keep the existing event loop, pin each client to a thread and then allow TLS to run in the IO threads. This avoids the problems Yossi mentioned.
I'd like to, but it's a very huge work, I think it's better to discuss the POC firstly, and if we all think it is a good way I can start upstream and merge it finally.
Comment From: zuiderkwast
OK, so how do we continue this? @soloestoy will you discuss it in a core team meeting or can you describe the high level design here?
If it is a very huge work, I think we need to think about how to do it incrementally.
Comment From: zuiderkwast
Related discussion: #7372
Comment From: soloestoy
OK, so how do we continue this? @soloestoy will you discuss it in a core team meeting or can you describe the high level design here?
If it is a very huge work, I think we need to think about how to do it incrementally.
@zuiderkwast Sorry for the late reply. I will open an issue to describe our implementation in the HLD in the near future.
Comment From: ShooterIT
Resolved by https://github.com/redis/redis/pull/13695