Following the work that was done on https://github.com/redis/redis/pull/11199, we believe we can refactor the code that decide when we need to perform command propagation and reduce the amount of conditions and variable that currently involve in the processes.
We believe that a single nesting_level variable can be good enough. Each time an execution unit starts, we increase the nesting_level (and decrease it when the execution unit end). When nesting_level gets to zero, we can perform post notifications jobs and command propagation.
Comment From: JimB123
Agreed.
server.core_propagation sort-of means - we are processing a command and in a transaction.
server.in_nested_call means the call() function has invoked a command processor. If >=1 this means that a command is in progress. When >1, we know that call() is has been invoked recursively.
While addressing this, I would also suggest an additional assertion be added to processEventsWhileBlocked(). Currently, this code expects that we are not within the context of a command - unless it's a LUA script or a module yield, in which case only CMD_ALLOW_BUSY commands are permitted. I'd suggest making that explicit with this assertion:
void processEventsWhileBlocked(void) {
serverAssert(server.core_propagates == 0 || isInsideYieldingLongCommand());
...
Comment From: JimB123
BTW - I tried that assertion above. It trips the test cases for DEBUG loading of AOF. I think this means that you can currently cause an engine crash by performing a DEBUG LOADAOF while at maxmemory. A recursive event loop is created. The entire recursive event loop will be transactionized. Eviction will assert when the 2nd (write) command is executed in the recursive call.