Background: Today, keyspace notifications for client connections are sent on two pubsub messages, in one the operation name (e.g. del) is used in the channel name, and the key name in the argument, and in the other, the key name is the channel name and the operation name in the argument. Additionally, there's a global config for enabling either of these messages, and an option to enable it only on some types and not all. Besides that, modules receive a single callback with both argument.

The problems with that are: 1. if a client app is interested in all operation on a certain type, but not on other types, it has no way to subscribe just for them, it can subscribe for sadd events, but if it'll subscribe for del he'll get events on all keys, and it can't even know their type by looking at the message. 2. #8759 adds a type for modules, but the fact some deployments have more than one module, or some modules more than one data type, so users still want to either register for notifications of just one type, or at least know the type when getting the notification.

suggested design: 1. Module APIs for RM_NotifyKeyspaceEvent and the callback for RM_SubscribeToKeyspaceEvents should in some way get an additional argument specifying the data type name (string). This will probably just be an additional new API, deprecating the old one. 2. All keyspace notification will publish a 3rd message to a new channel type which includes the data type name (e.g. __keyevent@0@<typename>__:del and __keyspace@0@<typename>__:foo) 3. Maybe in some way the other notifications (i.e. the existing non-type-specific publish messages could carry the type name in the message, or as a RESP3 attribute. 4. Modify all the code (many lines in redis), to provide the type name when calling notifyKeyspaceEvent()

Comment From: soloestoy

Just copy from the original PR.

How about extend the message content when the notify is REDISMODULE_NOTIFY_MODULE type.

The format of pushed messages is an array reply with three elements: * message - a fixed string * channel - the channel's name * content - the real message

And about the content, it could be other RESP type not only string, just like an array with module name, data type, event etc.

For example, in client side tracking, the content is an array with one string element:

$7
message
$20
__redis__:invalidate
*1
$1
a

And the code:

void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
    ...
    } else if (using_redirection && c->flags & CLIENT_PUBSUB) {
        /* We use a static object to speedup things, however we assume
         * that addReplyPubsubMessage() will not take a reference. */
        addReplyPubsubMessage(c,TrackingChannelName,NULL);
    ...
        addReplyArrayLen(c,1);
        addReplyBulkCBuffer(c,keyname,keylen);

Comment From: itamarhaber

Sounds good. My only concern is the overhead of generating two additional messages (i.e. doubling the current). A naive local (Darwin) benchmark made by calling pubsubPublishMessage twice more in notifyKeyspaceEvent with no additional compute.

Pre-change:

❯ redis-benchmark -t set -n 1000000
====== SET ======
  1000000 requests completed in 8.24 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1
  host configuration "save":
  host configuration "appendonly": no
  multi-thread: no

Latency by percentile distribution:
0.000% <= 0.087 milliseconds (cumulative count 1)
50.000% <= 0.215 milliseconds (cumulative count 565261)
75.000% <= 0.239 milliseconds (cumulative count 771329)
87.500% <= 0.295 milliseconds (cumulative count 877816)
93.750% <= 0.391 milliseconds (cumulative count 941183)
96.875% <= 0.415 milliseconds (cumulative count 974018)
98.438% <= 0.431 milliseconds (cumulative count 985653)
99.219% <= 0.471 milliseconds (cumulative count 993033)
99.609% <= 0.511 milliseconds (cumulative count 996532)
99.805% <= 0.543 milliseconds (cumulative count 998088)
99.902% <= 0.591 milliseconds (cumulative count 999124)
99.951% <= 0.631 milliseconds (cumulative count 999557)
99.976% <= 0.671 milliseconds (cumulative count 999775)
99.988% <= 0.711 milliseconds (cumulative count 999888)
99.994% <= 0.767 milliseconds (cumulative count 999942)
99.997% <= 0.815 milliseconds (cumulative count 999970)
99.998% <= 0.847 milliseconds (cumulative count 999986)
99.999% <= 0.975 milliseconds (cumulative count 999993)
100.000% <= 1.071 milliseconds (cumulative count 999997)
100.000% <= 1.111 milliseconds (cumulative count 999999)
100.000% <= 1.119 milliseconds (cumulative count 1000000)
100.000% <= 1.119 milliseconds (cumulative count 1000000)

Cumulative distribution of latencies:
0.000% <= 0.103 milliseconds (cumulative count 3)
23.982% <= 0.207 milliseconds (cumulative count 239824)
88.489% <= 0.303 milliseconds (cumulative count 884886)
96.342% <= 0.407 milliseconds (cumulative count 963424)
99.603% <= 0.503 milliseconds (cumulative count 996027)
99.931% <= 0.607 milliseconds (cumulative count 999315)
99.987% <= 0.703 milliseconds (cumulative count 999873)
99.997% <= 0.807 milliseconds (cumulative count 999966)
99.999% <= 0.903 milliseconds (cumulative count 999990)
99.999% <= 1.007 milliseconds (cumulative count 999995)
100.000% <= 1.103 milliseconds (cumulative count 999998)
100.000% <= 1.207 milliseconds (cumulative count 1000000)

Summary:
  throughput summary: 121359.23 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        0.238     0.080     0.215     0.399     0.455     1.119

Post-change

❯ redis-benchmark -t set -n 1000000
====== SET ======
  1000000 requests completed in 8.30 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1
  host configuration "save":
  host configuration "appendonly": no
  multi-thread: no

Latency by percentile distribution:
0.000% <= 0.079 milliseconds (cumulative count 1)
50.000% <= 0.223 milliseconds (cumulative count 637215)
75.000% <= 0.247 milliseconds (cumulative count 761134)
87.500% <= 0.319 milliseconds (cumulative count 879180)
93.750% <= 0.399 milliseconds (cumulative count 947217)
96.875% <= 0.415 milliseconds (cumulative count 973099)
98.438% <= 0.431 milliseconds (cumulative count 985017)
99.219% <= 0.471 milliseconds (cumulative count 992844)
99.609% <= 0.511 milliseconds (cumulative count 996197)
99.805% <= 0.567 milliseconds (cumulative count 998240)
99.902% <= 0.615 milliseconds (cumulative count 999090)
99.951% <= 0.655 milliseconds (cumulative count 999515)
99.976% <= 0.695 milliseconds (cumulative count 999773)
99.988% <= 0.735 milliseconds (cumulative count 999898)
99.994% <= 0.759 milliseconds (cumulative count 999942)
99.997% <= 0.783 milliseconds (cumulative count 999971)
99.998% <= 0.807 milliseconds (cumulative count 999988)
99.999% <= 0.823 milliseconds (cumulative count 999993)
100.000% <= 0.839 milliseconds (cumulative count 999997)
100.000% <= 0.855 milliseconds (cumulative count 1000000)
100.000% <= 0.855 milliseconds (cumulative count 1000000)

Cumulative distribution of latencies:
0.001% <= 0.103 milliseconds (cumulative count 5)
17.863% <= 0.207 milliseconds (cumulative count 178628)
86.466% <= 0.303 milliseconds (cumulative count 864662)
96.116% <= 0.407 milliseconds (cumulative count 961156)
99.574% <= 0.503 milliseconds (cumulative count 995735)
99.897% <= 0.607 milliseconds (cumulative count 998974)
99.980% <= 0.703 milliseconds (cumulative count 999801)
99.999% <= 0.807 milliseconds (cumulative count 999988)
100.000% <= 0.903 milliseconds (cumulative count 1000000)

Summary:
  throughput summary: 120554.55 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        0.244     0.072     0.223     0.407     0.455     0.855

Comment From: sundb

@soloestoy @oranagra If use RESP3, what will we do with the RESP2 client? Client side tracking does not support RESP3, and it is not practical to implement redirection with keyspace notification.