I'd like to implement a new command, that uses a RegEx for key matching instead of the Pattern Matching of PSUBSCRIBE.

There are multiple options how to do it and I would like to get your suggestions what's the best way to implement this feature.

1. Compiler Option Using an external library like https://github.com/intel/hyperscan for regex matching and allowing the user to integrate this feature using a compiler flag.

@leeyiw described this option in the following pull request: https://github.com/antirez/redis/pull/4721

2. Extend Module API

To be able to add a new redis command RSUBSCRIBE using redis the API should allow to add other matching methods.

Please let me know if one of the options is a way you would do it, or if there's a better way.

Comment From: leeyiw

Hyperscan regex compilation takes really a very looong time! So you should consider how to compile the database in case of block the main thread.

Comment From: sgbo

@leeyiw I don't know if I understand what you mean. When Client calls RSUBSCRIBE I would probably use the blocking API https://redis.io/topics/modules-blocking-ops to notify the client when the regex compilation is finished. The regex compilation could than run in a thread and would not block the main thread.

Comment From: leeyiw

@SGbo Yes, build database in separate thread is a good idea, but you should concern about the number of threads when there is a lot of clients running command like RSUBSCRIBE, perhaps you could merge some of them in one single thread.

Comment From: sgbo

I started implementing rsubscribe in my own repos as branch: https://github.com/SGbo/redis/tree/rsubscribe feel free to contribute / suggest things. For now I'm using pcre2 for regex parsing, because hyperscan does not run on ARM. The compilation of the regex is currently done in the matching process but will be moved to a thread within the subscription soon. A compiler flag is also missing.

Comment From: yoav-steinberg

@SGbo I think it would be possible to implement this entirely within a module. You'll need to create a thread which creates a virtual client which subscribes to all patterns. The module will add the new RSUBSCRIBE command. When the command is called it'll block the calling client (see https://redis.io/topics/modules-blocking-ops) and keep information regarding the selected regex. On each published message the thread will perform the relevant regext matches and delegate the message to the relevant blocking clients based on the regexes.

@oranagra @itamarhaber I think we need to consider, at least for now, what's our policy regarding adding a regex lib into Redis. As long as the policy is no, tickets like this should be closed, preferably with a good module implementation alternative.

Comment From: hpatro

We need to get in #8860 (solves #7298) to make a module to be able to subscribe to all channels* .

Comment From: yoav-steinberg

We need to get in #8860 (solves #7298) to make a module to be able to subscribe to all channels* .

Thanks, this indeed seems like and issue. I thought there was a generic solution for this using RedisModule_Call + a blocking command like PSUBSCRIBE. It turns out there isn't (@itamarhaber?). I'll just reference #7992 here which ideally should also support [P]SUBSCRIBE command.