The problem/use-case that the feature addresses

I'm currently writing an implementation of this full-text-search indexing pattern. The fact that ZINTER/SINTER don't take a LIMIT parameter means that a query could result in a huge reply from Redis. I could use the store variant instead, so the resulting set size wouldn't be an issue - but that complicates things quite a bit more than I would have liked (having to do cleanup, key name clashes for the resulting sets, etc.)

Description of the feature

For sorted sets: - The command should return as soon as the limit was reached, ideally sorted by rank (highest to lowest)

For sets: - I guess this should work the same way that SINTERCARD does (so whatever is most efficient).

Some issues I see: - Unable to paginate further into the result set (but maybe at that stage simply using the STORE variants coupled with something like {Z|S}SCAN would make more sense? - I assume there's no performance gain from this for sorted sets if the results are ranked - maybe that should be an opt-in behavior?

Alternatives you've considered

Using the STORE versions:

  1. Start a transaction when a query is made
  2. Use ZINTERSTORE/ZUNIONSTORE
  3. Scan the new set for the top N results based on score
  4. Remove the created set
  5. Return the result

The key for the new set should use something like an uuid v4 to avoid naming conflicts (I don't have authenticated users). The only issue with this is that it seems unnecessarily complex and is less efficient than a native solution would be.

Another option would be using a LUA script / Redis functions to accomplish the same. More efficient, but added complexity.

Lastly, specifically for my use case, I guess calling ZINTERCARD first until I have a decent result size may make sense. Meaning, if the query matches hundreds of thousands of documents, I could nudge the user to narrow the query until it reaches < 100 matches or something, and only then calling ZINTER, removing the need for the LIMIT parameter. However, this doesn't work for ZUNION, and it's a bit of a compromise regardless (the desired document may be in the top 100 results even before narrowing).

Additional information N/A

Comment From: oranagra

i see there's already a PR for that: #10874 (at least the ZSET part). maybe you can help evaluate it.

Comment From: ClaudiuCeia

Thanks @oranagra, I hadn't noticed. I'll try to test that PR locally and write some unit tests for it, then get back with an update.

Comment From: guybe7

@ClaudiuCeia can we close this issue as a dup of #10874?

Comment From: oranagra

@guybe7 that PR only handles sorted sets, and this issue is also about sets, so not strictly a dup (yet)