Describe the bug
XREAD from large STREAM maybe return nil if we read from initial position of the STREAM.
To reproduce
- XADD TEST * key value - create a large stream (10M+ events).
- While stream are populated, every minute call: XTRIM TEST MAXLEN ~ 10000000 LIMIT 1000000 (~10M events and limit 1M to delete)
- A client start to read events like XREAD COUNT 100 BLOCK 3000 STREAMS TEST 127.0.0.1:6379> XINFO STREAM TEST
1) "length"
2) (integer) 10410055
3) "radix-tree-keys"
4) (integer) 321336
5) "radix-tree-nodes"
6) (integer) 422422
17) "first-entry"
18) 1) "1649836573921-0"
2) 1) "heartbeat"
2) "018021ec-444c-7000-8445-be8d76677589"
19) "last-entry"
20) 1) "1649836615697-65"
2) 1) "start_trading"
2) "018021ec-e7f0-700d-a34e-98bc7719f089"
We have 10.5M events stored (10410055).
But:
127.0.0.1:6379> XREAD COUNT 1 STREAMS TEST $
(nil)
127.0.0.1:6379> XLEN TEST
(integer) 10410055
We full stop adding new entries - and can't read messages - in our code or manual by redis-cli too.
Additional information
We checking both 7.0-rc3 version and latest stable (6.2.6), building from source (Debian 11.3)
Comment From: aleksraiden
Read the help first:
When blocking sometimes we want to receive just entries that are added to the stream via XADD starting from the moment we block. In such a case we are not interested in the history of already added entries. For this use case, we would have to check the stream top element ID, and use such ID in the XREAD command line. This is not clean and requires to call other commands, so instead it is possible to use the special $ ID to signal the stream that we want only the new things.