We use the STREAMS feature to model some kind of event-stream. All works absolutely great, but currently there is no way to delete old entries. We cannot use MAXLEN of XADD since we do not know if we would delete too many entries. The only way would be to set MAXLEN really high, but we have a lot of streams and this would result in a huge waste of memory.
Are there any plans on implementing something like XDEL my-stream 1515010595957-0? The blog post mentions a feature to delete entries in the middle. I would not need this and I would not need XDEL to be precise either - it could leave more entries than necessary. But I need a way to cap my streams.
Comment From: nicois
This has been discussed here: https://github.com/antirez/redis/issues/4450
Personally I think @antirez 's issue could be solved via semantics: if the argument was named in such a way to not give a guarantee about immediately achieving the goal, that should address his concerns.
e.g. XADD PREFERRED_MAXAGE 10000000 mystream foo bar would check the maximum age of messages in the oldest chunk, and drop the chunk if it exceeded PREFERRED_MAXAGE. So as long as there is activity on a stream, it will do what we want.
So in my mind, it's mostly about finding a term which is correctly understood by the user.
Comment From: flunderpero
I read #4450 prior to opening this issue. I am not looking for a TTL or something like that, I want to be able to specifically remove items up to a timestamp/stream-pos. I don't think the same implications apply, but perhaps I am missing something here.
On the other hand, there exist commands like zremrangebyscore which will remove an arbitrary amount of elements from a container, so from a naive point of (user)-view I would think that XDEL mystream 1515010595957-0 should work.
Comment From: kenspirit
@flunderpero Agree with you. We do want to be able to remove a range of old entries up to a specific ID.
Comment From: JimChengLin
The simplest solution I think is creating new stream periodically. Then you can throw outdated stream in a single command while keep other valid stream untouched. I don't understand why Redis lacks this critical feature even after reading the related issue. Yes, it may cause blocking, but some people(like me) need it anyway. People have to do it in a far more ugly way currently.
Comment From: xiandong79
any update on this issuse?
Comment From: itamarhaber
This is being tracked at #6640 - closing in favor of, please feel free to reopen or create a new issue.