Dear Redis Team,

(I had already asked this in StackOverFlow, Redis GoogleGroup. No response. So I have to trouble you guys).

I am looking for suggestion/advice related to redis streams usage in production.

In pub/sub redis pushes the data to client itself. In Redis stream , looks like, the client has to constantly poll for new items.

I did read this - https://redis.io/topics/streams-intro#listening-for-new-items-with-xread

If I implement a consumer which is supposed to the process all the messages in more or less real time, Should I look for new items in an infinite loop with some BLOCK time. ? Is there any best practice for BLOCK time. For ex: 1 hour - is it good or bad? Does it have any impact on redis?

Comment From: itamarhaber

Hi @kitkars

In Redis stream , looks like, the client has to constantly poll for new items.

It just looks like this, but it isn't really so. When you issue an XREAD/XREADGROUP ... BLOCK, there's no polling involved during that period until the timeout is met. An extremely short timeout would indeed lead to polling-like, but if you use sensible timeout values (where sensible allows new entries to be published) this likeness will go away.

It is hard to judge what a sensible timeout value would be without more information on the use case. Generally, you want to balance among several things, including:

  • The rate of new messages: the faster these arrive, the shorter the timeout can be
  • The time spent blocking the client: the longer the timeout the longer the client is blocked. Theoretically, the client could timeout and move to other tasks.
  • Timeouts aren't bad, as they can tell the client that enough time has passed (e.g. several timeouts in succession) to indicate there's a potential upstream problem. Setting long/indefinite timeout values diminishes that.

I hope this makes some sense and helps with your ponderings.

Comment From: kitkars

Thanks @itamarhaber

This is how I interpreted your answer.

Since the client's only job is to process all these messages, then long timeouts are OK. It does not hurt anything on Redis side.

Comment From: itamarhaber

Cool - closing this question, but feel free to reopen if needed.