Redis Redis Cache Read Latency Increase with Time

We are using ElastiCache Redis (version 5.0.3) as our cache server and Redisson 3.8 as our client.

We find that with time one, the cache read latency will increase, for example, after the deployment, in the first 3 days, the latency is almost same, keep 2 to 3 milliseconds. In the next 3 days, the latency can increase to 20 milliseconds. It will keep increasing. This latency increase has no relation with traffic.

At the same time, I find that that the redis cluster cache miss provided by AWS (The number of unsuccessful read-only key lookups in the main dictionary. This is derived from keyspace_misses at Redis INFO.) will increase from the very beginning (after the deployment).

We can do deployment to fix the issue. After the deployment, the latency and cache miss go to normal.

Here are the write and read code.

Write

RBatch batch = redissonClient.createBatch(BatchOptions.defaults());
batch.getMapCache(key).putIfAbsentAsync(itemKey, itemValue, ttlValue, timeUnit);
batch.execute();

Read

RMapCache<ItemKeyType, ItemValueType> relatedVideosMap = redissonClient.getMapCache(key); Please help me to fix the issue.

Thanks.

Comment From: filipecosta90

Hi there @turbocv , from the time you wrote this question up until now are you still experiencing this issue? If solved can you please close the issue? If not, a couple of questions: - The latency you're mentioning is the average one reported by the server correct? - Is this latency continuous increase also deeply visible from the client? - If this is not related to the dataset how is the cluster cache miss changing over time and being affected by a deployment? It seems that the read latency is around 2-3 when you lack data and then ( when the cache misses are reduced latency increases as expected given your doing actual work on redis ).

Bringing this issue to the attention of @madolson