We have an application which loads a billion keys into Redis (yes billion) in a large cluster and after using them for a period of days, the data is wiped out to be used for the next batch of K-V. I know that it is documented that deleting keys does not necessarily cause malloc to release memory back to the system.
It would be nice if one of the following was implemented.
(1) After a flushall, best efforts are made to prune physical memory used to what is actually active.
(2) An administrative command (use at your own risk) which forces physical memory pruning.
Since the instance is being flushed, it is acceptable to block all activity while the pruning is in progress.
Either of these suggestions would help with resource allocations on particular instances especially in the case where a significantly smaller number of keys are added after the flushall.
Comment From: itamarhaber
Version 4 has both MEMORY PURGE as well as the active defragmentor - did
you check them out?
On Sep 28, 2017 1:50 AM, "allanwax" notifications@github.com wrote:
We have an application which loads a billion keys into Redis (yes billion) in a large cluster and after using them for a period of days, the data is wiped out to be used for the next batch of K-V. I know that it is documented that deleting keys does not necessarily cause malloc to release memory back to the system.
It would be nice if one of the following was implemented.
(1) After a flushall, best efforts are made to prune physical memory used to what is actually active. (2) An administrative command (use at your own risk) which forces physical memory pruning.
Since the instance is being flushed, it is acceptable to block all activity while the pruning is in progress.
Either of these suggestions would help with resource allocations on particular instances especially in the case where a significantly smaller number of keys are added after the flushall.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/antirez/redis/issues/4339, or mute the thread https://github.com/notifications/unsubscribe-auth/AFx1_JdIZiMSCQnWf_dRdXWIPgenyefpks5smtE6gaJpZM4PmgDz .
Comment From: allanwax
Couldn't find the doc on them using google. Thanks for the reference.
I would still be nice if if flushall called MEMORY PURGE after it's done
Comment From: itamarhaber
The PR for the docs is at https://github.com/antirez/redis-doc/pull/851 (/cc @antirez ;))
WRT to auto-purge, perhaps, although you could just as easily restart the server.
Comment From: itamarhaber
This issue seems like it can be safely closed - please reopen or create a new one if needed.