Hi,
i'm trying to understand the behaviour of activeDefragAlloc. activeDefragAlloc allocate a new pointer if je_get_defrag_hint return true. But please consider this example down below:
int * ptr = (int *)malloc_no_tcache(sizeof(int) * 1);
free_no_tcache(ptr);
ptr = activeDefragAlloc(ptr);
if(ptr != NULL) printf("je_get_defrag_hint again: %d\n", je_get_defrag_hint(ptr));
We have allocated memory for ptr and released it. je_get_defrag_hint inside activeDefragAlloc is true and in this case activeDefragAlloc returns new allocated pointer.
We check again the new pointer with je_get_defrag_hint, but it still return true. Why? I was expecting a new pointer to not be true. The point is to determined if that pointer later are worthwhile of moving or not.
I have noticed if free_no_tcache is replaced with free then that don't happen.
I hope @oranagra or someone can explain why that happens or if that is a bug
btw Jemalloc should be upgraded from 5.0.1. to latest version that solve some memory leaks and provide notable performance and portability enhancements.
Comment From: oranagra
je_get_defrag_hint returns true when the allocation comes from a slab that has a low utilization ratio, and it's not in currently active slub (the one that's used for new allocations), it doesn't necessarily means this slab will not become the next active slab, but statistically, eventually allocations will move from low utilization slabs to one's in full slabs.
Regarding your experiment, I don't understand something. Did you call activeDefragAlloc on a pointer that's already released?
Comment From: XsarahXX
@oranagra Thank you for confirming the logic behind the function
Yes i allocated the pointer with malloc_no_tcache then released it with free_no_tcache and after that called activeDefragAlloc on it.
activeDefragAlloc did not return NULL so it returned a new pointer. As the last thing i called je_get_defrag_hint on the new pointer and the return was true. I don't understand why je_get_defrag_hint return true becouse the pointer returned from activeDefragAlloc is a new one.
Comment From: oranagra
Seems like an invalid use of activeDefragAlloc. or actually an invalid use of anything.. activeDefragAlloc reads the memory of that pointer, it'll read memory already released.
Comment From: XsarahXX
Thank you very much for confirm of usage
any chance to upgrade Jemalloc to latest version?
Comment From: oranagra
I suppose we can / should do that in Redis 7.0. you mentioned memory leaks and portability issues, can you please provide references to the exact issues?
Comment From: XsarahXX
According to previous release (5.1.0) note of jemalloc it says: "This release is primarily about fine-tuning, ranging from several new features to numerous notable performance and portability enhancements. "
For the mentioned memory leaks:
I have downloaded latest Redis that use version 5.0.1 of Jemalloc then allocated and released pointers in a loop. Xcode memory usage went gradually up to 2gb+ .
Then i downloaded the latest Jemalloc version 5.2.1 to compare with, but without any code change and the usage went from 5.9mb to 6.1mb and stayed like that.
tested 20 times and the results is the same
Comment From: oranagra
we're using this version of jemalloc for quite some time, i seriously doubt there's a memory leak (and such a trivial one), and we didn't notice. Also, i'm sure it would have been mentioned in the release notes of a newer release, and possibly they would have even released a 5.0.2 to fix it. more likely this is some side effect of a decay mechanism changes in the new releases. anyway, we'll consider upgrading in 7.0. thank you.