Redis 7.0
SET K1 Redis
- Initial memory is 6 or 8 ?
- Memory expansion after updating value？
Comment From: oranagra
why won't you just test it (using MEMORY USAGE K1)
(and note that it doesn't include the hash-table (normally another 8 bytes per key)
Comment From: ifeisier
MEMORY USAGE K1 including unused space?
redis 7.0 sds：
struct sdshdr {
int len;
int free;
char buf[];
}
https://redis.io/commands/memory-usage/
Why is that?
127.0.0.1:6379[1]> SET "" ""
OK
127.0.0.1:6379[1]> MEMORY USAGE ""
(integer) 56
127.0.0.1:6379[1]> SET "" "11"
OK
127.0.0.1:6379[1]> MEMORY USAGE ""
(integer) 48
127.0.0.1:6379[1]>
https://developpaper.com/deep-understanding-of-rediss-simple-dynamic-string/
- Inert space release
Idle space release is used to optimize the string shortening operation of SDS: when the API of SDS needs to shorten the string saved in SDS, the program does not immediately use memory reallocation to recover the extra bytes after shortening, but uses the free attribute to record the number of these bytes and wait for future use.
Comment From: oranagra
yes, MEMORY USAGE includes the real footprint of the key, including buffers, extra reserved space, and even internal fragmentation due to the allocator bins.
i suppose the reason for the fact you shown that an empty string takes more memory than a 2 char one is this: https://github.com/redis/redis/blob/c66eaf4e4a019fe379556bd6f725524cca0e2d96/src/sds.c#L107-L109
it means that the sds header will be 3 bytes instead of 1 byte, and that will drive us to use the next bin size of the allocator, so it ends up adding 8 bytes.
Comment From: ifeisier
I don't understand, do you have an article? 😊😊😊
Comment From: oranagra
no, i have the source code!
the fact is that with current code, redis allocates a few bytes more for empty strings, compared to very short ones. the reason is that it considers short one as potentially being static, but considers that an empty one, is likely to be appended soon.
the other piece of the puzzle is that SDS strings have a header with several encoding types, the smallest one (used for strings with less than 32 chars), is only efficient for static strings, not ones that are gonna be appended, since it saves two more bytes, and because of that it doesn't have room to remember the size of the allocated buffer, which is needed in order to grow and know when a realloc is needed.
Comment From: ifeisier
- Space pre allocation
Space pre allocation is used to optimize the string growth operation of SDS: when the API of SDS modifies an SDS and needs to expand the space of SDS, the program will not only allocate the necessary space for modification, but also allocate additional unused space for SDS. The amount of unused space allocated additionally is determined by the following companies:
- If the length of SDS (that is, the value of len attribute) is less than 1MB after modifying the SDS, then the program allocates unused space of the same size as len attribute, and the value of len attribute of SDS will be the same as that of free attribute. For example, if the len of SDS is changed to 13 bytes after modification, the program will also allocate 13 bytes of unused space, and the actual length of the buf array of SDS will be 13 + 13 + 1 bytes (an extra byte is used to store empty characters).
- If the length of SDS is greater than or equal to 1MB after modification, the program will allocate 1MB of unused space. For example, if the len of SDS is changed to 30MB after modification, the program will allocate 1MB of unused space, and the actual length of the buf array of SDS is 30MB + 1MB + 1byte. Through space pre allocation strategy, redis can reduce the number of memory reallocation required for continuous string growth operations.
Is it right?
Comment From: oranagra
that's not only that. we also take ownership of the internal fragmentation (when you allocate 6 bytes, you get 8). bottom line is that sds strings know their used size and allocated size. but the smallest header type SDS5 doesn't bother to remember the allocated, so very good for resizing, which is why redis avoids using it on empty string. again because it assumes empty ones are probably gonna change, and assumes other small ones, are probably constant.
Comment From: ifeisier
Thank you