I'm using Redis db which have capacity of 1gm and I stored 10mb data in one key with json, one of my api is accessing that key 10hits per second, it is spiking suddenly throwing memory outage issue, causing spike, any leads?
Comment From: sundb
@Pradeep205 which command did you use?
can you give the INFO ALL?
Comment From: Pradeep205
get command to access the data from redis
Comment From: sundb
@Pradeep205 please also share the output of INFO ALL, thanks.
Comment From: Pradeep205
Server
redis_version:5.0.6 redis_git_sha1:0 redis_git_dirty:0 redis_build_id:0 redis_mode:standalone os:Amazon ElastiCache arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:0.0.0 process_id:1 run_id:bb9c428f7fc71a2b8613b07225d1b1d7270a7835 tcp_port:6379 uptime_in_seconds:10300 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:9758314 executable:- config_file:-
Clients
connected_clients:6 client_recent_max_input_buffer:4 client_recent_max_output_buffer:0 blocked_clients:0
Memory
used_memory:17276424 used_memory_human:16.48M used_memory_rss:19603456 used_memory_rss_human:18.70M used_memory_peak:29879376 used_memory_peak_human:28.50M used_memory_peak_perc:57.82% used_memory_overhead:4661320 used_memory_startup:4525496 used_memory_dataset:12615104 used_memory_dataset_perc:98.93% allocator_allocated:17767824 allocator_active:18276352 allocator_resident:38113280 used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:1103269725 maxmemory_human:1.03G maxmemory_policy:volatile-lru allocator_frag_ratio:1.03 allocator_frag_bytes:508528 allocator_rss_ratio:2.09 allocator_rss_bytes:19836928 rss_overhead_ratio:0.51 rss_overhead_bytes:-18509824 mem_fragmentation_ratio:1.14 mem_fragmentation_bytes:2368928 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:135744 mem_aof_buffer:0 mem_allocator:jemalloc-5.1.0 active_defrag_running:0 lazyfree_pending_objects:0
Persistence
loading:0 rdb_changes_since_last_save:1 rdb_bgsave_in_progress:0 rdb_last_save_time:1721024046 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0
Stats
total_connections_received:778 total_commands_processed:29962 instantaneous_ops_per_sec:3 total_net_input_bytes:12050753 total_net_output_bytes:109533498 instantaneous_input_kbps:0.08 instantaneous_output_kbps:12.15 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:72 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0
Replication
role:master connected_slaves:0 master_replid:-- master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
CPU
used_cpu_sys:6.126003 used_cpu_user:9.802325 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000
Commandstats
cmdstat_ping:calls=16323,usec=13959,usec_per_call=0.86 cmdstat_command:calls=4,usec=1270,usec_per_call=317.50 cmdstat_client:calls=346,usec=9035,usec_per_call=26.11 cmdstat_select:calls=74,usec=78,usec_per_call=1.05 cmdstat_set:calls=1,usec=6,usec_per_call=6.00 cmdstat_exists:calls=72,usec=111,usec_per_call=1.54 cmdstat_info:calls=12812,usec=862824,usec_per_call=67.34 cmdstat_config:calls=330,usec=6804,usec_per_call=20.62
SSL
ssl_enabled:no ssl_connections_to_previous_certificate:0 ssl_connections_to_current_certificate:0 ssl_current_certificate_not_before_date:(null) ssl_current_certificate_not_after_date:(null) ssl_current_certificate_serial:0
Cluster
cluster_enabled:0
Keyspace
db12:keys=1,expires=0,avg_ttl=0
Comment From: Pradeep205
Any update @sundb
Comment From: sundb
why is used_memory_peak_human only 28.5M? how do you get spiking?
Comment From: Pradeep205
@sundb This is current state I shared, when it started spiking I switched to MySQL, So it came down, It was reached 1GB when redis was connected
Comment From: sundb
but used_memory_peak_human doesn't decrese over time, it's always the pick during running.
Comment From: Pradeep205
if DB have only one key and have 10mb data only, how can memory increase if multiple user are accessing, want to understand how it works @sundb
Comment From: sundb
used_memory_human is 16.48M, when you execute the GET command for the 10mb key, the reply buffer will consume 10mb memory, 16.48 + 10 is close to 28.50.
If multiple user commands arrive in the same event loop, the spike will be higher.
Comment From: Pradeep205
Users are different and we are closing after every GET
Comment From: sundb
it the commnds are one by one, the spkie should be always 28.5mb.
Comment From: Pradeep205
That is what was not able to understand why it reached MAX limit and caused spike and burnt out
Comment From: sundb
since I don't know where you know the max limit is reached, but from info it's not, maybe you can raise an issue with AWS.