=== REDIS BUG REPORT START: Cut & paste starting from here === 8:S 22 Oct 2024 20:14:47.439 # Redis 7.0.13 crashed by signal: 11, si_code: 2 8:S 22 Oct 2024 20:14:47.439 # Accessing address: 0x5649f5cba1c0

------ INFO OUTPUT ------

Server redis_version:7.0.13 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:ec0335a785106d85 redis_mode:cluster os:Linux 4.18.0-372.49.1.el8_6.x86_64 x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:12.2.1 process_id:8 process_supervised:no run_id:6d2fc263ef161716e311c9ddffdc191ec86a5f99 tcp_port:6379 server_time_usec:1729628087436989 uptime_in_seconds:33905 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:1574839 executable:/data/redis-server config_file:/etc/config/redis.conf io_threads_active:0

Clients connected_clients:5 cluster_connections:34 maxclients:20000 client_recent_max_input_buffer:20480 client_recent_max_output_buffer:0 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory used_memory:54324952 used_memory_human:51.81M used_memory_rss:68157440 used_memory_rss_human:65.00M used_memory_peak:56807800 used_memory_peak_human:54.18M used_memory_peak_perc:95.63% used_memory_overhead:2177460 used_memory_startup:2126528 used_memory_dataset:52147492 used_memory_dataset_perc:99.90% allocator_allocated:54382344 allocator_active:55115776 allocator_resident:72847360 total_system_memory:270049476608 total_system_memory_human:251.50G used_memory_lua:44032 used_memory_vm_eval:44032 used_memory_lua_human:43.00K used_memory_scripts_eval:3240 number_of_cached_scripts:5 number_of_functions:0 number_of_libraries:0 used_memory_vm_functions:32768 used_memory_vm_total:76800 used_memory_vm_total_human:75.00K used_memory_functions:184 used_memory_scripts:3424 used_memory_scripts_human:3.34K maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.01 allocator_frag_bytes:733432 allocator_rss_ratio:1.32 allocator_rss_bytes:17731584 rss_overhead_ratio:0.94 rss_overhead_bytes:-4689920 mem_fragmentation_ratio:1.26 mem_fragmentation_bytes:14017136 mem_not_counted_for_evict:0 mem_replication_backlog:4 mem_total_replication_buffers:0 mem_clients_slaves:0 mem_clients_normal:7200 mem_cluster_links:36992 mem_aof_buffer:0 mem_allocator:jemalloc-5.2.1 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence loading:0 async_loading:0 current_cow_peak:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:400470 rdb_bgsave_in_progress:0 rdb_last_save_time:1729594182 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_saves:0 rdb_last_cow_size:2891776 rdb_last_load_keys_expired:0 rdb_last_load_keys_loaded:49 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_rewrites:0 aof_rewrites_consecutive_failures:0 aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0

Stats total_connections_received:15280 total_commands_processed:1152042 instantaneous_ops_per_sec:0 total_net_input_bytes:1067129013 total_net_output_bytes:594455029 total_net_repl_input_bytes:982501070 total_net_repl_output_bytes:454954431 instantaneous_input_kbps:3845.30 instantaneous_output_kbps:0.02 instantaneous_input_repl_kbps:3845.30 instantaneous_output_repl_kbps:0.00 rejected_connections:0 sync_full:172 sync_partial_ok:336 sync_partial_err:172 expired_keys:1179 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:100 evicted_keys:0 evicted_clients:0 total_eviction_exceeded_time:0 current_eviction_exceeded_time:0 keyspace_hits:482081 keyspace_misses:172097 pubsub_channels:0 pubsub_patterns:0 pubsubshard_channels:0 latest_fork_usec:1632 total_forks:14 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 total_active_defrag_time:0 current_active_defrag_time:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:10918 dump_payload_sanitizations:0 total_reads_processed:938969 total_writes_processed:1000936 io_threaded_reads_processed:0 io_threaded_writes_processed:0 reply_buffer_shrinks:12969 reply_buffer_expands:3536

Replication role:slave master_host:24:12::200 master_port:6386 master_link_status:down master_last_io_seconds_ago:-1 master_sync_in_progress:0 slave_read_repl_offset:48894430 slave_repl_offset:48894430 master_link_down_since_seconds:5 slave_priority:100 slave_read_only:1 replica_announced:1 connected_slaves:0 master_failover_state:no-failover master_replid:f79f54c2344bbbe4eee45336a3d22bae1c929929 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:48894430 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:48894431 repl_backlog_histlen:0

CPU used_cpu_sys:106.312813 used_cpu_user:90.563040 used_cpu_sys_children:0.191135 used_cpu_user_children:1.329496 used_cpu_sys_main_thread:106.026030 used_cpu_user_main_thread:90.400409

Modules Commandstats cmdstat_getset:calls=0,usec=0,usec_per_call=0.00,rejected_calls=246,failed_calls=0 cmdstat_smembers:calls=4777,usec=13552,usec_per_call=2.84,rejected_calls=5,failed_calls=0 cmdstat_xrange:calls=1,usec=5,usec_per_call=5.00,rejected_calls=0,failed_calls=0 cmdstat_hset:calls=2718,usec=20180,usec_per_call=7.42,rejected_calls=0,failed_calls=0 cmdstat_xread:calls=3773,usec=22769,usec_per_call=6.03,rejected_calls=486,failed_calls=0 cmdstat_hgetall:calls=30,usec=94,usec_per_call=3.13,rejected_calls=0,failed_calls=0 cmdstat_hkeys:calls=1640,usec=7756,usec_per_call=4.73,rejected_calls=5,failed_calls=0 cmdstat_zrem:calls=298,usec=2703,usec_per_call=9.07,rejected_calls=0,failed_calls=0 cmdstat_xgroup|delconsumer:calls=104,usec=1178,usec_per_call=11.33,rejected_calls=70,failed_calls=0 cmdstat_xgroup|create:calls=1,usec=11,usec_per_call=11.00,rejected_calls=0,failed_calls=0 cmdstat_flushdb:calls=4,usec=164290,usec_per_call=41072.50,rejected_calls=0,failed_calls=0 cmdstat_command:calls=36,usec=71285,usec_per_call=1980.14,rejected_calls=0,failed_calls=0 cmdstat_type:calls=381,usec=1515,usec_per_call=3.98,rejected_calls=0,failed_calls=0 cmdstat_scan:calls=2620,usec=34233,usec_per_call=13.07,rejected_calls=0,failed_calls=0 cmdstat_xpending:calls=13620,usec=62466,usec_per_call=4.59,rejected_calls=21,failed_calls=0 cmdstat_xreadgroup:calls=200531,usec=947226,usec_per_call=4.72,rejected_calls=42,failed_calls=0 cmdstat_xadd:calls=300,usec=6450,usec_per_call=21.50,rejected_calls=0,failed_calls=0 cmdstat_xinfo|stream:calls=73,usec=813,usec_per_call=11.14,rejected_calls=0,failed_calls=0 cmdstat_role:calls=4024,usec=13411,usec_per_call=3.33,rejected_calls=0,failed_calls=0 cmdstat_get:calls=74665,usec=244068,usec_per_call=3.27,rejected_calls=259,failed_calls=0 cmdstat_select:calls=274,usec=229,usec_per_call=0.84,rejected_calls=0,failed_calls=0 cmdstat_eval:calls=427,usec=18397,usec_per_call=43.08,rejected_calls=5,failed_calls=0 cmdstat_subscribe:calls=88,usec=791,usec_per_call=8.99,rejected_calls=0,failed_calls=0 cmdstat_exists:calls=1028,usec=2259,usec_per_call=2.20,rejected_calls=2,failed_calls=0 cmdstat_client|id:calls=24,usec=13,usec_per_call=0.54,rejected_calls=0,failed_calls=0 cmdstat_client|setname:calls=2259,usec=6424,usec_per_call=2.84,rejected_calls=0,failed_calls=0 cmdstat_sadd:calls=1,usec=3,usec_per_call=3.00,rejected_calls=0,failed_calls=0 cmdstat_del:calls=194547,usec=676718,usec_per_call=3.48,rejected_calls=0,failed_calls=0 cmdstat_slowlog|get:calls=2259,usec=9200,usec_per_call=4.07,rejected_calls=0,failed_calls=0 cmdstat_slowlog|len:calls=2259,usec=2321,usec_per_call=1.03,rejected_calls=0,failed_calls=0 cmdstat_multi:calls=491,usec=227,usec_per_call=0.46,rejected_calls=0,failed_calls=0 cmdstat_ping:calls=7186,usec=5568,usec_per_call=0.77,rejected_calls=2,failed_calls=0 cmdstat_xclaim:calls=229,usec=2377,usec_per_call=10.38,rejected_calls=0,failed_calls=0 cmdstat_xdel:calls=298,usec=1745,usec_per_call=5.86,rejected_calls=0,failed_calls=0 cmdstat_zrangebyscore:calls=135227,usec=906433,usec_per_call=6.70,rejected_calls=210,failed_calls=0 cmdstat_getrange:calls=882,usec=1219,usec_per_call=1.38,rejected_calls=0,failed_calls=0 cmdstat_config|get:calls=37400,usec=642377,usec_per_call=17.18,rejected_calls=0,failed_calls=0 cmdstat_config|set:calls=4569,usec=10912,usec_per_call=2.39,rejected_calls=0,failed_calls=0 cmdstat_set:calls=196565,usec=2119135,usec_per_call=10.78,rejected_calls=0,failed_calls=0 cmdstat_hget:calls=3415,usec=8718,usec_per_call=2.55,rejected_calls=0,failed_calls=0 cmdstat_info:calls=5572,usec=1765280,usec_per_call=316.81,rejected_calls=0,failed_calls=0 cmdstat_cluster|reset:calls=1,usec=2395,usec_per_call=2395.00,rejected_calls=0,failed_calls=0 cmdstat_cluster|forget:calls=8765,usec=122332,usec_per_call=13.96,rejected_calls=0,failed_calls=7786 cmdstat_cluster|slots:calls=6484,usec=336746,usec_per_call=51.93,rejected_calls=5,failed_calls=0 cmdstat_cluster|info:calls=6243,usec=403517,usec_per_call=64.64,rejected_calls=1,failed_calls=0 cmdstat_cluster|myid:calls=3985,usec=4291,usec_per_call=1.08,rejected_calls=0,failed_calls=0 cmdstat_cluster|shards:calls=3985,usec=272683,usec_per_call=68.43,rejected_calls=0,failed_calls=0 cmdstat_cluster|replicate:calls=157,usec=53040,usec_per_call=337.83,rejected_calls=0,failed_calls=2 cmdstat_cluster|failover:calls=65,usec=88372,usec_per_call=1359.57,rejected_calls=0,failed_calls=0 cmdstat_cluster|meet:calls=141,usec=1772,usec_per_call=12.57,rejected_calls=0,failed_calls=0 cmdstat_cluster|addslots:calls=2,usec=275,usec_per_call=137.50,rejected_calls=0,failed_calls=1 cmdstat_cluster|nodes:calls=3985,usec=352492,usec_per_call=88.45,rejected_calls=0,failed_calls=0 cmdstat_psync:calls=508,usec=16639,usec_per_call=32.75,rejected_calls=0,failed_calls=0 cmdstat_hmset:calls=2,usec=8,usec_per_call=4.00,rejected_calls=0,failed_calls=0 cmdstat_zadd:calls=5381,usec=86976,usec_per_call=16.16,rejected_calls=8,failed_calls=0 cmdstat_evalsha:calls=155091,usec=4096892,usec_per_call=26.42,rejected_calls=407,failed_calls=3 cmdstat_zcard:calls=9304,usec=12303,usec_per_call=1.32,rejected_calls=6,failed_calls=0 cmdstat_exec:calls=491,usec=12093,usec_per_call=24.63,rejected_calls=0,failed_calls=0 cmdstat_latency|latest:calls=2259,usec=3145,usec_per_call=1.39,rejected_calls=0,failed_calls=0 cmdstat_latency|histogram:calls=2259,usec=2469522,usec_per_call=1093.19,rejected_calls=0,failed_calls=0 cmdstat_replconf:calls=30579,usec=30895,usec_per_call=1.01,rejected_calls=0,failed_calls=0 cmdstat_hello:calls=7461,usec=42521,usec_per_call=5.70,rejected_calls=0,failed_calls=0 cmdstat_xack:calls=298,usec=848,usec_per_call=2.85,rejected_calls=0,failed_calls=0

Errorstats errorstat_CLUSTERDOWN:count=272 errorstat_ERR:count=7789 errorstat_LOADING:count=8 errorstat_MOVED:count=1500 errorstat_NOSCRIPT:count=3 errorstat_UNBLOCKED:count=1346

Latencystats latency_percentiles_usec_smembers:p50=2.007,p99=10.047,p99.9=31.103 latency_percentiles_usec_xrange:p50=5.023,p99=5.023,p99.9=5.023 latency_percentiles_usec_hset:p50=6.015,p99=22.015,p99.9=60.159 latency_percentiles_usec_hgetall:p50=3.007,p99=5.023,p99.9=5.023 latency_percentiles_usec_hkeys:p50=4.015,p99=17.023,p99.9=35.071 latency_percentiles_usec_zrem:p50=8.031,p99=31.103,p99.9=53.247 latency_percentiles_usec_xgroup|delconsumer:p50=10.047,p99=31.103,p99.9=89.087 latency_percentiles_usec_xgroup|create:p50=11.007,p99=11.007,p99.9=11.007 latency_percentiles_usec_flushdb:p50=100.351,p99=83361.791,p99.9=83361.791 latency_percentiles_usec_command:p50=1974.271,p99=2359.295,p99.9=2359.295 latency_percentiles_usec_type:p50=4.015,p99=10.047,p99.9=46.079 latency_percentiles_usec_scan:p50=10.047,p99=49.151,p99.9=89.087 latency_percentiles_usec_xpending:p50=4.015,p99=17.023,p99.9=33.023 latency_percentiles_usec_xreadgroup:p50=16.063,p99=50.175,p99.9=61.183 latency_percentiles_usec_xadd:p50=17.023,p99=106.495,p99.9=186.367 latency_percentiles_usec_xinfo|stream:p50=10.047,p99=24.063,p99.9=26.111 latency_percentiles_usec_role:p50=2.007,p99=20.095,p99.9=32.127 latency_percentiles_usec_get:p50=2.007,p99=16.063,p99.9=35.071 latency_percentiles_usec_select:p50=1.003,p99=2.007,p99.9=19.071 latency_percentiles_usec_eval:p50=34.047,p99=176.127,p99.9=311.295 latency_percentiles_usec_subscribe:p50=7.007,p99=30.079,p99.9=30.079 latency_percentiles_usec_exists:p50=2.007,p99=6.015,p99.9=17.023 latency_percentiles_usec_client|id:p50=1.003,p99=1.003,p99.9=1.003 latency_percentiles_usec_client|setname:p50=2.007,p99=11.007,p99.9=19.071 latency_percentiles_usec_sadd:p50=3.007,p99=3.007,p99.9=3.007 latency_percentiles_usec_del:p50=3.007,p99=11.007,p99.9=36.095 latency_percentiles_usec_slowlog|get:p50=4.015,p99=11.007,p99.9=29.055 latency_percentiles_usec_slowlog|len:p50=1.003,p99=2.007,p99.9=11.007 latency_percentiles_usec_multi:p50=0.001,p99=1.003,p99.9=13.055 latency_percentiles_usec_ping:p50=1.003,p99=6.015,p99.9=13.055 latency_percentiles_usec_xclaim:p50=9.023,p99=32.127,p99.9=66.047 latency_percentiles_usec_xdel:p50=5.023,p99=16.063,p99.9=22.015 latency_percentiles_usec_zrangebyscore:p50=6.015,p99=20.095,p99.9=55.039 latency_percentiles_usec_getrange:p50=1.003,p99=3.007,p99.9=14.015 latency_percentiles_usec_config|get:p50=3.007,p99=244.735,p99.9=337.919 latency_percentiles_usec_config|set:p50=2.007,p99=8.031,p99.9=26.111 latency_percentiles_usec_set:p50=11.007,p99=30.079,p99.9=78.335 latency_percentiles_usec_hget:p50=2.007,p99=9.023,p99.9=38.143 latency_percentiles_usec_info:p50=18.047,p99=1114.111,p99.9=1507.327 latency_percentiles_usec_cluster|reset:p50=2408.447,p99=2408.447,p99.9=2408.447 latency_percentiles_usec_cluster|forget:p50=2.007,p99=130.047,p99.9=548.863 latency_percentiles_usec_cluster|slots:p50=49.151,p99=101.375,p99.9=270.335 latency_percentiles_usec_cluster|info:p50=60.159,p99=118.271,p99.9=227.327 latency_percentiles_usec_cluster|myid:p50=1.003,p99=2.007,p99.9=6.015 latency_percentiles_usec_cluster|shards:p50=71.167,p99=122.367,p99.9=195.583 latency_percentiles_usec_cluster|replicate:p50=352.255,p99=614.399,p99.9=774.143 latency_percentiles_usec_cluster|failover:p50=1556.479,p99=2654.207,p99.9=3342.335 latency_percentiles_usec_cluster|meet:p50=11.007,p99=72.191,p99.9=138.239 latency_percentiles_usec_cluster|addslots:p50=78.335,p99=197.631,p99.9=197.631 latency_percentiles_usec_cluster|nodes:p50=84.479,p99=187.391,p99.9=446.463 latency_percentiles_usec_psync:p50=29.055,p99=85.503,p99.9=104.447 latency_percentiles_usec_hmset:p50=1.003,p99=7.007,p99.9=7.007 latency_percentiles_usec_zadd:p50=14.015,p99=63.231,p99.9=157.695 latency_percentiles_usec_evalsha:p50=26.111,p99=79.359,p99.9=182.271 latency_percentiles_usec_zcard:p50=1.003,p99=5.023,p99.9=14.015 latency_percentiles_usec_exec:p50=21.119,p99=83.455,p99.9=207.871 latency_percentiles_usec_latency|latest:p50=1.003,p99=3.007,p99.9=13.055 latency_percentiles_usec_latency|histogram:p50=1056.767,p99=2088.959,p99.9=2195.455 latency_percentiles_usec_replconf:p50=1.003,p99=5.023,p99.9=15.039 latency_percentiles_usec_hello:p50=5.023,p99=15.039,p99.9=28.031 latency_percentiles_usec_xack:p50=3.007,p99=6.015,p99.9=15.039

Cluster cluster_enabled:1

Keyspace db0:keys=49,expires=1,avg_ttl=0

------ CLIENT LIST OUTPUT ------ id=15424 addr=10.128.1.190:34942 laddr=10.128.1.186:6379 fd=18 name= age=36 idle=25 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=3 id=15428 addr=10.129.0.184:38018 laddr=10.128.1.186:6379 fd=11 name= age=15 idle=15 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=3 id=15431 addr=10.130.0.120:48994 laddr=10.128.1.186:6379 fd=37 name= age=11 idle=11 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=3 id=15435 addr=[::1]:59266 laddr=[::1]:6379 fd=100 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17160 events=r cmd=NULL user=default redir=-1 resp=2 id=15436 addr=[fd01:0:0:2::f6a]:51370 laddr=[fd01:0:0:3::42b7]:6379 fd=101 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=4 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17160 events=r cmd=NULL user=default redir=-1 resp=2

------ MODULES INFO OUTPUT ------

------ CONFIG DEBUG OUTPUT ------ lazyfree-lazy-user-del no list-compress-depth 0 replica-read-only yes lazyfree-lazy-user-flush no io-threads-do-reads no slave-read-only yes io-threads 1 lazyfree-lazy-server-del no lazyfree-lazy-expire no lazyfree-lazy-eviction no repl-diskless-sync yes sanitize-dump-payload no repl-diskless-load disabled proto-max-bulk-len 512mb client-query-buffer-limit 1gb activedefrag no

------ FAST MEMORY TEST ------ 8:S 22 Oct 2024 20:14:47.441 # Bio thread for job type #0 terminated 8:S 22 Oct 2024 20:14:47.441 # Bio thread for job type https://github.com/RedisLabs/redisraft/issues/1 terminated 8:S 22 Oct 2024 20:14:47.441 # Bio thread for job type https://github.com/RedisLabs/redisraft/issues/2 terminated *** Preparing to test memory region 5649f5ea6000 (2306048 bytes) *** Preparing to test memory region 5649f7ea9000 (20480 bytes) *** Preparing to test memory region 7f481c800000 (33554432 bytes) *** Preparing to test memory region 7f481e980000 (76021760 bytes) *** Preparing to test memory region 7f48232f0000 (32768 bytes) *** Preparing to test memory region 7f48232fa000 (139264 bytes) *** Preparing to test memory region 7f482331f000 (139264 bytes) *** Preparing to test memory region 7f4823f4d000 (114688 bytes) *** Preparing to test memory region 7f4823f69000 (2621440 bytes) *** Preparing to test memory region 7f48241e9000 (94208 bytes) *** Preparing to test memory region 7f4824200000 (8388608 bytes) *** Preparing to test memory region 7f4824a00000 (299008 bytes) *** Preparing to test memory region 7f4824e5a000 (12288 bytes) *** Preparing to test memory region 7f4824f7b000 (12288 bytes) .O.O.O.O.O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

Please report the crash by opening an issue on github:

   http://github.com/redis/redis/issues

If a Redis module was involved, please open in the module's repo instead.

Suspect RAM error? Use redis-server --test-memory to verify it.

Some other issues could be detected by redis-server --check-system Segmentation fault (core dumped)