=== REDIS BUG REPORT START: Cut & paste starting from here ===
2673:S 17 May 2022 00:02:31.715 # Redis 5.0.7 crashed by signal: 11
2673:S 17 May 2022 00:02:31.715 # Crashed running the instruction at: 0x4a249e
2673:S 17 May 2022 00:02:31.715 # Accessing address: (nil)
2673:S 17 May 2022 00:02:31.715 # Failed assertion: <no assertion failed> (<no file>:0)

------ STACK TRACE ------
EIP:
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](raxRemove+0x10e)[0x4a249e]

Backtrace:
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](logStackTrace+0x29)[0x470e19]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](sigsegvHandler+0xac)[0x4714bc]
/lib64/libpthread.so.0(+0xf5f0)[0x7f7ab4af25f0]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](raxRemove+0x10e)[0x4a249e]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](slotToKeyUpdateKey+0x18d)[0x44668d]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](dbSyncDelete+0x52)[0x4467c2]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](delGenericCommand+0x6e)[0x4477ee]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](call+0x9b)[0x43049b]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](processCommand+0x33f)[0x43173f]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](processInputBuffer+0x175)[0x440405]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](processInputBufferAndReplicate+0x1e)[0x44053e]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](aeProcessEvents+0x2a0)[0x42a8c0]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](aeMain+0x2b)[0x42ab8b]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster](main+0x4c0)[0x427a30]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f7ab4737505]
/home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster][0x427c8a]

------ INFO OUTPUT ------
# Server
redis_version:5.0.7
redis_git_sha1:9eda4983
redis_git_dirty:1
redis_build_id:7f913f29c5e1b1ec
redis_mode:cluster
os:Linux 3.10.0-957.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:4.8.5
process_id:2673
run_id:f95ebbf3d026abe3b7a9cb0336258765122bb14c
tcp_port:6379
uptime_in_seconds:1666685
uptime_in_days:19
hz:10
configured_hz:10
lru_clock:8549783
executable:/home/dba/redis6379/masaike/bin/redis-server
config_file:/home/dba/redis6379/masaike/config/redis.conf

# Clients
connected_clients:2
client_recent_max_input_buffer:4
client_recent_max_output_buffer:0
blocked_clients:0

# Memory
used_memory:43746761464
used_memory_human:40.74G
used_memory_rss:64850657280
used_memory_rss_human:60.40G
used_memory_peak:61840916128
used_memory_peak_human:57.59G
used_memory_peak_perc:70.74%
used_memory_overhead:19602667256
used_memory_startup:1449992
used_memory_dataset:24144094208
used_memory_dataset_perc:55.19%
allocator_allocated:43746946216
allocator_active:63864541184
allocator_resident:64770514944
total_system_memory:168831148032
total_system_memory_human:157.24G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:128849018880
maxmemory_human:120.00G
maxmemory_policy:allkeys-lru
allocator_frag_ratio:1.46
allocator_frag_bytes:20117594968
allocator_rss_ratio:1.01
allocator_rss_bytes:905973760
rss_overhead_ratio:1.00
rss_overhead_bytes:80142336
mem_fragmentation_ratio:1.48
mem_fragmentation_bytes:21103828392
mem_not_counted_for_evict:3512
mem_replication_backlog:1048576
mem_clients_slaves:0
mem_clients_normal:66616
mem_aof_buffer:3512
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:537378214
rdb_bgsave_in_progress:0
rdb_last_save_time:1651050266
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:388
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:48711303168
aof_current_size:20289568914
aof_base_size:16464252626
aof_pending_rewrite:0
aof_buffer_length:226
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:683046
total_commands_processed:12949722795
instantaneous_ops_per_sec:27878
total_net_input_bytes:426474457353
total_net_output_bytes:445630143022
instantaneous_input_kbps:1313.18
instantaneous_output_kbps:0.06
rejected_connections:0
sync_full:0
sync_partial_ok:3
sync_partial_err:0
expired_keys:16960717
expired_stale_perc:5.96
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:7422809604
keyspace_misses:4977642266
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:1017282
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

# Replication
role:slave
master_host:10.99.49.44
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:874779203998
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:22592a6b8f06df695e457968b3832919f972f81e
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:874779203782
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:874778155207
repl_backlog_histlen:1048576

# CPU
used_cpu_sys:113986.743593
used_cpu_user:82089.503530
used_cpu_sys_children:161.676930
used_cpu_user_children:2027.761947

# Commandstats
cmdstat_config:calls=209809,usec=8195023,usec_per_call=39.06
cmdstat_get:calls=12400451870,usec=19230085311,usec_per_call=1.55
cmdstat_del:calls=107332789,usec=694353346,usec_per_call=6.47
cmdstat_setex:calls=166839989,usec=710483165,usec_per_call=4.26
cmdstat_psync:calls=3,usec=312,usec_per_call=104.00
cmdstat_command:calls=7,usec=20834,usec_per_call=2976.29
cmdstat_ping:calls=9506898,usec=2851106,usec_per_call=0.30
cmdstat_hset:calls=131602719,usec=466417854,usec_per_call=3.54
cmdstat_client:calls=70090,usec=147429,usec_per_call=2.10
cmdstat_replconf:calls=1031213,usec=1646133,usec_per_call=1.60
cmdstat_select:calls=4,usec=5,usec_per_call=1.25
cmdstat_latency:calls=50635,usec=98792,usec_per_call=1.95
cmdstat_expire:calls=131602717,usec=95201892,usec_per_call=0.72
cmdstat_slowlog:calls=101125,usec=444516,usec_per_call=4.40
cmdstat_auth:calls=477757,usec=680048,usec_per_call=1.42
cmdstat_cluster:calls=234378,usec=26055988,usec_per_call=111.17
cmdstat_info:calls=210792,usec=35501715,usec_per_call=168.42

# Cluster
cluster_enabled:1

# Keyspace
db0:keys=172033812,expires=172033812,avg_ttl=3253842108

------ CLIENT LIST OUTPUT ------
id=517240 addr=10.172.49.44:6379 fd=32 name= age=606459 idle=0 flags=M db=0 sub=0 psub=0 multi=-1 qbuf=316 qbuf-free=32452 obl=0 oll=0 omem=0 events=r cmd=del
id=682600 addr=10.189.136.109:37324 fd=10 name= age=1643 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=config

------ CURRENT CLIENT INFO ------
id=517240 addr=10.172.49.44:6379 fd=32 name= age=606459 idle=0 flags=M db=0 sub=0 psub=0 multi=-1 qbuf=316 qbuf-free=32452 obl=0 oll=0 omem=0 events=r cmd=del
argv[0]: 'DEL'
argv[1]: '/><AD><BF><B3><96>xb'

------ REGISTERS ------
2673:S 17 May 2022 00:02:31.716 # 
RAX:00000000000066e8 RBX:00000000000066e8
RCX:00000000000000ad RDX:00007f75d9e8ace8
RDI:00007f7ab4228690 RSI:0000000000000005
RBP:00007fff023ac870 RSP:00007fff023ac6d0
R8 :00007f7ab4201200 R9 :00007f7ab4400900
R10:0ef54d213ed1c76f R11:0000000000000012
R12:00007fff023ac718 R13:000000000000000a
R14:012230011a32012a R15:0000000000000005
RIP:00000000004a249e EFL:0000000000010246
CSGSFS:0000000000000033
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6df) -> 00007f7ab56218f0
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6de) -> 00007f7300000000
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6dd) -> 00007f75d4a889e0
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6dc) -> 00007f6f82cf4800
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6db) -> 00007f77e5da3600
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6da) -> 00007f7ab43bc200
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d9) -> 00007f7ab43d4070
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d8) -> 0000000000000020
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d7) -> 0000000000000005
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d6) -> 00007fff023ac718
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d5) -> 00007f75d4a889e0
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d4) -> 00007f7a91875d48
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d3) -> 0000000000000000
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d2) -> 00007f7ab4228690
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d1) -> 00007f7ab4228690
2673:S 17 May 2022 00:02:31.716 # (00007fff023ac6d0) -> 00007f7ab420b420

------ FAST MEMORY TEST ------
2673:S 17 May 2022 00:02:31.716 # Bio thread for job type #0 terminated
2673:S 17 May 2022 00:02:31.717 # Bio thread for job type #1 terminated
2673:S 17 May 2022 00:02:31.717 # Bio thread for job type #2 terminated
*** Preparing to test memory region 78c000 (2252800 bytes)
*** Preparing to test memory region 1093000 (135168 bytes)
*** Preparing to test memory region 7f6b2f400000 (63816335360 bytes)
*** Preparing to test memory region 7f7a0b053000 (2692218880 bytes)
*** Preparing to test memory region 7f7aab7d4000 (10485760 bytes)
*** Preparing to test memory region 7f7aac1d5000 (10485760 bytes)
*** Preparing to test memory region 7f7aacbd6000 (10485760 bytes)
*** Preparing to test memory region 7f7aad5d6000 (3145728 bytes)
*** Preparing to test memory region 7f7ab3e00000 (8388608 bytes)
*** Preparing to test memory region 7f7ab4ade000 (20480 bytes)
*** Preparing to test memory region 7f7ab4cfb000 (16384 bytes)
*** Preparing to test memory region 7f7ab5621000 (20480 bytes)
*** Preparing to test memory region 7f7ab562c000 (4096 bytes)
*** Preparing to test memory region 7f7ab562d000 (4096 bytes)
*** Preparing to test memory region 7f7ab5630000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: raxRemove (base: 0x4a2390)
Module: /home/dba/redis6379/masaike/bin/redis-server 10.99.49.43:6379 [cluster] (base 0x400000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x4a2390 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
2673:S 17 May 2022 00:06:01.738 # dump of function (hexdump of 398 bytes):
4157415641554989d5415455534881ec580100004c8b374c8d64244848897c240848894c241848c74424380000000048c74424402000000041f706f8ffffff4c89642430c7842448010000000000004c8974242848897c24100f84970600004531ff4885d24889f50f84880600006690450fb61e498d560441f6c3040f85ce000000418b06c1e8034885c00f84f4010000420fb64c3d0031db41384e04750eeb1e0f1f8000000000380c1a74094883c3014839c372f24839c30f84c60100004983c701488b742438483b7424400f84ed000000488b4424304c8934f04883c6014889742438418b0648c1e303c1e8038d500448f7da83e2074801c231c041f60604498d541604480f44c34c8b340241f706f8ffffff4c8974242874094d39fd0f874bffffff450fb61e4489d883e00484c04c897424200f844e0100004c89ff31dbe9b7000000662e0f1f840000000000418b0ec1e9034885c90f84390100004d39fd0f86880000004a8d743d000fb60641384604757a4d89e8bb010000004d29f8eb1c0f1f4400004c39c3746b44

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

Comment From: oranagra

seems similar to #8254 #5693 etc. we are out of luck unless maybe you can consistently reproduce it?

in any case, this is an old version that we no longer maintain, and it is likely to be solved in newer versions (we never saw this issue in newer versions, were some of these mechanisms were completely replaced).

p.s. please mention which OS / docker / image you're using, so we can confirm it's not some known platform issue.

Comment From: yuanhengWang

Hi,We use CentOS Linux release 7.6.1810 (Core)

regards~