=== REDIS BUG REPORT START: Cut & paste starting from here === 1639087:C 12 Sep 2024 08:50:50.184 # ------------------------------------------------ 1639087:C 12 Sep 2024 08:50:50.184 # !!! Software Failure. Press left mouse button to continue 1639087:C 12 Sep 2024 08:50:50.184 # Guru Meditation: Unknown hash encoding #rdb.c:682

------ STACK TRACE ------

Backtrace: redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889aa382] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ab1be] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889abb3a] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889abdf3] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ac1e2] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ac585] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118897ee1a] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118896c041] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118896c2fd] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889678c0] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fc96f229d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fc96f229e40] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x561188967fa5]

------ INFO OUTPUT ------

Server

redis_version:7.0.3 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:76796e0f0ae09c8e redis_mode:cluster os:Linux 6.2.0-1018-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:11.4.0 process_id:1639087 process_supervised:systemd run_id:0eaac0d3b5bc8e2198623057401fc8394080b488 tcp_port:26379 server_time_usec:1726131050054149 uptime_in_seconds:5406 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:14855018 executable:/usr/local/bin/redis-server config_file:/etc/redis/redis.conf io_threads_active:0

Clients

connected_clients:26 cluster_connections:10 maxclients:25000 client_recent_max_input_buffer:20480 client_recent_max_output_buffer:20504 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory

used_memory:88925096 used_memory_human:84.81M used_memory_rss:151900160 used_memory_rss_human:144.86M used_memory_peak:92142720 used_memory_peak_human:87.87M used_memory_peak_perc:96.51% used_memory_overhead:5792132 used_memory_startup:2757752 used_memory_dataset:83132964 used_memory_dataset_perc:96.48% allocator_allocated:91110136 allocator_active:127995904 allocator_resident:132538368 total_system_memory:8315179008 total_system_memory_human:7.74G used_memory_lua:31744 used_memory_vm_eval:31744 used_memory_lua_human:31.00K used_memory_scripts_eval:0 number_of_cached_scripts:0 number_of_functions:0 number_of_libraries:0 used_memory_vm_functions:32768 used_memory_vm_total:64512 used_memory_vm_total_human:63.00K used_memory_functions:184 used_memory_scripts:184 used_memory_scripts_human:184B maxmemory:4572839936 maxmemory_human:4.26G maxmemory_policy:allkeys-lru allocator_frag_ratio:1.40 allocator_frag_bytes:36885768 allocator_rss_ratio:1.04 allocator_rss_bytes:4542464 rss_overhead_ratio:1.15 rss_overhead_bytes:19361792 mem_fragmentation_ratio:1.71 mem_fragmentation_bytes:62976992 mem_not_counted_for_evict:2048 mem_replication_backlog:82020 mem_total_replication_buffers:82016 mem_clients_slaves:0 mem_clients_normal:190104 mem_cluster_links:10880 mem_aof_buffer:2048 mem_allocator:jemalloc-5.2.1 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence

loading:0 async_loading:0 current_cow_peak:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:39 rdb_bgsave_in_progress:0 rdb_last_save_time:1726129249 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_saves:4 rdb_last_cow_size:2117632 rdb_last_load_keys_expired:0 rdb_last_load_keys_loaded:2467 aof_enabled:1 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_rewrites:0 aof_rewrites_consecutive_failures:0 aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0 aof_current_size:23017543 aof_base_size:648737 aof_pending_rewrite:0 aof_buffer_length:0 aof_pending_bio_fsync:0 aof_delayed_fsync:0

Stats

total_connections_received:516 total_commands_processed:44809 instantaneous_ops_per_sec:9 total_net_input_bytes:723429 total_net_output_bytes:30086510 total_net_repl_input_bytes:0 total_net_repl_output_bytes:13725432 instantaneous_input_kbps:0.11 instantaneous_output_kbps:0.31 instantaneous_input_repl_kbps:0.00 instantaneous_output_repl_kbps:0.00 rejected_connections:0 sync_full:1 sync_partial_ok:0 sync_partial_err:1 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:55 evicted_keys:0 evicted_clients:0 total_eviction_exceeded_time:0 current_eviction_exceeded_time:0 keyspace_hits:93822 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 pubsubshard_channels:0 latest_fork_usec:2676 total_forks:4 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 total_active_defrag_time:0 current_active_defrag_time:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:33 dump_payload_sanitizations:0 total_reads_processed:29019 total_writes_processed:23865 io_threaded_reads_processed:0 io_threaded_writes_processed:0 reply_buffer_shrinks:65 reply_buffer_expands:21

Replication

role:master connected_slaves:1 slave0:ip=10.12.72.110,port=6379,state=online,offset=71486,lag=1 master_failover_state:no-failover master_replid:5b1ce70e0033ec292cd0b68167bfea1637b8ecb8 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:71486 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:415236096 repl_backlog_first_byte_offset:1 repl_backlog_histlen:71486

CPU

used_cpu_sys:0.011656 used_cpu_user:0.116561 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000 used_cpu_sys_main_thread:0.011656 used_cpu_user_main_thread:0.116561

Modules

module:name=ReJSON,ver=20407,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors] module:name=search,ver=20612,api=1,filters=0,usedby=[],using=[ReJSON],options=[]

Commandstats

cmdstat_exists:calls=11,usec=32,usec_per_call=2.91,rejected_calls=2,failed_calls=0 cmdstat_replconf:calls=5371,usec=7394,usec_per_call=1.38,rejected_calls=0,failed_calls=0 cmdstat_ping:calls=16193,usec=20219,usec_per_call=1.25,rejected_calls=9,failed_calls=0 cmdstat_config|get:calls=519,usec=71476,usec_per_call=137.72,rejected_calls=0,failed_calls=0 cmdstat_info:calls=465,usec=111920,usec_per_call=240.69,rejected_calls=0,failed_calls=0 cmdstat_search.CLUSTERREFRESH:calls=5403,usec=375683,usec_per_call=69.53,rejected_calls=0,failed_calls=0 cmdstat_json.set:calls=67,usec=20763,usec_per_call=309.90,rejected_calls=19,failed_calls=0 cmdstat_set:calls=21,usec=263,usec_per_call=12.52,rejected_calls=1,failed_calls=0 cmdstat_client|setname:calls=480,usec=1186,usec_per_call=2.47,rejected_calls=0,failed_calls=0 cmdstat_FT.SEARCH:calls=14,usec=138591,usec_per_call=9899.36,rejected_calls=0,failed_calls=0 cmdstat_auth:calls=516,usec=2403,usec_per_call=4.66,rejected_calls=0,failed_calls=0 cmdstat_latency|latest:calls=458,usec=773,usec_per_call=1.69,rejected_calls=0,failed_calls=0 cmdstat_latency|histogram:calls=458,usec=142513,usec_per_call=311.16,rejected_calls=0,failed_calls=0 cmdstat_cluster|info:calls=458,usec=22874,usec_per_call=49.94,rejected_calls=0,failed_calls=0 cmdstat_cluster|slots:calls=7989,usec=248057,usec_per_call=31.05,rejected_calls=0,failed_calls=0 cmdstat_cluster|myid:calls=5403,usec=13246,usec_per_call=2.45,rejected_calls=0,failed_calls=0 cmdstat_del:calls=10,usec=440,usec_per_call=44.00,rejected_calls=0,failed_calls=0 cmdstat_get:calls=11,usec=39,usec_per_call=3.55,rejected_calls=0,failed_calls=0 cmdstat_psync:calls=1,usec=2696,usec_per_call=2696.00,rejected_calls=2,failed_calls=0 cmdstat_slowlog|len:calls=458,usec=553,usec_per_call=1.21,rejected_calls=0,failed_calls=0 cmdstat_slowlog|get:calls=458,usec=1604,usec_per_call=3.50,rejected_calls=0,failed_calls=0 cmdstat__FT.SEARCH:calls=45,usec=38128,usec_per_call=847.29,rejected_calls=0,failed_calls=0

Errorstats

errorstat_LOADING:count=8 errorstat_MOVED:count=22 errorstat_NOAUTH:count=3

Latencystats

latency_percentiles_usec_exists:p50=4.015,p99=4.015,p99.9=4.015 latency_percentiles_usec_replconf:p50=1.003,p99=2.007,p99.9=12.031 latency_percentiles_usec_ping:p50=1.003,p99=2.007,p99.9=5.023 latency_percentiles_usec_config|get:p50=154.623,p99=209.919,p99.9=270.335 latency_percentiles_usec_info:p50=240.639,p99=356.351,p99.9=413.695 latency_percentiles_usec_search.CLUSTERREFRESH:p50=69.119,p99=93.183,p99.9=123.391 latency_percentiles_usec_json.set:p50=305.151,p99=481.279,p99.9=516.095 latency_percentiles_usec_set:p50=12.031,p99=23.039,p99.9=23.039 latency_percentiles_usec_client|setname:p50=2.007,p99=11.007,p99.9=12.031 latency_percentiles_usec_FT.SEARCH:p50=1441.791,p99=109576.191,p99.9=109576.191 latency_percentiles_usec_auth:p50=4.015,p99=13.055,p99.9=15.039 latency_percentiles_usec_latency|latest:p50=2.007,p99=4.015,p99.9=10.047 latency_percentiles_usec_latency|histogram:p50=303.103,p99=614.399,p99.9=765.951 latency_percentiles_usec_cluster|info:p50=49.151,p99=63.231,p99.9=94.207 latency_percentiles_usec_cluster|slots:p50=30.079,p99=52.223,p99.9=69.119 latency_percentiles_usec_cluster|myid:p50=2.007,p99=4.015,p99.9=7.007 latency_percentiles_usec_del:p50=34.047,p99=75.263,p99.9=75.263 latency_percentiles_usec_get:p50=3.007,p99=6.015,p99.9=6.015 latency_percentiles_usec_psync:p50=2703.359,p99=2703.359,p99.9=2703.359 latency_percentiles_usec_slowlog|len:p50=1.003,p99=9.023,p99.9=21.119 latency_percentiles_usec_slowlog|get:p50=4.015,p99=8.031,p99.9=11.007 latency_percentiles_usec__FT.SEARCH:p50=164.863,p99=17825.791,p99.9=17825.791

Cluster

cluster_enabled:1

Keyspace

db0:keys=39729,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------ id=78 addr=10.238.160.30:50470 laddr=10.12.72.13:6379 fd=43 name= age=5379 idle=26 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=85 addr=10.238.160.39:47864 laddr=10.12.72.13:6379 fd=58 name= age=5377 idle=32 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=52 addr=10.12.72.110:60022 laddr=10.12.72.13:6379 fd=47 name= age=5400 idle=39 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=15 addr=10.12.72.140:44872 laddr=10.12.72.13:6379 fd=20 name= age=5406 idle=5406 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2 id=84 addr=10.12.72.128:38714 laddr=10.12.72.13:6379 fd=57 name= age=5377 idle=16 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=40 addr=10.12.72.110:60006 laddr=10.12.72.13:6379 fd=45 name= age=5403 idle=1 flags=S db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=1 omem=20504 tot-mem=42776 events=r cmd=replconf user=default redir=-1 resp=2 id=47 addr=10.12.72.13:38892 laddr=10.12.72.13:6379 fd=49 name= age=5402 idle=6 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=17 addr=10.12.72.110:42146 laddr=10.12.72.13:6379 fd=22 name= age=5406 idle=5406 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2 id=18 addr=10.12.72.128:49302 laddr=10.12.72.13:6379 fd=23 name= age=5406 idle=5406 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2 id=4724 addr=10.12.72.129:34730 laddr=10.12.72.13:6379 fd=34 name= age=1298 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=30 addr=10.12.72.13:38872 laddr=10.12.72.13:6379 fd=25 name= age=5405 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=39 addr=10.12.72.13:38882 laddr=10.12.72.13:6379 fd=44 name= age=5403 idle=208 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=_FT.SEARCH user=default redir=-1 resp=2 id=89 addr=10.240.160.17:42132 laddr=10.12.72.13:6379 fd=46 name= age=5375 idle=72 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=json.set user=default redir=-1 resp=2 id=32 addr=10.12.72.140:44874 laddr=10.12.72.13:6379 fd=27 name= age=5404 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=5867 addr=10.12.72.127:40822 laddr=10.12.72.13:6379 fd=48 name= age=294 idle=294 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2 id=100 addr=10.238.160.33:54140 laddr=10.12.72.13:6379 fd=60 name= age=5366 idle=4 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=33 addr=10.12.72.128:49314 laddr=10.12.72.13:6379 fd=28 name= age=5404 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=53 addr=10.12.72.127:52994 laddr=10.12.72.13:6379 fd=51 name= age=5400 idle=33 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=4721 addr=10.12.72.127:35664 laddr=10.12.72.13:6379 fd=40 name= age=1298 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=66 addr=10.12.72.129:56672 laddr=10.12.72.13:6379 fd=53 name= age=5388 idle=1 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=5968 addr=10.12.72.129:46220 laddr=10.12.72.13:6379 fd=56 name= age=205 idle=205 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2 id=36 addr=10.12.72.110:42170 laddr=10.12.72.13:6379 fd=31 name= age=5404 idle=2 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2 id=76 addr=10.238.160.24:57554 laddr=10.12.72.13:6379 fd=33 name= age=5380 idle=30 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=69 addr=10.238.160.36:44966 laddr=10.12.72.13:6379 fd=54 name= age=5386 idle=71 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=49 addr=10.240.96.118:42605 laddr=10.12.72.13:6379 fd=50 name= age=5401 idle=93 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=json.set user=default redir=-1 resp=2 id=118 addr=10.12.72.140:42296 laddr=10.12.72.13:6379 fd=61 name= age=5349 idle=25 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2 id=162 addr=10.238.160.31:58444 laddr=10.12.72.13:6379 fd=62 name= age=5313 idle=39 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2

------ MODULES INFO OUTPUT ------

ReJSON_trace

ReJSON_trace: 0: redis_module::base_info_func 1: rejson::__info_func 2: modulesCollectInfo at /redis-7.0.3/src/module.c:9383:9 3: logModulesInfo at /redis-7.0.3/src/debug.c:1744:22 4: printCrashReport at /redis-7.0.3/src/debug.c:2000:5 _serverPanic at /redis-7.0.3/src/debug.c:1107:9 5: rdbSaveObjectType at /redis-7.0.3/src/rdb.c:682:13 6: rdbSaveKeyValuePair at /redis-7.0.3/src/rdb.c:1125:9 7: rdbSaveDb at /redis-7.0.3/src/rdb.c:1286:20 8: rdbSaveRio at /redis-7.0.3/src/rdb.c:1342:17 9: rdbSave at /redis-7.0.3/src/rdb.c:1421:9 10: rdbSaveBackground at /redis-7.0.3/src/rdb.c:1481:18 11: serverCron at /redis-7.0.3/src/server.c:1314:17 12: processTimeEvents at /redis-7.0.3/src/ae.c:327:22 aeProcessEvents at /redis-7.0.3/src/ae.c:466:22 13: aeMain at /redis-7.0.3/src/ae.c:496:9 14: main at /redis-7.0.3/src/server.c:7066:5 15: 16: __libc_start_main 17: _start

search_version

search_version:2.6.12 search_redis_version:7.0.3 - oss

search_index

search_number_of_indexes:2

search_fields_statistics

search_fields_text:Text=38 search_fields_numeric:Numeric=2

search_dialect_statistics

search_dialect_1:1 search_dialect_2:0 search_dialect_3:0

search_runtime_configurations

search_concurrent_mode:OFF search_enableGC:ON search_minimal_term_prefix:2 search_maximal_prefix_expansions:200 search_query_timeout_ms:500 search_timeout_policy:return search_cursor_read_size:1000 search_cursor_max_idle_time:300000 search_max_doc_table_size:1000000 search_max_search_results:1000000 search_max_aggregate_results:-1 search_search_pool_size:20 search_index_pool_size:8 search_gc_scan_size:100 search_min_phonetic_term_length:3

------ CONFIG DEBUG OUTPUT ------ io-threads 1 proto-max-bulk-len 512mb lazyfree-lazy-server-del no lazyfree-lazy-expire no client-query-buffer-limit 1gb activedefrag no lazyfree-lazy-eviction no replica-read-only yes lazyfree-lazy-user-flush no lazyfree-lazy-user-del no io-threads-do-reads no list-compress-depth 0 sanitize-dump-payload no repl-diskless-load disabled repl-diskless-sync no slave-read-only yes

------ FAST MEMORY TEST ------ 1639087:C 12 Sep 2024 08:50:50.196 # Bio thread for job type #0 terminated 1639087:C 12 Sep 2024 08:50:50.196 # Redis 7.0.3 crashed by signal: 11, si_code: 1 1639087:C 12 Sep 2024 08:50:50.196 # Accessing address: 0x7fc960bfe910 1639087:C 12 Sep 2024 08:50:50.196 # Crashed running the instruction at: 0x7fc96f292bdd

------ STACK TRACE ------ EIP: /lib/x86_64-linux-gnu/libc.so.6(pthread_cancel+0x1d)[0x7fc96f292bdd]

Backtrace: /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc96f242520] /lib/x86_64-linux-gnu/libc.so.6(pthread_cancel+0x1d)[0x7fc96f292bdd] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889f2cd5] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889d869e] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889d874a] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889d8d50] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889aa382] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ab1be] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889abb3a] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889abdf3] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ac1e2] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889ac585] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118897ee1a] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118896c041] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x56118896c2fd] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x5611889678c0] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fc96f229d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fc96f229e40] redis-rdb-bgsave 127.0.0.1:26379 cluster[0x561188967fa5]

------ REGISTERS ------ 1639087:C 12 Sep 2024 08:50:50.197 # RAX:160639205df54a00 RBX:0000000000000001 RCX:000000000000001c RDX:00005614e85ecb6e RDI:00007fc960bfe640 RSI:0000561189197010 RBP:00007fc96fa60040 RSP:00007fff1ce1ce30 R8 :00005611894ee6b0 R9 :000000007fffffff R10:0000000000000000 R11:94106fc7aba441b3 R12:0000561188bf8920 R13:0000561188b16628 R14:0000561188b16660 R15:000000006dadd5c8 RIP:00007fc96f292bdd EFL:0000000000010206 CSGSFS:002b000000000033 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3f) -> 0000000000000000 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3e) -> 0000000600000007 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3d) -> 160639205df54a00 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3c) -> 0000000600000007 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3b) -> 0000000000000000 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce3a) -> 0000000000000000 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce39) -> 00007fc9613ffc68 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce38) -> 00007fc96f2964d0 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce37) -> 00007fff1ce1ce70 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce36) -> 00007fc96e2e0300 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce35) -> 00007fc96f29659c 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce34) -> 0000000000000000 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce33) -> 00007fc9613ff640 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce32) -> 00007fc96ede0905 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce31) -> 00005611893d9d40 1639087:C 12 Sep 2024 08:50:50.197 # (00007fff1ce1ce30) -> 00007fc958000b70

------ INFO OUTPUT ------

Server

redis_version:7.0.3 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:76796e0f0ae09c8e redis_mode:cluster os:Linux 6.2.0-1018-azure x86_64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:11.4.0 process_id:1639087 process_supervised:systemd run_id:0eaac0d3b5bc8e2198623057401fc8394080b488 tcp_port:26379 server_time_usec:1726131050054149 uptime_in_seconds:5406 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:14855018 executable:/usr/local/bin/redis-server config_file:/etc/redis/redis.conf io_threads_active:0

Clients

connected_clients:26 cluster_connections:10 maxclients:25000 client_recent_max_input_buffer:20480 client_recent_max_output_buffer:20504 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory

used_memory:93983320 used_memory_human:89.63M used_memory_rss:151900160 used_memory_rss_human:144.86M used_memory_peak:93983320 used_memory_peak_human:89.63M used_memory_peak_perc:102.00% used_memory_overhead:5792132 used_memory_startup:2757752 used_memory_dataset:88191188 used_memory_dataset_perc:96.67% allocator_allocated:91110136 allocator_active:127995904 allocator_resident:132538368 total_system_memory:8315179008 total_system_memory_human:7.74G used_memory_lua:31744 used_memory_vm_eval:31744 used_memory_lua_human:31.00K used_memory_scripts_eval:0 number_of_cached_scripts:0 number_of_functions:0 number_of_libraries:0 used_memory_vm_functions:32768 used_memory_vm_total:64512 used_memory_vm_total_human:63.00K used_memory_functions:184 used_memory_scripts:184 used_memory_scripts_human:184B maxmemory:4572839936 maxmemory_human:4.26G maxmemory_policy:allkeys-lru allocator_frag_ratio:1.40 allocator_frag_bytes:36885768 allocator_rss_ratio:1.04 allocator_rss_bytes:4542464 rss_overhead_ratio:1.15 rss_overhead_bytes:19361792 mem_fragmentation_ratio:1.71 mem_fragmentation_bytes:62976992 mem_not_counted_for_evict:2048 mem_replication_backlog:82020 mem_total_replication_buffers:82016 mem_clients_slaves:0 mem_clients_normal:190104 mem_cluster_links:10880 mem_aof_buffer:2048 mem_allocator:jemalloc-5.2.1 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence

loading:0 async_loading:0 current_cow_peak:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:39 rdb_bgsave_in_progress:0 rdb_last_save_time:1726129249 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_saves:4 rdb_last_cow_size:2117632 rdb_last_load_keys_expired:0 rdb_last_load_keys_loaded:2467 aof_enabled:1 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_rewrites:0 aof_rewrites_consecutive_failures:0 aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0 aof_current_size:23017543 aof_base_size:648737 aof_pending_rewrite:0 aof_buffer_length:0 aof_pending_bio_fsync:0 aof_delayed_fsync:0

Stats

total_connections_received:516 total_commands_processed:44809 instantaneous_ops_per_sec:9 total_net_input_bytes:723429 total_net_output_bytes:30086510 total_net_repl_input_bytes:0 total_net_repl_output_bytes:13725432 instantaneous_input_kbps:0.11 instantaneous_output_kbps:0.31 instantaneous_input_repl_kbps:0.00 instantaneous_output_repl_kbps:0.00 rejected_connections:0 sync_full:1 sync_partial_ok:0 sync_partial_err:1 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:55 evicted_keys:0 evicted_clients:0 total_eviction_exceeded_time:0 current_eviction_exceeded_time:0 keyspace_hits:93822 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 pubsubshard_channels:0 latest_fork_usec:2676 total_forks:4 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 total_active_defrag_time:0 current_active_defrag_time:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:33 dump_payload_sanitizations:0 total_reads_processed:29019 total_writes_processed:23865 io_threaded_reads_processed:0 io_threaded_writes_processed:0 reply_buffer_shrinks:65 reply_buffer_expands:21

Replication

role:master connected_slaves:1 slave0:ip=10.12.72.110,port=6379,state=online,offset=71486,lag=1 master_failover_state:no-failover master_replid:5b1ce70e0033ec292cd0b68167bfea1637b8ecb8 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:71486 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:415236096 repl_backlog_first_byte_offset:1 repl_backlog_histlen:71486

..........

------ MODULES INFO OUTPUT ------

ReJSON_trace

ReJSON_trace: 0: redis_module::base_info_func 1: rejson::__info_func 2: modulesCollectInfo at /redis-7.0.3/src/module.c:9383:9 3: logModulesInfo at /redis-7.0.3/src/debug.c:1744:22 4: printCrashReport at /redis-7.0.3/src/debug.c:2000:5 sigsegvHandler at /redis-7.0.3/src/debug.c:1982:5 5: 6: InvertedIndex_WriteForwardIndexEntry 7: Indexer_Process 8: Indexer_Add 9: Document_AddToIndexes 10: IndexSpec_UpdateDoc.part.0 11: Indexes_UpdateMatchingWithSchemaRules 12: HashNotificationCallback 13: moduleNotifyKeyspaceEvent at /redis-7.0.3/src/module.c:7877:13 14: notifyKeyspaceEvent at /redis-7.0.3/src/notify.c:114:6 15: RM_NotifyKeyspaceEvent at /redis-7.0.3/src/module.c:7846:5 16: redis_module::raw::notify_keyspace_event 17: rejson::commands::json_set 18: rejson::RedisModule_OnLoad::__do_command 19: RedisModuleCommandDispatcher at /redis-7.0.3/src/module.c:791:5 20: call at /redis-7.0.3/src/server.c:3319:5 21: processCommand at /redis-7.0.3/src/server.c:3948:9 22: processCommandAndResetClient at /redis-7.0.3/src/networking.c:2444:9 processInputBuffer at /redis-7.0.3/src/networking.c:2548:17 23: readQueryFromClient at /redis-7.0.3/src/networking.c:2684:9 24: callHandler at /redis-7.0.3/src/connhelpers.h:79:18 tlsHandleEvent at /redis-7.0.3/src/tls.c:674:22 25: aeProcessEvents at /redis-7.0.3/src/ae.c:436:17 26: aeMain at /redis-7.0.3/src/ae.c:496:9 27: main at /redis-7.0.3/src/server.c:7066:5 28: 29: __libc_start_main 30: _start

search_version

search_version:2.6.12 search_redis_version:7.0.3 - oss

search_index

search_number_of_indexes:2

search_fields_statistics

search_fields_text:Text=38 search_fields_numeric:Numeric=2

search_dialect_statistics

search_dialect_1:1 search_dialect_2:0 search_dialect_3:0

search_runtime_configurations

search_concurrent_mode:OFF search_enableGC:ON search_minimal_term_prefix:2 search_maximal_prefix_expansions:200 search_query_timeout_ms:500 search_timeout_policy:return search_cursor_read_size:1000 search_cursor_max_idle_time:300000 search_max_doc_table_size:1000000 search_max_search_results:1000000 search_max_aggregate_results:-1 search_search_pool_size:20 search_index_pool_size:8 search_gc_scan_size:100 search_min_phonetic_term_length:3

------ CONFIG DEBUG OUTPUT ------ io-threads 1 proto-max-bulk-len 512mb lazyfree-lazy-server-del no lazyfree-lazy-expire no client-query-buffer-limit 1gb activedefrag no lazyfree-lazy-eviction no replica-read-only yes lazyfree-lazy-user-flush no lazyfree-lazy-user-del no io-threads-do-reads no list-compress-depth 0 sanitize-dump-payload no repl-diskless-load disabled repl-diskless-sync no slave-read-only yes

------ FAST MEMORY TEST ------ 1634826:M 12 Sep 2024 08:51:25.745 # Bio thread for job type #0 terminated 1634826:M 12 Sep 2024 08:51:25.745 # Bio thread for job type #1 terminated 1634826:M 12 Sep 2024 08:51:25.745 # Bio thread for job type #2 terminated *** Preparing to test memory region 561188bdf000 (2306048 bytes) *** Preparing to test memory region 561189197000 (4108288 bytes) *** Preparing to test memory region 7fc944000000 (1114112 bytes) *** Preparing to test memory region 7fc94a780000 (25825280 bytes) *** Preparing to test memory region 7fc950100000 (47185920 bytes) *** Preparing to test memory region 7fc952eff000 (44040192 bytes) *** Preparing to test memory region 7fc955900000 (8388608 bytes) *** Preparing to test memory region 7fc956100000 (32641024 bytes) *** Preparing to test memory region 7fc95c000000 (4194304 bytes) *** Preparing to test memory region 7fc95c57f000 (13107200 bytes) *** Preparing to test memory region 7fc95d200000 (8388608 bytes) *** Preparing to test memory region 7fc95da00000 (4194304 bytes) *** Preparing to test memory region 7fc95dfff000 (8388608 bytes) *** Preparing to test memory region 7fc95e800000 (8388608 bytes) *** Preparing to test memory region 7fc95f000000 (2097152 bytes) *** Preparing to test memory region 7fc95f3fd000 (8388608 bytes) *** Preparing to test memory region 7fc95fbfe000 (8388608 bytes) *** Preparing to test memory region 7fc9603ff000 (8388608 bytes) *** Preparing to test memory region 7fc960c00000 (8388608 bytes) *** Preparing to test memory region 7fc9617e9000 (8388608 bytes) *** Preparing to test memory region 7fc961fea000 (8388608 bytes) *** Preparing to test memory region 7fc9627eb000 (8388608 bytes) *** Preparing to test memory region 7fc962fec000 (8388608 bytes) *** Preparing to test memory region 7fc9637ed000 (8388608 bytes) *** Preparing to test memory region 7fc963fee000 (8388608 bytes) *** Preparing to test memory region 7fc9647ef000 (8388608 bytes) *** Preparing to test memory region 7fc964ff0000 (8388608 bytes) *** Preparing to test memory region 7fc9657f1000 (8388608 bytes) *** Preparing to test memory region 7fc965ff2000 (8388608 bytes) *** Preparing to test memory region 7fc9667f3000 (8388608 bytes) *** Preparing to test memory region 7fc966ff4000 (8388608 bytes) *** Preparing to test memory region 7fc9677f5000 (8388608 bytes) *** Preparing to test memory region 7fc967ff6000 (8388608 bytes) *** Preparing to test memory region 7fc9687f7000 (8388608 bytes) *** Preparing to test memory region 7fc968ff8000 (8388608 bytes) *** Preparing to test memory region 7fc9697f9000 (8388608 bytes) *** Preparing to test memory region 7fc969ffa000 (8388608 bytes) *** Preparing to test memory region 7fc96a7fb000 (8388608 bytes) *** Preparing to test memory region 7fc96affc000 (8388608 bytes) *** Preparing to test memory region 7fc96b7fd000 (8388608 bytes) *** Preparing to test memory region 7fc96bffe000 (8388608 bytes) *** Preparing to test memory region 7fc96c7ff000 (8388608 bytes) *** Preparing to test memory region 7fc96d000000 (8388608 bytes) *** Preparing to test memory region 7fc96dd43000 (24576 bytes) *** Preparing to test memory region 7fc96de80000 (5767168 bytes) *** Preparing to test memory region 7fc96e800000 (8388608 bytes) *** Preparing to test memory region 7fc96f1ff000 (4096 bytes) *** Preparing to test memory region 7fc96f41b000 (53248 bytes) *** Preparing to test memory region 7fc96fa41000 (12288 bytes) *** Preparing to test memory region 7fc96fa5e000 (32768 bytes) *** Preparing to test memory region 7fc96fabc000 (8192 bytes) *** Preparing to test memory region 7fc96fc28000 (4096 bytes) *** Preparing to test memory region 7fc96fd17000 (8192 bytes) .O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------ Symbol: (null) (base: (nil)) Module: /usr/local/bin/redis-modules-RediSearchv2.6.12_Ubuntuv22.so (base 0x7fc96d800000) $ xxd -r -p /tmp/dump.hex /tmp/dump.bin $ objdump --adjust-vma=(nil) -D -b binary -m i386:x86-64 /tmp/dump.bin

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

   Please report the crash by opening an issue on github:

       http://github.com/redis/redis/issues

If a Redis module was involved, please open in the module's repo instead.

Suspect RAM error? Use redis-server --test-memory to verify it.

Some other issues could be detected by redis-server --check-system 1639145:C 12 Sep 2024 08:51:27.430 * Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. 1639145:C 12 Sep 2024 08:51:27.430 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 1639145:C 12 Sep 2024 08:51:27.430 # Redis version=7.0.3, bits=64, commit=00000000, modified=0, pid=1639145, just started 1639145:C 12 Sep 2024 08:51:27.430 # Configuration loaded 1639145:M 12 Sep 2024 08:51:27.436 * Increased maximum number of

Additional information

DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

Comment From: adihegde23

Entire Crash report is attacted.