Hi,

I observed the following crash when running RedisGraph on my cluster.


redis[105815]: #012#012=== REDIS BUG REPORT START: Cut & paste starting from here ===
redis[105815]: Redis 5.0.7 crashed by signal: 11
redis[105815]: Crashed running the instruction at: 0x498458
redis[105815]: Accessing address: 0x595245555136
redis[105815]: Failed assertion: <no assertion failed> (<no file>:0)
redis[105815]: #012------ STACK TRACE ------
redis[105815]: #012------ INFO OUTPUT ------
redis[105815]: # Server#015#012redis_version:5.0.7#015#012redis_git_sha1:00000000#015#012redis_git_dirty:0#015#012redis_build_id:b146fde079fc675b#015#012redis_mode:cluster#015#012os:Linux 3.10.0-514.16.1.el7.x86_64 x86_64#015#012arch_bits:64#015#012multiplexing_api:epoll#015#012atomicvar_api:atomic-builtin#015#012gcc_version:6.3.1#015#012process_id:105815#015#012run_id:0eaddea2c38e1238e3a3ab7623c5b93616352ce3#015#012tcp_port:7000#015#012uptime_in_seconds:18448#015#012uptime_in_days:0#015#012hz:10#015#012configured_hz:10#015#012lru_clock:7293965#015#012executable:/opt/redis-cluster/7000/./redis-server#015#012config_file:/opt/redis-cluster/7000/./redis.conf#015#012#015#012# Clients#015#012connected_clients:9#015#012client_recent_max_input_buffer:814002#015#012client_recent_max_output_buffer:0#015#012blocked_clients:0#015#012#015#012# Memory#015#012used_memory:41847792#015#012used_memory_human:39.91M#015#012used_memory_rss:110915584#015#012used_memory_rss_human:105.78M#015#012used_memory_peak:43823736#015#012used_memory_peak_human:41.79M#015#012used_memory_peak_perc:95.49%#015#012used_memory_overhead:2944392#015#012used_memory_startup:1496584#015#012used_memory_dataset:38903400#015#012used_memory_dataset_perc:96.41%#015#012allocator_allocated:44707416#015#012allocator_active:49094656#015#012allocator_resident:54784000#015#012total_system_memory:8186183680#015#012total_system_memory_human:7.62G#015#012used_memory_lua:37888#015#012used_memory_lua_human:37.00K#015#012used_memory_scripts:0#015#012used_memory_scripts_human:0B#015#012number_of_cached_scripts:0#015#012maxmemory:0#015#012maxmemory_human:0B#015#012maxmemory_policy:noeviction#015#012allocator_frag_ratio:1.10#015#012allocator_frag_bytes:4387240#015#012allocator_rss_ratio:1.12#015#012allocator_rss_bytes:5689344#015#012rss_overhead_ratio:2.02#015#012rss_overhead_bytes:56131584#015#012mem_fragmentation_ratio:2.53#015#012mem_fragmentation_bytes:67091848#015#012mem_not_counted_for_evict:0#015#012mem_replication_backlog:1048576#015#012mem_clients_slaves:16922#015#012mem_clients_normal:381702#015#012mem_aof_buffer:0#015#012mem_allocator:jemalloc-5.1.0#015#012active_defrag_running:0#015#012lazyfree_pending_objects:0#015#012#015#012# Persistence#015#012loading:0#015#012rdb_changes_since_last_save:8682#015#012rdb_bgsave_in_progress:0#015#012rdb_last_save_time:1584352109#015#012rdb_last_bgsave_status:ok#015#012rdb_last_bgsave_time_sec:0#015#012rdb_current_bgsave_time_sec:-1#015#012rdb_last_cow_size:31518720#015#012aof_enabled:0#015#012aof_rewrite_in_progress:0#015#012aof_rewrite_scheduled:0#015#012aof_last_rewrite_time_sec:-1#015#012aof_current_rewrite_time_sec:-1#015#012aof_last_bgrewrite_status:ok#015#012aof_last_write_status:ok#015#012aof_last_cow_size:0#015#012#015#012# Stats#015#012total_connections_received:47#015#012total_commands_processed:129018#015#012instantaneous_ops_per_sec:122#015#012total_net_input_bytes:39109065#015#012total_net_output_bytes:52938616#015#012instantaneous_input_kbps:107.76#015#012instantaneous_output_kbps:45.88#015#012rejected_connections:0#015#012sync_full:1#015#012sync_partial_ok:0#015#012sync_partial_err:1#015#012expired_keys:0#015#012expired_stale_perc:0.00#015#012expired_time_cap_reached_count:0#015#012evicted_keys:0#015#012keyspace_hits:212151#015#012keyspace_misses:57#015#012pubsub_channels:0#015#012pubsub_patterns:0#015#012latest_fork_usec:4441#015#012migrate_cached_sockets:0#015#012slave_expires_tracked_keys:0#015#012active_defrag_hits:0#015#012active_defrag_misses:0#015#012active_defrag_key_hits:0#015#012active_defrag_key_misses:0#015#012#015#012# Replication#015#012role:master#015#012connected_slaves:1#015#012slave0:ip=10.78.241.31,port=7002,state=online,offset=31279259,lag=1#015#012master_replid:4b1312143868e2306dd3c9657a73f9f56f220d9b#015#012master_replid2:0000000000000000000000000000000000000000#015#012master_repl_offset:31302977#015#012second_repl_offset:-1#015#012repl_backlog_active:1#015#012repl_backlog_size:1048576#015#012repl_backlog_first_byte_offset:30254402#015#012repl_backlog_histlen:1048576#015#012#015#012# CPU#015#012used_cpu_sys:19.411186#015#012used_cpu_user:77.020254#015#012used_cpu_sys_children:0.176951#015#012used_cpu_user_children:0.987104#015#012#015#012# Commandstats#015#012cmdstat_monitor:calls=2,usec=2,usec_per_call=1.00#015#012cmdstat_slowlog:calls=1,usec=17,usec_per_call=17.00#015#012cmdstat_command:calls=1,usec=1022,usec_per_call=1022.00#015#012cmdstat_info:calls=1978,usec=130447,usec_per_call=65.95#015#012cmdstat_scan:calls=35,usec=292,usec_per_call=8.34#015#012cmdstat_ping:calls=60,usec=57,usec_per_call=0.95#015#012cmdstat_psync:calls=1,usec=1586,usec_per_call=1586.00#015#012cmdstat_exists:calls=28,usec=84,usec_per_call=3.00#015#012cmdstat_del:calls=2,usec=73,usec_per_call=36.50#015#012cmdstat_flushall:calls=1,usec=4497,usec_per_call=4497.00#015#012cmdstat_replconf:calls=18367,usec=31019,usec_per_call=1.69#015#012cmdstat_type:calls=5,usec=4,usec_per_call=0.80#015#012cmdstat_cluster:calls=2402,usec=691572,usec_per_call=287.92#015#012cmdstat_echo:calls=2,usec=3,usec_per_call=1.50#015#012cmdstat_client:calls=35,usec=506,usec_per_call=14.46#015#012cmdstat_keys:calls=2,usec=18,usec_per_call=9.00#015#012cmdstat_graph.QUERY:calls=106088,usec=1937412,usec_per_call=18.26#015#012cmdstat_unlink:calls=8,usec=238,usec_per_call=29.75#015#012#015#012# Cluster#015#012cluster_enabled:1#015#012#015#012# Keyspace#015#012db0:keys=12,expires=0,avg_ttl=0
redis[105815]: #012------ CLIENT LIST OUTPUT ------
redis[105815]: id=4 addr=10.78.241.31:56656 fd=22 name= age=18408 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=replconf#012id=126314 addr=10.78.185.163:35978 fd=23 name=RedisInsight age=889 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=cluster#012id=211820 addr=172.21.31.106:53289 fd=32 name= age=2 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=210077 addr=172.21.31.106:53285 fd=29 name= age=12 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=126315 addr=10.78.185.163:35986 fd=24 name= age=889 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=info#012id=160250 addr=172.21.31.106:53199 fd=28 name= age=434 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=160047 addr=172.21.31.106:53198 fd=25 name= age=435 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=210761 addr=172.21.31.106:53287 fd=31 name= age=8 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=210714 addr=172.21.31.106:53286 fd=30 name= age=8 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY#012id=126318 addr=172.21.31.106:53125 fd=27 name= age=846 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=graph.QUERY
redis[105815]: #012------ REGISTERS ------
redis[105815]: #012RAX:00007f7e775d53a8 RBX:00007f7e776fbef8#012RCX:00007f7e40000000 RDX:0000000000000000#012RDI:000059524555512e RSI:0000000000000000#012RBP:00007f7e7e253c30 RSP:00007f7e758d1508#012R8 :000000000000001a R9 :0000000000000018#012R10:0000000000000002 R11:0000000000000000#012R12:00007f7e5c2016f0 R13:00007f7e72e2d600#012R14:00007f7e7e232b90 R15:0000000000000000#012RIP:0000000000498458 EFL:0000000000010246#012CSGSFS:0000000000000033
redis[105815]: (00007f7e758d1517) -> 702d646165726874
redis[105815]: (00007f7e758d1516) -> 00007f7e76ceb2c4
redis[105815]: (00007f7e758d1515) -> 0000000001ec79d8
redis[105815]: (00007f7e758d1514) -> 0000000001ec7a08
redis[105815]: (00007f7e758d1513) -> 0000000001ec79b0
redis[105815]: (00007f7e758d1512) -> 0000000001ec79a0
redis[105815]: (00007f7e758d1511) -> 0000000001ec7a88
redis[105815]: (00007f7e758d1510) -> 0000000001ec9cb0
redis[105815]: (00007f7e758d150f) -> 0000000000000000
redis[105815]: (00007f7e758d150e) -> 0000000000039019
redis[105815]: (00007f7e758d150d) -> 00000000000000b0
redis[105815]: (00007f7e758d150c) -> 00007f7e775d53c8
redis[105815]: (00007f7e758d150b) -> 00000000000000b0
redis[105815]: (00007f7e758d150a) -> 00007f7e584b3290
redis[105815]: (00007f7e758d1509) -> 00007f7e71207078
redis[105815]: (00007f7e758d1508) -> 00007f7e76cb7593
redis[105815]: #012------ FAST MEMORY TEST ------
redis[105815]: Bio thread for job type #0 terminated
redis[105815]: Bio thread for job type #1 terminated
redis[105815]: Bio thread for job type #2 terminated
redis[105815]: Redis 5.0.7 crashed by signal: 7
redis[105815]: Crashed running the instruction at: 0x42cd12
redis[105815]: Accessing address: (nil)
redis[105815]: Failed assertion: <no assertion failed> (<no file>:0)
redis[105815]: #012------ STACK TRACE ------
redis[105815]: #012------ INFO OUTPUT ------
kernel: redis-server[105815]: segfault at 9f ip 0000000000444c35 sp 00007fff595a6a90 error 4 in redis-server[400000+189000]
abrt-hook-ccpp: Process 105815 (redis-server) of user 0 killed by SIGSEGV - dumping core
abrt-server: Executable '/opt/redis-cluster/7000/redis-server' doesn't belong to any package and ProcessUnpackaged is set to 'no'
abrt-server: 'post-create' on '/var/spool/abrt/ccpp-2020-03-16-15:21:09-105815' exited with 1
abrt-server: Deleting problem directory '/var/spool/abrt/ccpp-2020-03-16-15:21:09-105815'
redis[121686]: Marking node 59b4a8c31af1b293fbfef393dca6ebc7876cf7a1 as failing (quorum reached).
redis[121686]: Cluster state changed: fail
redis[105847]: Failover auth denied to f074c400d6789f1980083d44fe3eb69f4000b476: its master is up
redis[105847]: Marking node 59b4a8c31af1b293fbfef393dca6ebc7876cf7a1 as failing (quorum reached).
redis[105847]: Cluster state changed: fail
redis[105847]: Failover auth granted to f074c400d6789f1980083d44fe3eb69f4000b476 for epoch 11
redis[121686]: Cluster state changed: ok

Comment From: antirez

Hello, please report on the RedisGraph repository. Anyway the stack trace is missing unfortunately, so very hard to understand what this could be. Not actionable here, closing.

Comment From: DvirDukhan

@adityaarun I opened an issue for you at https://github.com/RedisGraph/RedisGraph/issues/1024 Let's take it there please.