Crash report
Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context.
=== REDIS BUG REPORT START: Cut & paste starting from here ===
1:M 15 Oct 2021 19:55:18.339 # Redis 6.2.3 crashed by signal: 11, si_code: 1
1:M 15 Oct 2021 19:55:18.340 # Accessing address: 0xffffffffffffffc9
1:M 15 Oct 2021 19:55:18.340 # Crashed running the instruction at: 0x40040220f1
------ STACK TRACE ------
EIP:
/usr/lib/redis/modules/redisearch.so(DocTable_Get+0x41)[0x40040220f1]
Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x400217e730]
/usr/lib/redis/modules/redisearch.so(DocTable_Get+0x41)[0x40040220f1]
/usr/lib/redis/modules/redisearch.so(+0xb461d)[0x400403461d]
/usr/lib/redis/modules/redisearch.so(Indexer_Add+0x55)[0x4004034af5]
/usr/lib/redis/modules/redisearch.so(Document_AddToIndexes+0x14d)[0x4004024c6d]
/usr/lib/redis/modules/redisearch.so(AddDocumentCtx_Submit+0x116)[0x4004024e76]
/usr/lib/redis/modules/redisearch.so(IndexSpec_UpdateWithHash+0xba)[0x400405166a]
/usr/lib/redis/modules/redisearch.so(Indexes_UpdateMatchingWithSchemaRules+0x45)[0x4004051c35]
/usr/lib/redis/modules/redisearch.so(HashNotificationCallback+0x11d)[0x400403d15d]
/usr/local/bin/redis-server *:6379(moduleNotifyKeyspaceEvent+0xbe)[0x40000d570e]
/usr/local/bin/redis-server *:6379(notifyKeyspaceEvent+0x21)[0x40000c3131]
/usr/local/bin/redis-server *:6379(hsetCommand+0xfd)[0x400009016d]
/usr/local/bin/redis-server *:6379(call+0xad)[0x400004cc6d]
/usr/local/bin/redis-server *:6379(processCommand+0x5b3)[0x400004e7a3]
/usr/local/bin/redis-server *:6379(processCommandAndResetClient+0x1c)[0x4000061f3c]
/usr/local/bin/redis-server *:6379(processInputBuffer+0xea)[0x40000645aa]
/usr/local/bin/redis-server *:6379(+0xf3bd8)[0x40000f3bd8]
/usr/local/bin/redis-server *:6379(aeProcessEvents+0x2a1)[0x4000045ce1]
/usr/local/bin/redis-server *:6379(aeMain+0x1d)[0x4000045f3d]
/usr/local/bin/redis-server *:6379(main+0x314)[0x40000426f4]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x40021b109b]
/usr/local/bin/redis-server *:6379(_start+0x2a)[0x4000042bda]
------ REGISTERS ------
1:M 15 Oct 2021 19:55:18.344 #
RAX:0000000000000001 RBX:000000400be35d00
RCX:00000000000f4240 RDX:000000400bbbc8b0
RDI:000000400bac4608 RSI:0000000000000003
RBP:000000400bac4580 RSP:0000004001c17408
R8 :000000400295dc30 R9 :00000040022c85f0
R10:000000400234fd30 R11:00000040023174a0
R12:000000400bac4608 R13:0000004001c1b470
R14:000000400bac4580 R15:0000004004237108
RIP:00000040040220f1 EFL:0000000000000202
CSGSFS:002b000000000033
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17417) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17416) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17415) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17414) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17413) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17412) -> 0000000100000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17411) -> 000000400bac4580
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17410) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740f) -> 0000004001c2d6d0
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740e) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740d) -> 0000004001c19470
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740c) -> 000000400be35d00
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740b) -> 000000400be35300
1:M 15 Oct 2021 19:55:18.344 # (0000004001c1740a) -> 0000000000000020
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17409) -> 0000000000000000
1:M 15 Oct 2021 19:55:18.344 # (0000004001c17408) -> 000000400403461d
------ INFO OUTPUT ------
# Server
redis_version:6.2.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:33cdf76a1080ed11
redis_mode:standalone
os:Linux 5.10.47-linuxkit x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:8.3.0
process_id:1
process_supervised:no
run_id:356d958df5ab8dd186d5d5941ebd8b8d8c51e82b
tcp_port:6379
server_time_usec:1634327718338771
uptime_in_seconds:17
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:6937766
executable:/usr/local/bin/redis-server
config_file:
io_threads_active:0
# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:8
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
# Memory
used_memory:5509304
used_memory_human:5.25M
used_memory_rss:0
used_memory_rss_human:0B
used_memory_peak:5589208
used_memory_peak_human:5.33M
used_memory_peak_perc:98.57%
used_memory_overhead:5317928
used_memory_startup:5317816
used_memory_dataset:191376
used_memory_dataset_perc:99.94%
allocator_allocated:5823000
allocator_active:6262784
allocator_resident:9768960
total_system_memory:2085416960
total_system_memory_human:1.94G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.08
allocator_frag_bytes:439784
allocator_rss_ratio:1.56
allocator_rss_bytes:3506176
rss_overhead_ratio:0.00
rss_overhead_bytes:-9768960
mem_fragmentation_ratio:0.00
mem_fragmentation_bytes:-5444608
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:5
rdb_bgsave_in_progress:0
rdb_last_save_time:1634327701
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:3
total_commands_processed:11
instantaneous_ops_per_sec:0
total_net_input_bytes:661
total_net_output_bytes:1298
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:4
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:1
dump_payload_sanitizations:0
total_reads_processed:9
total_writes_processed:6
io_threaded_reads_processed:0
io_threaded_writes_processed:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:d1b4b16a906e9e313a22bc96d0a82726cb0d408e
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.089429
used_cpu_user:0.417337
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.080766
used_cpu_user_main_thread:0.404841
# Modules
module:name=search,ver=20012,api=1,filters=0,usedby=[],using=[],options=[]
module:name=rg,ver=10007,api=1,filters=0,usedby=[],using=[ai],options=[]
module:name=graph,ver=20411,api=1,filters=0,usedby=[],using=[],options=[]
module:name=timeseries,ver=10410,api=1,filters=0,usedby=[],using=[],options=[]
module:name=ReJSON,ver=10008,api=1,filters=0,usedby=[],using=[],options=[]
module:name=ai,ver=10003,api=1,filters=0,usedby=[rg],using=[],options=[]
module:name=bf,ver=20206,api=1,filters=0,usedby=[],using=[],options=[]
# Commandstats
cmdstat_hset:calls=2,usec=6047,usec_per_call=3023.50,rejected_calls=0,failed_calls=0
cmdstat_FT.CREATE:calls=1,usec=3164,usec_per_call=3164.00,rejected_calls=0,failed_calls=0
cmdstat_config:calls=1,usec=482,usec_per_call=482.00,rejected_calls=0,failed_calls=0
cmdstat_FT.SEARCH:calls=1,usec=4265,usec_per_call=4265.00,rejected_calls=0,failed_calls=0
cmdstat_FT.INFO:calls=2,usec=2834,usec_per_call=1417.00,rejected_calls=0,failed_calls=1
cmdstat_info:calls=4,usec=2038,usec_per_call=509.50,rejected_calls=0,failed_calls=0
# Errorstats
errorstat_Unknown:count=1
# Cluster
cluster_enabled:0
# Keyspace
db0:keys=2,expires=0,avg_ttl=0
------ CLIENT LIST OUTPUT ------
id=17 addr=192.168.16.1:60870 laddr=192.168.16.2:6379 fd=13 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=108 qbuf-free=40846 argv-mem=66 obl=4 oll=0 omem=0 tot-mem=61554 events=r cmd=hset user=default redir=-1
------ CURRENT CLIENT INFO ------
id=17 addr=192.168.16.1:60870 laddr=192.168.16.2:6379 fd=13 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=108 qbuf-free=40846 argv-mem=66 obl=4 oll=0 omem=0 tot-mem=61554 events=r cmd=hset user=default redir=-1
argv[0]: 'HSET'
argv[1]: 'org:1'
argv[2]: 'name'
argv[3]: 'Dropcontact SAS'
argv[4]: 'keys'
argv[5]: 'dropcontact.io www.dropcontact.com'
1:M 15 Oct 2021 19:55:18.347 # key 'org:1' found in DB containing the following object:
1:M 15 Oct 2021 19:55:18.347 # Object type: 4
1:M 15 Oct 2021 19:55:18.347 # Object encoding: 5
1:M 15 Oct 2021 19:55:18.347 # Object refcount: 1
------ MODULES INFO OUTPUT ------
# rg
rg_nexecutions:0
rg_nregistrations:0
# rg_regisrations
# rg_python_stats
rg_TotalAllocated:10561476
rg_PeakAllocated:4043256
rg_CurrAllocated:3733377
# rg_python_requirements
# graph_executing commands
# ai_git
ai_git_sha:7f808a934dff121e188cb76fdfcc3eb1f9ec7cbf
# ai_load_time_configs
ai_threads_per_queue:1
ai_inter_op_parallelism:0
ai_intra_op_parallelism:0
------ FAST MEMORY TEST ------
1:M 15 Oct 2021 19:55:18.352 # Bio thread for job type #0 terminated
1:M 15 Oct 2021 19:55:18.352 # Bio thread for job type #1 terminated
1:M 15 Oct 2021 19:55:18.352 # Bio thread for job type #2 terminated
*** Preparing to test memory region 4000201000 (2682880 bytes)
*** Preparing to test memory region 4001c57000 (12288 bytes)
*** Preparing to test memory region 4001de0000 (8192 bytes)
*** Preparing to test memory region 4002168000 (16384 bytes)
*** Preparing to test memory region 4002189000 (16384 bytes)
*** Preparing to test memory region 400234a000 (40960 bytes)
*** Preparing to test memory region 4002600000 (2097152 bytes)
*** Preparing to test memory region 4002954000 (2097152 bytes)
*** Preparing to test memory region 4003000000 (4194304 bytes)
*** Preparing to test memory region 400377e000 (4096 bytes)
*** Preparing to test memory region 4003780000 (8388608 bytes)
*** Preparing to test memory region 4004235000 (12288 bytes)
*** Preparing to test memory region 4004239000 (8388608 bytes)
*** Preparing to test memory region 40052b6000 (12288 bytes)
*** Preparing to test memory region 40052ee000 (8388608 bytes)
*** Preparing to test memory region 4005aef000 (8388608 bytes)
*** Preparing to test memory region 40062f0000 (8388608 bytes)
*** Preparing to test memory region 4006af1000 (8388608 bytes)
*** Preparing to test memory region 40072f2000 (8388608 bytes)
*** Preparing to test memory region 4007af3000 (8388608 bytes)
*** Preparing to test memory region 40082f4000 (8388608 bytes)
*** Preparing to test memory region 4009175000 (143360 bytes)
*** Preparing to test memory region 40091aa000 (8388608 bytes)
*** Preparing to test memory region 40099ab000 (8388608 bytes)
*** Preparing to test memory region 400a1ac000 (8388608 bytes)
*** Preparing to test memory region 400a9ad000 (8388608 bytes)
*** Preparing to test memory region 400b1ae000 (8388608 bytes)
*** Preparing to test memory region 400b9ae000 (5767168 bytes)
*** Preparing to test memory region 400bf45000 (4096 bytes)
*** Preparing to test memory region 400bf63000 (8388608 bytes)
*** Preparing to test memory region 400c764000 (8388608 bytes)
*** Preparing to test memory region 400cf65000 (8388608 bytes)
*** Preparing to test memory region 400d766000 (8388608 bytes)
*** Preparing to test memory region 400df67000 (8388608 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: DocTable_Get (base: 0x40040220b0)
Module: /usr/lib/redis/modules/redisearch.so (base 0x4003f80000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x40040220b0 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
1:M 15 Oct 2021 19:55:19.226 # dump of function (hexdump of 193 bytes):
31c04885f67429483b77107723488b4f0889f24839ce733889d2483b5718734848c1e20448035730488b024885c07511c30f1f8000000000488b40084885c0741f483b70c875f14883e838c30f1f40004889f031d248f7f1ebbe660f1f440000c30f1f800000000031c0c30f1f44000031c04885f6745348397710724d488b4f0889f24839ce72084889f031d248f7f189d231c0483b5718733048c1e204480357307426488b124885d2750deb1c6690488b52084885d27417483972c875f1f642
Additional information
- Mac OS / ARM Chip - with redislabs/redismod docker
- Add some data, create a simple index.
- Query once: it's ok - Query twice: it crashes
Comment From: itamarhaber
Hi @patoch
As this is a RediSearch issue, kindly close it and resubmit against https://github.com/redisearch/redisearch
Comment From: oranagra
Wow, 7 different modules. redis search is the one that crashed, but maybe it's due to that mix. When submitting a bug report in https://github.com/RediSearch/RediSearch pease be sure to turn their attention to your configuration.
Comment From: patoch
I will post it to the RediSearch forum and will turn off unused modules. Many thanks,