=== REDIS BUG REPORT START: Cut & paste starting from here ===
30910:M 12 Aug 2019 00:55:10.575 # Redis 5.0.5 crashed by signal: 11
30910:M 12 Aug 2019 00:55:10.575 # Crashed running the instruction at: 0x4e1525
30910:M 12 Aug 2019 00:55:10.575 # Accessing address: 0x11dd0
30910:M 12 Aug 2019 00:55:10.575 # Failed assertion:
------ STACK TRACE ------ EIP: /usr/bin/redis-server *:6379(je_malloc_usable_size+0x85)[0x4e1525]
Backtrace: /usr/bin/redis-server :6379(logStackTrace+0x45)[0x473e75] /usr/bin/redis-server :6379(sigsegvHandler+0xb9)[0x474639] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7ffa201fa390] /usr/bin/redis-server :6379(je_malloc_usable_size+0x85)[0x4e1525] /usr/bin/redis-server :6379(zfree+0xe)[0x43750e] /usr/bin/redis-server :6379(rdbSave+0x26e)[0x4515de] /usr/bin/redis-server :6379(saveCommand+0x5f)[0x45416f] /usr/bin/redis-server :6379(call+0xaf)[0x43003f] /usr/bin/redis-server :6379(processCommand+0x377)[0x430707] /usr/bin/redis-server :6379(processInputBuffer+0x175)[0x440b45] /usr/bin/redis-server :6379(aeProcessEvents+0x36d)[0x429ccd] /usr/bin/redis-server :6379(aeMain+0x2b)[0x429efb] /usr/bin/redis-server :6379(main+0x4ae)[0x426aae] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7ffa1fe3f830] /usr/bin/redis-server *:6379(_start+0x29)[0x426d19]
------ INFO OUTPUT ------
Server
redis_version:5.0.5 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:96192a109b5fb62e redis_mode:standalone os:Linux 4.4.0-1088-aws x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:5.4.0 process_id:30910 run_id:29e96ab638b845e66b891da0daebf300efdd5745 tcp_port:6379 uptime_in_seconds:37887 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:5290222 executable:/usr/bin/redis-server config_file:/etc/redis/redis.conf
Clients
connected_clients:24 client_recent_max_input_buffer:2 client_recent_max_output_buffer:0 blocked_clients:0
Memory
used_memory:1462264 used_memory_human:1.39M used_memory_rss:4812800 used_memory_rss_human:4.59M used_memory_peak:1562904 used_memory_peak_human:1.49M used_memory_peak_perc:93.56% used_memory_overhead:1329704 used_memory_startup:791368 used_memory_dataset:132560 used_memory_dataset_perc:19.76% allocator_allocated:1454416 allocator_active:1822720 allocator_resident:4751360 total_system_memory:16222552064 total_system_memory_human:15.11G used_memory_lua:32768 used_memory_lua_human:32.00K used_memory_scripts:496 used_memory_scripts_human:496B number_of_cached_scripts:2 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.25 allocator_frag_bytes:368304 allocator_rss_ratio:2.61 allocator_rss_bytes:2928640 rss_overhead_ratio:1.01 rss_overhead_bytes:61440 mem_fragmentation_ratio:3.39 mem_fragmentation_bytes:3392512 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:537216 mem_aof_buffer:0 mem_allocator:jemalloc-5.1.0 active_defrag_running:0 lazyfree_pending_objects:0
Persistence
loading:0 rdb_changes_since_last_save:122 rdb_bgsave_in_progress:0 rdb_last_save_time:1565571048 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:368640 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0
Stats
total_connections_received:29 total_commands_processed:245040 instantaneous_ops_per_sec:13 total_net_input_bytes:13067558 total_net_output_bytes:1307439 instantaneous_input_kbps:0.76 instantaneous_output_kbps:0.05 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:1 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:15156 keyspace_misses:15160 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:227 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0
Replication
role:master connected_slaves:0 master_replid:969dbee204ba92baf8bc8a65b0bd1287455aafdc master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
CPU
used_cpu_sys:8.012000 used_cpu_user:8.640000 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000
Commandstats
cmdstat_info:calls=8,usec=373,usec_per_call=46.62 cmdstat_ping:calls=2,usec=0,usec_per_call=0.00 cmdstat_config:calls=4,usec=15,usec_per_call=3.75 cmdstat_smembers:calls=1,usec=1,usec_per_call=1.00 cmdstat_lpop:calls=166669,usec=138867,usec_per_call=0.83 cmdstat_hgetall:calls=7,usec=4,usec_per_call=0.57 cmdstat_expire:calls=7577,usec=11036,usec_per_call=1.46 cmdstat_get:calls=15154,usec=10471,usec_per_call=0.69 cmdstat_spop:calls=4,usec=7,usec_per_call=1.75 cmdstat_zrangebyscore:calls=7577,usec=22197,usec_per_call=2.93 cmdstat_time:calls=8845,usec=12057,usec_per_call=1.36 cmdstat_set:calls=13,usec=26,usec_per_call=2.00 cmdstat_scard:calls=7577,usec=3585,usec_per_call=0.47 cmdstat_hset:calls=8845,usec=24348,usec_per_call=2.75 cmdstat_sadd:calls=10,usec=27,usec_per_call=2.70 cmdstat_evalsha:calls=15156,usec=221512,usec_per_call=14.62 cmdstat_setnx:calls=7577,usec=14854,usec_per_call=1.96 cmdstat_auth:calls=7,usec=8,usec_per_call=1.14 cmdstat_hdel:calls=5,usec=12,usec_per_call=2.40 cmdstat_script:calls=2,usec=76,usec_per_call=38.00
Cluster
cluster_enabled:0
Keyspace
db0:keys=11,expires=1,avg_ttl=175783
Comment From: itamarhaber
Hello @shanthivardhan
Thanks for reporting this crash.
a) It looks like the crash log is missing its final part - was it not present in the log? b) Is this crash reproducible?
Comment From: xeron
Not sure if related but 5.0.6 crashes with Segmentation fault: 11 message on macOS Catalina if built with MALLOC=jemalloc. Removing MALLOC=jemalloc fixes the issue.
Comment From: mitolH
@itamarhaber HELP! the same crash info
=== REDIS BUG REPORT START: Cut & paste starting from here ===
74557:C 14 Jan 2020 07:51:40.046 # Redis 5.0.4 crashed by signal: 11
74557:C 14 Jan 2020 07:51:40.047 # Crashed running the instruction at: 0x44a54b
74557:C 14 Jan 2020 07:51:40.047 # Accessing address: 0xffffffffffffffff
74557:C 14 Jan 2020 07:51:40.047 # Failed assertion:
------ STACK TRACE ------ EIP: redis-rdb-bgsave 127.0.0.1:6379(rdbSaveObject+0x1fb)[0x44a54b]
Backtrace: redis-rdb-bgsave 127.0.0.1:6379(logStackTrace+0x41)[0x46f931] redis-rdb-bgsave 127.0.0.1:6379(sigsegvHandler+0xc9)[0x4700b9] /lib64/libpthread.so.0(+0xf5f0)[0x7fa0ebe185f0] redis-rdb-bgsave 127.0.0.1:6379(rdbSaveObject+0x1fb)[0x44a54b] redis-rdb-bgsave 127.0.0.1:6379(rdbSaveKeyValuePair+0x111)[0x44acf1] redis-rdb-bgsave 127.0.0.1:6379(rdbSaveRio+0x232)[0x44b212] redis-rdb-bgsave 127.0.0.1:6379(rdbSave+0xb4)[0x44b714] redis-rdb-bgsave 127.0.0.1:6379(rdbSaveBackground+0x15e)[0x44ba4e] redis-rdb-bgsave 127.0.0.1:6379(serverCron+0x80c)[0x42ae7c] redis-rdb-bgsave 127.0.0.1:6379(aeProcessEvents+0x360)[0x423d00] redis-rdb-bgsave 127.0.0.1:6379(aeMain+0x2b)[0x423f5b] redis-rdb-bgsave 127.0.0.1:6379(main+0x4e0)[0x420bd0] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa0eba5d505] redis-rdb-bgsave 127.0.0.1:6379[0x420e23]
------ INFO OUTPUT ------
Server
redis_version:5.0.4 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:2eba388922a6d94c redis_mode:standalone os:Linux 3.10.0-514.el7.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:4.8.5 process_id:74557 run_id:316ebb623b82a7a69f8a4489b93075ae104536cb tcp_port:6379 uptime_in_seconds:783029 uptime_in_days:9 hz:10 configured_hz:10 lru_clock:1929996 executable:/usr/bin/redis-server config_file:/etc/redis.conf
Clients
connected_clients:0 client_recent_max_input_buffer:2 client_recent_max_output_buffer:0 blocked_clients:0
Memory
used_memory:142424968 used_memory_human:135.83M used_memory_rss:149819392 used_memory_rss_human:142.88M used_memory_peak:144066640 used_memory_peak_human:137.39M used_memory_peak_perc:98.86% used_memory_overhead:790064 used_memory_startup:787728 used_memory_dataset:141634904 used_memory_dataset_perc:100.00% allocator_allocated:142527208 allocator_active:145948672 allocator_resident:149781504 total_system_memory:67387404288 total_system_memory_human:62.76G used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.02 allocator_frag_bytes:3421464 allocator_rss_ratio:1.03 allocator_rss_bytes:3832832 rss_overhead_ratio:1.00 rss_overhead_bytes:37888 mem_fragmentation_ratio:1.05 mem_fragmentation_bytes:7395544 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:0 mem_aof_buffer:0 mem_allocator:jemalloc-3.6.0 active_defrag_running:0 lazyfree_pending_objects:0
Persistence
loading:0 rdb_changes_since_last_save:5091 rdb_bgsave_in_progress:0 rdb_last_save_time:1578770170 rdb_last_bgsave_status:err rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:3334144 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0
Stats
total_connections_received:289 total_commands_processed:119348554 instantaneous_ops_per_sec:0 total_net_input_bytes:14215984722 total_net_output_bytes:40321500052 instantaneous_input_kbps:0.00 instantaneous_output_kbps:0.00 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:43734 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:107772349 keyspace_misses:4847992 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:4558 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0
Replication
role:master connected_slaves:0 master_replid:a85ce0437bcd3124177405237b1236e02108549a master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
CPU
used_cpu_sys:0.000000 used_cpu_user:0.005697 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000
Commandstats
cmdstat_zrevrangebyscore:calls=1934,usec=22417,usec_per_call=11.59 cmdstat_zremrangebyscore:calls=12857,usec=56830,usec_per_call=4.42 cmdstat_hgetall:calls=3222197,usec=31101396,usec_per_call=9.65 cmdstat_zcount:calls=38764,usec=225483,usec_per_call=5.82 cmdstat_del:calls=55210,usec=217406,usec_per_call=3.94 cmdstat_sismember:calls=6,usec=28,usec_per_call=4.67 cmdstat_zscan:calls=145,usec=1903,usec_per_call=13.12 cmdstat_zrank:calls=1926,usec=11179,usec_per_call=5.80 cmdstat_zadd:calls=17872,usec=184309,usec_per_call=10.31 cmdstat_pexpireat:calls=44538,usec=103805,usec_per_call=2.33 cmdstat_pexpire:calls=6,usec=22,usec_per_call=3.67 cmdstat_mget:calls=1902,usec=9289,usec_per_call=4.88 cmdstat_smembers:calls=3909330,usec=16213668,usec_per_call=4.15 cmdstat_expire:calls=69147,usec=375048,usec_per_call=5.42 cmdstat_zrevrange:calls=6,usec=36,usec_per_call=6.00 cmdstat_hmget:calls=23978398,usec=92445320,usec_per_call=3.86 cmdstat_set:calls=88687,usec=721826,usec_per_call=8.14 cmdstat_rpush:calls=427,usec=36770,usec_per_call=86.11 cmdstat_exists:calls=4476,usec=9696,usec_per_call=2.17 cmdstat_zrangebyscore:calls=42990,usec=342713,usec_per_call=7.97 cmdstat_get:calls=13649,usec=29525,usec_per_call=2.16 cmdstat_zrem:calls=11615,usec=99401,usec_per_call=8.56 cmdstat_sadd:calls=1924,usec=10544,usec_per_call=5.48 cmdstat_hmset:calls=1205089,usec=7489794,usec_per_call=6.22 cmdstat_zscore:calls=8248,usec=77990,usec_per_call=9.46 cmdstat_psetex:calls=386,usec=2125,usec_per_call=5.51 cmdstat_hset:calls=5191153,usec=36940034,usec_per_call=7.12 cmdstat_zrange:calls=1971,usec=5216,usec_per_call=2.65 cmdstat_lrange:calls=11748,usec=60446,usec_per_call=5.15 cmdstat_auth:calls=244,usec=525,usec_per_call=2.15 cmdstat_zincrby:calls=28595,usec=356329,usec_per_call=12.46 cmdstat_hget:calls=81380749,usec=324720775,usec_per_call=3.99 cmdstat_srem:calls=2365,usec=13627,usec_per_call=5.76
Cluster
cluster_enabled:0
Keyspace
db0:keys=14,expires=10,avg_ttl=1379401000
------ CLIENT LIST OUTPUT ------
------ REGISTERS ------ 74557:C 14 Jan 2020 07:51:40.049 # RAX:00007fa0dac72380 RBX:00000000004db2f4 RCX:0000000000000000 RDX:00007fa0dac72380 RDI:00007fa0eb0d3680 RSI:0000000000000000 RBP:00007ffd9d33dcb0 RSP:00007ffd9d33d940 R8 :00007fa0eb043ce0 R9 :0000000000000001 R10:00007fa0e0969c15 R11:0000000000000031 R12:0000000000011b9d R13:00007fa0eb1bf070 R14:00000000004db308 R15:0000000000000000 RIP:000000000044a54b EFL:0000000000010202 CSGSFS:0000000000000033 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94f) -> 00007ffd9d33dc40 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94e) -> 00007fa0eba8827d 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94d) -> 0000000000000000 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94c) -> 00007ffd9d33dc40 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94b) -> 0000000000000009 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d94a) -> 0000000000000001 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d949) -> 00007fa0eb043c00 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d948) -> ff39f2ae007f80b7 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d947) -> 00007fa0ec97d8d3 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d946) -> 00007ffd9d33dbc0 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d945) -> 00007fa0eba481a8 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d944) -> 3900000500000000 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d943) -> 0000000000000000 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d942) -> 00007fa000000005 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d941) -> 00007fa0eb0f7190 74557:C 14 Jan 2020 07:51:40.049 # (00007ffd9d33d940) -> 0000000000000009
------ FAST MEMORY TEST ------ *** Preparing to test memory region 71c000 (90112 bytes) *** Preparing to test memory region 7fa0da400000 (146800640 bytes) *** Preparing to test memory region 7fa0e32d4000 (8388608 bytes) *** Preparing to test memory region 7fa0e3ad5000 (8388608 bytes) *** Preparing to test memory region 7fa0e42d6000 (8388608 bytes) *** Preparing to test memory region 7fa0eb000000 (8388608 bytes) *** Preparing to test memory region 7fa0ebe04000 (20480 bytes) *** Preparing to test memory region 7fa0ec021000 (16384 bytes) *** Preparing to test memory region 7fa0ec966000 (4096 bytes) *** Preparing to test memory region 7fa0ecb59000 (20480 bytes) *** Preparing to test memory region 7fa0ecb85000 (8192 bytes) *** Preparing to test memory region 7fa0ecb87000 (4096 bytes) *** Preparing to test memory region 7fa0ecb8a000 (4096 bytes) .O.O.O.O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------ Symbol: rdbSaveObject (base: 0x44a350) Module: redis-rdb-bgsave 127.0.0.1:6379 (base 0x400000) $ xxd -r -p /tmp/dump.hex /tmp/dump.bin $ objdump --adjust-vma=0x44a350 -D -b binary -m i386:x86-64 /tmp/dump.bin
74557:C 14 Jan 2020 07:51:40.723 # dump of function (hexdump of 635 bytes): 4157415641554154554889fd534881ec3802000064488b042528000000488984242802000031c00fb60689c283e20f0f84db00000080fa010f84d202000080fa020f84f900000080fa030f846005000080fa040f842701000080fa060f844003000080fa050f85ae0700004c8b6608498b1c2448c74424100000000048897c2418c744242800000000c744242c0000000048c744243000000000488b3348895c2420e849e2ffff83f8ff0f845002000048984801442410498b742408488d7c2410ff531831f64889efe822e2ffff83f8ff0f8429020000488b7c2430489848014424104885ff740fe8132a0500488b7c2430e84973feff8b44242885c00f85fd010000488b442410eb0b660f1f440000e82bf2ffff488b8c24280200006448330c25280000000f85a10600004881c4380200005b5d415c415d415e415fc3669083e0f03c200f849f0500003c600f857f060000488b7e084889742408e8ff7b0200488b7424084889c24889ef488b7608e86beeffffeb9e660f1f84000000000083e0f03c500f843c0500003c200f1f000f8575060000488b7e084889742408e87cbdfdff488b7424084989c54889ef488b4608488b704848037028e830e1ffff4c63e04983fcff0f840d050000488d1dd00d09004c8d35dd0d09000f1f4400004c89efe898bdfdff4885c04889c20f84e6040000488b324c8b7a080fb64eff89ca83e20780fa040f87f70500000fb6d2486314934801daffe20f1f80000000008b56f74889efe8b5edffff4883f8ff0f847b040000410fb657ff4901c489d083e0073c040f87b30500000fb6c0496304864c01f0ffe0662e0f1f8400000000000fb756fbebbd662e0f1f8400000000000fb656fdebad662e0f1f84
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Comment From: itamarhaber
@mitolH this doesn't look like the same crash - please submit a new issue.
Comment From: mitolH
tks for your replay #6781 @itamarhaber the new issue.
Comment From: antirez
Please can you attach your Redis executable to the issue?
On Thu, Jan 16, 2020, 07:58 Mitol notifications@github.com wrote:
tks for your replay #6781 https://github.com/antirez/redis/issues/6781 @itamarhaber https://github.com/itamarhaber the new issue.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/antirez/redis/issues/6314?email_source=notifications&email_token=AAAQAYAO2I3KFIKQBFEYF43Q6AARBA5CNFSM4ILABGOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJC73OI#issuecomment-575012281, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAQAYFSPC34OAIMQZCNTGLQ6AARBANCNFSM4ILABGOA .