While debugging something I found this bug:
If you disconnect the client (redis-cli in this case) before DEBUG RELOAD finished, redis crashes.
(PS it's in Mac OS, didn't test on Linux)
Stack trace:
26836:M 29 Mar 15:33:13.327 * DB saved on disk
26836:M 29 Mar 15:33:14.597 # DB reloaded by DEBUG RELOAD
=== REDIS BUG REPORT START: Cut & paste starting from here ===
26836:M 29 Mar 15:33:14.597 # Redis 4.0.8 crashed by signal: 11
26836:M 29 Mar 15:33:14.597 # Crashed running the instruction at: 0x1080249d8
26836:M 29 Mar 15:33:14.597 # Accessing address: 0x8
26836:M 29 Mar 15:33:14.597 # Failed assertion: <no assertion failed> (<no file>:0)
------ STACK TRACE ------
EIP:
0 redis-server 0x00000001080249d8 call + 376
Backtrace:
0 redis-server 0x000000010806d34e logStackTrace + 110
1 redis-server 0x000000010806d70c sigsegvHandler + 236
2 libsystem_platform.dylib 0x00007fff709a8f5a _sigtramp + 26
3 ??? 0x0000000000000000 0x0 + 0
4 redis-server 0x00000001080251a9 processCommand + 1305
5 redis-server 0x0000000108035502 processInputBuffer + 322
6 redis-server 0x000000010801d031 aeProcessEvents + 689
7 redis-server 0x000000010801d31b aeMain + 43
8 redis-server 0x000000010802818f main + 1727
9 libdyld.dylib 0x00007fff70727115 start + 1
------ INFO OUTPUT ------
# Server
redis_version:4.0.8
redis_git_sha1:f17d8296
redis_git_dirty:1
redis_build_id:da88da1bb8a52bc7
redis_mode:standalone
os:Darwin 17.4.0 x86_64
arch_bits:64
multiplexing_api:kqueue
atomicvar_api:atomic-builtin
gcc_version:4.2.1
process_id:26836
run_id:a29e823ab08d7427b693529bebe1b4e8912be271
tcp_port:6379
uptime_in_seconds:62
uptime_in_days:0
hz:10
lru_clock:12377352
executable:/Users/dvirvolk/code/RediSearch/src/redis-server
config_file:
# Clients
connected_clients:0
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
# Memory
used_memory:58436304
used_memory_human:55.73M
used_memory_rss:69758976
used_memory_rss_human:66.53M
used_memory_peak:60937280
used_memory_peak_human:58.11M
used_memory_peak_perc:95.90%
used_memory_overhead:33894032
used_memory_startup:963232
used_memory_dataset:24542272
used_memory_dataset_perc:42.70%
total_system_memory:17179869184
total_system_memory_human:16.00G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.19
mem_allocator:libc
active_defrag_running:0
lazyfree_pending_objects:0
# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1522326793
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
# Stats
total_connections_received:51
total_commands_processed:950245
instantaneous_ops_per_sec:0
total_net_input_bytes:38959994
total_net_output_bytes:4764107
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
# Replication
role:master
connected_slaves:0
master_replid:c4fe79dfe4e1673048cf52bb69ebf2198ad3767b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:6.02
used_cpu_user:7.13
used_cpu_sys_children:0.00
used_cpu_user_children:0.00
# Commandstats
cmdstat_set:calls=950242,usec=1197181,usec_per_call=1.26
cmdstat_save:calls=1,usec=407585,usec_per_call=407585.00
cmdstat_info:calls=1,usec=136,usec_per_call=136.00
cmdstat_command:calls=1,usec=452,usec_per_call=452.00
# Cluster
cluster_enabled:0
# Keyspace
db0:keys=613554,expires=0,avg_ttl=0
------ CLIENT LIST OUTPUT ------
------ REGISTERS ------
26836:M 29 Mar 15:33:14.597 #
RAX:0000000000000000 RBX:0000000000000000
RCX:00000000001e8480 RDX:0000000000000000
RDI:00007ffee7be75c8 RSI:00000001080f25c8
RBP:00007ffee7be7610 RSP:00007ffee7be75b0
R8 :000000000000000f R9 :0000000000000040
R10:00007fffa975ff70 R11:ffffffffffffffff
R12:0000000000000000 R13:00000000000dbeb4
R14:00007fd919000000 R15:000000000019e1fd
RIP:00000001080249d8 EFL:0000000000010202
CS :000000000000002b FS:0000000000000000 GS:0000000000000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75bf) -> 00007fd919000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75be) -> 0000000000000006
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75bd) -> 00000001080251a9
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75bc) -> 00007ffee7be7650
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75bb) -> 000000010803570c
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75ba) -> 00007fd919000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b9) -> 0000000000000100
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b8) -> 00000001080f25c8
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b7) -> 00007fd919000098
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b6) -> 0000000f174acf63
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b5) -> 0000000000000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b4) -> 00007ffe00091c31
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b3) -> 000000005abcdd0a
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b2) -> 0000000000000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b1) -> 0000000000000000
26836:M 29 Mar 15:33:14.597 # (00007ffee7be75b0) -> 00007fd9174acf63
------ DUMPING CODE AROUND EIP ------
Symbol: call (base: 0x108024860)
Module: /usr/local/bin/redis-server (base 0x108018000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x108024860 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
26836:M 29 Mar 15:33:14.598 # dump of function (hexdump of 504 bytes):
554889e54157415641554154534883ec388975d44989fe458ba698000000488d1543dd0c00488bb21002000048837e28004489e074438b824403000085c04489e07536498b46480fb7402066a910084489e07525498b46108b5028498b4e40458b46384c89f7e845a90100488d15f6dc0c00418b869800000025ff3fe7ff41898698000000488b8228080000488b8a3008000048894db0488945a848c7822808000000000000c7823008000000000000488b8298070000488945c8488d7db831f6e8d2c30a00488b5db84c636dc0498b46484c89f7ff5008488d7db831f6e8b5c30a00488d357edc0c00488b4db84c637dc04829d9488b9e9807000083be4403000000418b86980000007413448b45d44489c283e2fcf6c401440f45c2eb04448b45d44869c940420f004d29eff6c4017435488b96f00a00004885d274296685c0790e808a9900000080418b8698000000f6c4407411488b86f00a0000818898000000004000004901cf41f6c001448945d4747c498b4648488b4808483b0d7d660c00746b488d0d46f80a00f6402120488d3d2eea0a00480f45f9488b8e400b00004885c9742b48bacff753e3a59bc4204c89f848f7ea4889d048c1e83f48c1fa074801c24839ca7c084889d6e8aef60600498b7640418b56384c89f74c89f9e87b710500488d357cdb0c00448b45d441f6c0027425498b
Function at 0x10803f210 is replicationFeedMonitors
Function at 0x1080d0cf8 is bit_tohex
Function at 0x1080940e0 is latencyAddSample
Function at 0x10807bbc0 is slowlogPushEntryIfNeeded
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/antirez/redis/issues
Suspect RAM error? Use redis-server --test-memory to verify it.
Segmentation fault: 11
Comment From: soloestoy
Hi, @dvirsky the reason is that redis does not wanna block when we are loading RDB file by using rdbLoad. So, in this function, we register a callback rdbLoadProgressCallback to process events, but unfortunately if we disconnect the current client during rdbLoad, that will result in segmentation fault.
DEBUG LOADAOF also has this problem.
Fixed in PR #4804 , please check it @antirez
Comment From: soloestoy
The way I fix this issue just like SCRIPT KILL.
Comment From: dvirsky
@soloestoy thanks, I didn't look into the code, just wanted to report it, even though it's pretty low priority IMO.
Comment From: antirez
Thanks guys! Please allow me to disappear a few days between the Easter vacations and the fact I'm working at a Super 💨Secret 🌺Stealth🍗project about Redis to be released soon... (*)
* if I'll be able to make it working
Comment From: soloestoy
I'm working at a Super 💨Secret 🌺Stealth🍗project about Redis to be released soon... (*)
Pretty cool!!! @antirez Maybe redis 5.0? I guess, ha-ha.
Comment From: oranagra
handled by the above mentioned PR.