Hi there! Just now i tried restored my redis cluster from backup, but master server wiped out database every time at startup when anothe slave trying to sync with him duting master startup.
Step by step: 1. I stopped redis-server on master 2. Remove all files from /var/lib/redis 3. restore dump.rdb from backup (about 800MB) 4. start redis server
After this redis-server started with empty database. Logs:
26978:C 30 Mar 2020 19:03:32.266 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
26978:C 30 Mar 2020 19:03:32.266 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=26978, just started
26978:C 30 Mar 2020 19:03:32.267 # Configuration loaded
26979:M 30 Mar 2020 19:03:32.270 * Running mode=standalone, port=6379.
26979:M 30 Mar 2020 19:03:32.270 # Server initialized
26979:M 30 Mar 2020 19:03:32.271 * DB loaded from disk: 0.000 seconds
26979:M 30 Mar 2020 19:03:32.271 * Ready to accept connections
26979:M 30 Mar 2020 19:03:32.550 * Replica 192.168.9.7:6379 asks for synchronization
26979:M 30 Mar 2020 19:03:32.550 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '09b92ecda1e8c9969c0011ef26d24c31f081dfa6', my replication IDs are '810dbb9d51c44468f6e8d4c8ce8a578c1a563580' and '0000000000000000000000000000000000000000')
26979:M 30 Mar 2020 19:03:32.550 * Starting BGSAVE for SYNC with target: disk
26979:M 30 Mar 2020 19:03:32.551 * Background saving started by pid 26984
26984:C 30 Mar 2020 19:03:32.553 * DB saved on disk
And dump.rdb size is a few kilobytes. But if I stopped all slaves servers, and repeat steps 1-4 redis master starts normally:
28676:C 30 Mar 2020 19:08:26.453 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
28676:C 30 Mar 2020 19:08:26.453 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=28676, just started
28676:C 30 Mar 2020 19:08:26.453 # Configuration loaded
28677:M 30 Mar 2020 19:08:26.458 * Running mode=standalone, port=6379.
28677:M 30 Mar 2020 19:08:26.458 # Server initialized
28677:M 30 Mar 2020 19:08:35.124 * DB loaded from disk: 8.665 seconds
28677:M 30 Mar 2020 19:08:35.124 * Ready to accept connections
28677:M 30 Mar 2020 19:08:57.798 * Background saving started by pid 28764
28764:C 30 Mar 2020 19:09:13.207 * DB saved on disk
28764:C 30 Mar 2020 19:09:13.239 * RDB: 12 MB of memory used by copy-on-write
28677:M 30 Mar 2020 19:09:13.355 * Background saving terminated with success
28677:M 30 Mar 2020 19:09:30.091 * Replica 192.168.9.1:6379 asks for synchronization
28677:M 30 Mar 2020 19:09:30.091 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for 'f7cd545f159d14760f8911adb7a61b0871150d59', my replication IDs are 'e48ce68eec1a481f085656b547eb145f2e52cb8b' and '0000000000000000000000000000000000000000')
28677:M 30 Mar 2020 19:09:30.091 * Starting BGSAVE for SYNC with target: disk
28677:M 30 Mar 2020 19:09:30.115 * Background saving started by pid 28993
and all is ok with databases and keys.
Maybe I missed something?
# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
# redis-server --version
Redis server v=5.0.3 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=afa0decbb6de285f
Comment From: trevor211
Can you give the master configs in both situations?
Comment From: rayderua
On all situations config was same:
appendonly yes
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
replica-serve-stale-data yes
replica-priority 10
rdbchecksum yes
port 6379
save 900 1
save 300 10
save 60 10000
list-max-ziplist-entries 512
set-max-intset-entries 512
lua-time-limit 5000
slowlog-max-len 128
maxclients 4064
latency-monitor-threshold 0
aof-rewrite-incremental-fsync yes
rdbcompression yes
pidfile "/var/run/redis/redis-server.pid"
auto-aof-rewrite-percentage 100
hz 10
list-max-ziplist-value 64
hash-max-ziplist-entries 512
dbfilename "dump.rdb"
replica-read-only yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
no-appendfsync-on-rewrite no
stop-writes-on-bgsave-error yes
hash-max-ziplist-value 64
logfile "/var/log/redis/redis-server.log"
tcp-keepalive 0
auto-aof-rewrite-min-size 64mb
loglevel notice
bind 0.0.0.0
hll-sparse-max-bytes 3000
notify-keyspace-events ""
appendfilename "appendonly.aof"
daemonize yes
timeout 0
databases 22
tcp-backlog 511
activerehashing yes
slowlog-log-slower-than 10000
aof-load-truncated yes
dir "/var/lib/redis"
Comment From: trevor211
Your redis-server is 5.0.3 and the master's appendonly is set to 'yes', it's impossible to load dump.rdb when startup. See https://github.com/antirez/redis/blob/5.0.3/src/server.c#L3852 Please check the aof file when you do your test.