Greetings Team!
In one of our heavy write use case, with shard size over 60GB and AOF(no RDB) enabled, triggering the sentinel fail-over resulting in Full syncs, according to this documentation , that shouldn't cause full syncs.
20404:M 27 Apr 2023 19:40:47.949 # Connection with master lost.
20404:M 27 Apr 2023 19:40:47.949 * Caching the disconnected master state.
20404:M 27 Apr 2023 19:40:47.949 * Discarding previously cached master state.
20404:M 27 Apr 2023 19:40:47.949 # Setting secondary replication ID to 691f96323dbfd7763e2d712e3b481ea6916735d9, valid up to offset: 7579144700778. New replication ID is 00f662899e24b5b8eeb1345e2ed55407a62af0db
20404:M 27 Apr 2023 19:40:47.949 * MASTER MODE enabled (user request from 'id=667 addr=X.X.X.X:59596 laddr=X.X.X.X16379 fd=397 name=sentinel-89efe05b-cmd age=172862 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=140 qbuf-free=40814 argv-mem=4 obl=36 oll=0 omem=0 tot-mem=61468 events=r cmd=exec user=default redir=-1')
20404:M 27 Apr 2023 19:40:47.952 # CONFIG REWRITE executed with success.
20404:M 27 Apr 2023 19:40:48.943 * Replica X.X.X.X:16379 asks for synchronization
20404:M 27 Apr 2023 19:40:48.943 * Partial resynchronization not accepted: Requested offset for second ID was 7579144822181, but I can reply up to 7579144700778
20404:M 27 Apr 2023 19:40:48.943 * Starting BGSAVE for SYNC with target: disk
20404:M 27 Apr 2023 19:40:49.613 * Background saving started by pid 38188
20404:M 27 Apr 2023 19:40:59.828 * Replica X.X.X.X:16383 asks for synchronization
20404:M 27 Apr 2023 19:40:59.828 * Partial resynchronization not accepted: Requested offset for second ID was 7579144828920, but I can reply up to 7579144700778
20404:M 27 Apr 2023 19:40:59.829 * Waiting for end of BGSAVE for SYNC
20404:M 27 Apr 2023 19:41:19.389 * Replica X.X.X.X:16379 asks for synchronization
20404:M 27 Apr 2023 19:41:19.389 * Partial resynchronization not accepted: Requested offset for second ID was 7579144828920, but I can reply up to 7579144700778
20404:M 27 Apr 2023 19:41:19.397 * Waiting for end of BGSAVE for SYNC
38188:C 27 Apr 2023 19:51:38.712 * DB saved on disk
is there some config we can tune to avoid full syncs upon fail-over or is this a bug?
Comment From: satheeshaGowda
Hello @itamarhaber @oranagra @hwware @oranagra , hope all is well!
would you mind if I request you to share some insight here ?
Comment From: oranagra
Requested offset for second ID was 7579144822181, but I can reply up to 7579144700778
there are about 120kb of data processed on the master and not on the replica.
this switch over mechanism can only when the replica is fully in-sync with the master.
e.g. using a coordinated failover after applying CLIENT PAUSE WRITE.
Maybe this discussion can help #12097