video: https://youtu.be/ewZYMclzDSw
Redis - Linux: (os:Linux 5.10.0-kali9-amd64 x86_64 redis_version:6.2.6) 7 bytes
Redis - Windows: (5.0.14.1) 8 bytes
Error
[11248] 19 Mar 01:52:16.029 # Server initialized
[11248] 19 Mar 01:52:16.029 * Ready to accept connections
[11248] 19 Mar 01:52:29.117 * Before turning into a replica, using my master parameters to synthesize a cached master:
may be able to synchronize with the new master with just a partial transfer.
[11248] 19 Mar 01:53:06.901 * REPLICAOF 192.168.211.130:21000 enabled (user request from 'id=3 addr=192.168.211.130:606
8 fd=9 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=50 qbuf-free=32718 obl=0 oll=0 omem=0 events=r cmd=sl
veof')
[11248] 19 Mar 01:53:06.901 * Connecting to MASTER 192.168.211.130:21000
[11248] 19 Mar 01:53:06.901 * MASTER <-> REPLICA sync started
[11248] 19 Mar 01:53:07.915 * Non blocking connect for SYNC fired the event.
[11248] 19 Mar 01:53:07.915 # Sending command to master in replication handshake: -Writing to master: 找不到指定的模块。
[11248] 19 Mar 01:53:07.993 * Connecting to MASTER 192.168.211.130:21000
[11248] 19 Mar 01:53:07.993 * MASTER <-> REPLICA sync started
[11248] 19 Mar 01:53:09.007 * Non blocking connect for SYNC fired the event.
[11248] 19 Mar 01:53:09.007 * Master replied to PING, replication can continue...
[11248] 19 Mar 01:53:09.007 * Trying a partial resynchronization (request c35812a3f9110497c80f2f9f697516606610db0f:1).
[11248] 19 Mar 01:53:09.007 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
[11248] 19 Mar 01:53:09.007 * Discarding previously cached master state.
[11248] 19 Mar 01:53:09.007 * MASTER <-> REPLICA sync: receiving 9 bytes from master
[11248] 19 Mar 01:53:09.007 * MASTER <-> REPLICA sync: Flushing old data
[11248] 19 Mar 01:53:09.007 * MASTER <-> REPLICA sync: Loading DB in memory
[11248] 19 Mar 01:53:09.007 # Wrong signature trying to load DB from file
[11248] 19 Mar 01:53:09.007 # Failed trying to load the MASTER synchronization DB from disk
[11248] 19 Mar 01:53:09.085 * Connecting to MASTER 192.168.211.130:21000
[11248] 19 Mar 01:53:09.085 * MASTER <-> REPLICA sync started
[11248] 19 Mar 01:53:09.085 * Non blocking connect for SYNC fired the event.
[11248] 19 Mar 01:53:11.019 # Setting secondary replication ID to c35812a3f9110497c80f2f9f697516606610db0f, valid up to
offset: 1. New replication ID is a32b859b87b31d8669e4118a2b39ef22f0a3af96
[11248] 19 Mar 01:53:11.019 * MASTER MODE enabled (user request from 'id=3 addr=192.168.211.130:60608 fd=9 name= age=42
idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=32734 obl=0 oll=0 omem=0 events=r cmd=slaveof')
[11248] 19 Mar 01:53:39.037 * Before turning into a replica, using my master parameters to synthesize a cached master:
may be able to synchronize with the new master with just a partial transfer.
[11248] 19 Mar 01:53:39.037 * REPLICAOF 192.168.211.130:21000 enabled (user request from 'id=5 addr=192.168.211.130:606
0 fd=10 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=50 qbuf-free=32718 obl=0 oll=0 omem=0 events=r cmd=s
aveof')
[11248] 19 Mar 01:53:39.661 * Connecting to MASTER 192.168.211.130:21000
[11248] 19 Mar 01:53:39.661 * MASTER <-> REPLICA sync started
[11248] 19 Mar 01:53:40.675 * Non blocking connect for SYNC fired the event.
[11248] 19 Mar 01:53:40.675 # Sending command to master in replication handshake: -Writing to master: 找不到指定的模块。
[11248] 19 Mar 01:53:40.753 * Connecting to MASTER 192.168.211.130:21000
[11248] 19 Mar 01:53:40.753 * MASTER <-> REPLICA sync started
[11248] 19 Mar 01:53:41.268 * Non blocking connect for SYNC fired the event.
[11248] 19 Mar 01:53:41.268 * Master replied to PING, replication can continue...
[11248] 19 Mar 01:53:41.268 * Trying a partial resynchronization (request a32b859b87b31d8669e4118a2b39ef22f0a3af96:1).
[11248] 19 Mar 01:53:41.268 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
[11248] 19 Mar 01:53:41.268 * Discarding previously cached master state.
[11248] 19 Mar 01:53:41.268 * MASTER <-> REPLICA sync: receiving 8 bytes from master
[11248] 19 Mar 01:53:41.268 * MASTER <-> REPLICA sync: Flushing old data
[11248] 19 Mar 01:53:41.268 * MASTER <-> REPLICA sync: Loading DB in memory
[11248] 19 Mar 01:53:41.268 # Short read or OOM loading DB. Unrecoverable error, aborting now.
[11248] 19 Mar 01:53:41.268 # Internal error in RDB reading function at rdb.c:2185 -> Unexpected EOF reading RDB file
[offset 0] Checking RDB file test2.txt
--- RDB ERROR DETECTED ---
[offset 0] Unexpected EOF reading RDB file
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
Comment From: oranagra
@yuyan-sec please describe what you're trying to do. The description mentions both windows and Linux, and several different versions...
Also, please note that this redis (this project) doesn't support windows, so if you used some other port / fork of Redis, you better report problems in the project that maintains it.
Comment From: oranagra
Also note that short read during replication could just mean that the master dropped the connection for some reason, so better check the master log file too, maybe it'll specify why the connection was dropped.
Comment From: yuyan-sec
Linux kali 5.10.0-kali9-amd64 #1 SMP Debian 5.10.46-4kali1 (2021-08-09) x86_64 GNU/Linux
Redis Version: 6.2.6
当文件小于 9 个字节 Redis就会主动关闭
Google Translate
Redis will automatically close when the file is less than 9 bytes
8349:M 18 Mar 2022 23:51:32.030 * MASTER MODE enabled (user request from 'id=9 addr=192.168.211.1:50799 laddr=192.168.211.130:6379 fd=7 name= age=10 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=40920 argv-mem=12 obl=0 oll=0 omem=0 tot-mem=61476 events=r cmd=slaveof user=default redir=-1')
8349:S 18 Mar 2022 23:51:45.506 * Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
8349:S 18 Mar 2022 23:51:45.506 * Connecting to MASTER 192.168.211.1:21000
8349:S 18 Mar 2022 23:51:45.506 * MASTER <-> REPLICA sync started
8349:S 18 Mar 2022 23:51:45.506 * REPLICAOF 192.168.211.1:21000 enabled (user request from 'id=11 addr=192.168.211.1:50800 laddr=192.168.211.130:6379 fd=7 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=48 qbuf-free=40906 argv-mem=25 obl=0 oll=0 omem=0 tot-mem=61489 events=r cmd=slaveof user=default redir=-1')
8349:S 18 Mar 2022 23:51:48.556 * Non blocking connect for SYNC fired the event.
8349:S 18 Mar 2022 23:51:48.557 * Master replied to PING, replication can continue...
8349:S 18 Mar 2022 23:51:48.558 * Trying a partial resynchronization (request 4fe7770348b4e6dfcb93f31591bd7f1486770aaf:1).
8349:S 18 Mar 2022 23:51:48.558 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
8349:S 18 Mar 2022 23:51:48.558 * Discarding previously cached master state.
8349:S 18 Mar 2022 23:51:48.558 * MASTER <-> REPLICA sync: receiving 8 bytes from master to disk
8349:S 18 Mar 2022 23:51:48.558 * MASTER <-> REPLICA sync: Flushing old data
8349:S 18 Mar 2022 23:51:48.558 * MASTER <-> REPLICA sync: Loading DB in memory
8349:S 18 Mar 2022 23:51:48.559 # Short read or OOM loading DB. Unrecoverable error, aborting now.
8349:S 18 Mar 2022 23:51:48.559 # Internal error in RDB reading offset 0, function at rdb.c:2742 -> Unexpected EOF reading RDB file
[offset 0] Checking RDB file 1.txt
--- RDB ERROR DETECTED ---
[offset 0] Unexpected EOF reading RDB file
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
Comment From: oranagra
Sorry, I still don't understand any of this.. Why is the rdb file named 1.txt? Is this the name you gave it in the config file.
The rdb file seemed to have been received from the master, but despite my request you didn't share the master log file.
I don't understand your reference to 9 bytes (and 7 bytes, and 8 bytes), and the screen shots that show it's much larger.
And I don't understand you you insist to post Chinese text, and a Google translate reference (instead of just posting the English part).
I still don't understand the reference to Windows and 5.0.14 at the top.
Comment From: enjoy-binbin
@oranagra i am guessing he is doing some RDB file attack? we do have the rdb file check like:
if (rioRead(&rdb,buf,9) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
rdbCheckError("Wrong signature trying to load DB from file");
goto err;
}
so if you hack the dump.rdb, let say echo 1234567 > dump.rdb
[root@binblog redis]# echo 1234567 > dump.rdb
[root@binblog redis]# src/redis-server
22607:M 19 Mar 2022 17:25:12.179 * The AOF directory appendonlydir doesn't exist
22607:M 19 Mar 2022 17:25:12.179 # Short read or OOM loading DB. Unrecoverable error, aborting now.
22607:M 19 Mar 2022 17:25:12.179 # Internal error in RDB reading offset 0, function at rdb.c:3232 -> Unexpected EOF reading RDB file
[offset 0] Checking RDB file dump.rdb
--- RDB ERROR DETECTED ---
[offset 0] Unexpected EOF reading RDB file
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
echo 12345678 > dump.rdb
[root@binblog redis]# echo 12345678 > dump.rdb
[root@binblog redis]# src/redis-server
22716:M 19 Mar 2022 17:26:00.548 * The AOF directory appendonlydir doesn't exist
22716:M 19 Mar 2022 17:26:00.548 # Wrong signature trying to load DB from file
22716:M 19 Mar 2022 17:26:00.549 # Fatal error loading the DB: Invalid argument. Exiting.
he is using a tool that will create master-slave repl then hack the rbd.file and crash the server (6.2.6) I did crash the server by following his steps (the video) in 7.0, it will also crash the server by hacking rdb.file I provide the steps to reproduce: 1. src/redis-server redis.conf --enable-protected-configs no (7.0) 2. git clone git@github.com:r35tart/RedisWriteFile.git (https://github.com/r35tart/RedisWriteFile) 3. echo 1234567 > 1.txt 4. python3 RedisWriteFile.py --rhost 127.0.0.1 --lhost 127.0.0.1 --rfile=2.txt --lfile=1.txt 5. the tool hacking the rbd.file (I'm guessing the wrong rdb file stream is being transmitted), and make the server become a replica, so when the replica start loading the rbd file
There will be two cases: 1. echo 1234567 > 1.txt, it will crash the server
3187:S 19 Mar 2022 17:54:48.648 * Trying a partial resynchronization (request 08f6269970bce63495c06596c34ade3818d4ad1e:1).
3187:S 19 Mar 2022 17:54:48.648 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
3187:S 19 Mar 2022 17:54:48.648 * MASTER <-> REPLICA sync: receiving 8 bytes from master to disk
3187:S 19 Mar 2022 17:54:48.648 * Discarding previously cached master state.
3187:S 19 Mar 2022 17:54:48.648 * MASTER <-> REPLICA sync: Flushing old data
3187:S 19 Mar 2022 17:54:48.648 * MASTER <-> REPLICA sync: Loading DB in memory
3187:S 19 Mar 2022 17:54:48.652 # Short read or OOM loading DB. Unrecoverable error, aborting now.
3187:S 19 Mar 2022 17:54:48.652 # Internal error in RDB reading offset 0, function at rdb.c:3232 -> Unexpected EOF reading RDB file
[offset 0] Checking RDB file dump.rdb
--- RDB ERROR DETECTED ---
[offset 0] Unexpected EOF reading RDB file
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
to save your time (the server exit in here, maybe in this case, we shouldn't exit, like https://github.com/redis/redis/commit/35d5cd71dec98053f46e2edbefb998eba81545e3, just for test or we handle it in if (rioRead(rdb,buf,9) == 0) goto eoferr;):
} else if (rdbFileBeingLoaded) {
/* If we're loading an rdb file form disk, run rdb check (and exit) */
serverLog(LL_WARNING, "%s", msg);
char *argv[2] = {"",rdbFileBeingLoaded};
redis_check_rdb_main(2,argv,NULL);
- echo 123456789 > 1.txt, it will not crash the server, reject the request (remain in master)
2453:S 19 Mar 2022 17:50:00.729 * Trying a partial resynchronization (request 75f465b6e2591844015b2b629c7b2619e4841501:1).
2453:S 19 Mar 2022 17:50:00.729 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
2453:S 19 Mar 2022 17:50:00.729 * MASTER <-> REPLICA sync: receiving 10 bytes from master to disk
2453:S 19 Mar 2022 17:50:00.729 * Discarding previously cached master state.
2453:S 19 Mar 2022 17:50:00.729 * MASTER <-> REPLICA sync: Flushing old data
2453:S 19 Mar 2022 17:50:00.729 * MASTER <-> REPLICA sync: Loading DB in memory
2453:S 19 Mar 2022 17:50:00.733 # Wrong signature trying to load DB from file
2453:S 19 Mar 2022 17:50:00.733 # Failed trying to load the MASTER synchronization DB from disk: Invalid argument
@yuyan-sec can you describe it in Chinese to see if I have missed anything? 英文表述不太方便的话，阔以用中文描述一遍吗？（你应该是想表述用那个工具，通过修改 rdb 文件，会让服务器崩溃）
Comment From: yuyan-sec
主从复制可以写文件，如果写的文件小于 9 个字节 Redis就是崩溃，导致Redis服务关闭。
Comment From: enjoy-binbin
@yuyan-sec 哦哦，所以是那个工具模拟主从，让对方服务器充当从库，然后跟工具拉起的服务器进行主从复制，然后直接往目标服务器写文件（模拟rdb文件的传输，而不是正常的rdb传输？），然后会让目标服务器在拿到 rdb 文件的时候，写进磁盘，进行加载的时候，然后就有可能崩溃（根据写入的字节 它撞到不同的分支代码，一个进行了退出，一个没有）
Comment From: yuyan-sec
对的，当字节大于9个字节 Redis是正常写入的，小于9个字节就会崩溃，具体什么原因导致的我就不知道了，我也是无意中发现的。
Comment From: enjoy-binbin
@yuyan-sec 原因我这边知道了，感谢反馈，是个蛮有意思的案例，这个同样会发生在 redis7.0，我们会看看怎么处理 它的原因是：redis 有一个 rdb 版本号的检查，文件的前9个字节（REDIS 和 4个字节的rdb文件版本号），小于的时候走了另外个代码路径（那边进行了进程退出），大于等于的时候就进行了版本号的检查（这边没进行进程退出）issue 我就先继续保持打开状态 thanks for the report, we'll see how to deal with it
Comment From: oranagra
Well, clearly there was some communication issue in that report. But still, redis will exit (not crash) on a corrupt RDB. it does it in many places, not just when reading the header, and I don't see that as a problem. If a user has permission to AUTH into redis, and issue a SLAVREOF command, that's similar to being able to also SHUTDOWN or FLUSHALL. The only thing that would have worried me is to avoid heap corruption and remote code execution. Hence the rdb sanitization campaign that was part of 6.2
Comment From: enjoy-binbin
@oranagra yes. one failed in REDIS magic check, the other one exit (not crash, my bad) like other places
@yuyan-sec 对于有问题的传输rdb文件，redis 实际上会在很多地方做检查，发现有错误就会进行退出。只是刚好在这个 rdb 头部分的检查里有一些差异，一个进行了退出，另外一个没有退出。这中间有点可能不一致，不过并不是什么严重的问题，因为你都能直接进行主从复制了，实际上你对那个服务器本身就有很大的权限，进行破坏是很容易的
so marking to be close?
Comment From: yuyan-sec
好的，关闭问题吧