Create an RDB parsing library, an API for C programs to be able to parse and understand the contents of the RDB file. Currently many project implement this on their own in various languages, but they become outdated when we change the RDB format, so putting such a library as part of redis, would mean we update it in the same PR that changes Redis. TBD.
Comment From: moticless
As part of our effort to create a formal RDB parser for Redis, we have written the first draft of the parser, which is now available for your reference. Currently, the parser only handles string and list data types. We would greatly appreciate any feedback or thoughts you may have. You can find the parser at the following link: https://github.com/redis/librdb
Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future.
Comment From: oranagra
the project mentioned above is feature complete and ready for prime time. closing this ticket.
Comment From: oranagra
p.s. at some point we may want to somehow bind this library with the redis repo in a way that will force whoever changes the rdb format to also update the parser code in the same PR.
@moticless AFAIK, currently some CI in that other repo will fail as soon as the redis unstable changes. is that right? does it have some daily CI run (not triggered by any manual pushes)?
Comment From: oranagra
i suppose we better refer to the lib from the README and / or from redis.io. re-opening the ticket for that purpose.
Comment From: moticless
@oranagra , librdb CI checkout and run against redis 5.0, 6.0, 6.2 and unstable. It doesn't have daily CI.
Shall i schedule it daily on unstable?
Comment From: oranagra
yes. weekly maybe. and if it'll get disabled. maybe we'll move it to the redis repo
Comment From: oranagra
@moticless @itamarhaber can you work together to put a link to this library in the right places in redis.io (both as a library and a tool). and then close this issue?
Comment From: oranagra
https://github.com/redis/redis-doc/pull/2675