@antirez during a talk with @yoav-steinberg about the limitations of active defrag on lists (it can't stop an iteration on a list type and resume it on the next cycle, so it may cause a latency spike or freeze if the db has huge lists), an idea came up for adding scan-like feature for lists.
The idea is to add a feature of named bookmarks for lists in adlist.c. Each list will maintain a list of named bookmarks referring to some specific nodes mid-list node. when a node is removed from the list, the bookmark is updated to point to the next one (or NULL). The bookmark list is always small (32?), and if there are too many bookmarks, we just scrap the oldest one (FIFO). When someone (active defrag or LSCAN command) scans a list, he provides a (hopefully unique) bookmark name, so that he can resume the scan later (even if nodes were added or removed from the list). The documentation will state that bookmarks are not guaranteed to survive and a scan can error and will have to be re-started.
although limited, it can mean a lot to some use cases. and it's quite simple to implement. let me know what you think
Comment From: oranagra