Redis Support auto-rebalance feature in Redis Cluster

Currently, Redis Cluster can implements auto-failover which helps us a lot. i think it can also provide such a feature : auto-rebalance according to the percentage of memory-used / maxmemory and several other factors like cpu when we encounter follower scenarios: 1. some node captures most of data 2. added a new master node into the Cluster . Although we can make it resort to client api, but if the Cluster can provide an option to make it Automatic or manual , that will be better. Anyway, As a user, maybe we should implement it on our own according to our specific scenario.

Comment From: antirez

Hello, Automatic-rebalance is planned for iteration 2 of Redis Cluster, however manual rebalance already works but is fully manual, that is, you need to tell Redis Cluster from which node to what other node to move how many hash slots.

The idea for the future is: 1. Each server adds used/free memory in the cluster bus message headers, so each node knows how other nodes are from the POV of memory usage. 2. When nodes are not balanced, an automatic hash slot migration starts in order to fix the issue until an acceptable threshold is reached again.

We need to see Redis Cluster v1 more deployed / used / battle tested in order to start new developments, for now the focus is into supporting what we got so far. Thanks for your feedback! Closing since this is one of the fundamental feature planned for Redis developments, so for sure we'll not forget :-)

Comment From: Spectrumleeee

OK, thank you!

Comment From: neopaf

Thank you, Salvatore. We're seriously planning on having Redis as a potential backup to our super-installation of Couchbase clusters. The lack of even manual rebalance is one of show stoppers for us. Support team on the Customer side will be super-sad if we are to tell them that "Rebalance" magic button will disappear, and the need of manual calculating of which slots go where will be required.

Please consider a two-phase process here? I guess, I'd better create a separate issue about simple rebalance utility.

Comment From: allanwax

Note for the future: When a new master is added to the cluster, it should not auto-rebalance until at least one slave is assigned to it. Otherwise there could be a failure in the migration and K-V could be lost.