In redis cluster environment...
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 cluster nodes
5b1c8632b07fe88e659637e9e006d69d6b96e33b 192.168.0.161:6380@16380 master - 0 1633098067207 2 connected 5461-10922
063ac3ce7227485cf3958ee076b59ac3e3ada33b 192.168.0.160:6379@16379 master - 0 1633098068230 1 connected 0-5460
b6a025e635132ecb8e805c9d6cf30338eb158753 192.168.0.163:6382@16382 slave 6bcc4e0f5719c1521f98b9c249cd30fb95083dad 0 1633098069000 3 connected
bab6edba09a7548091b5bcf361cb5b06566df152 192.168.0.165:6384@16384 slave 5b1c8632b07fe88e659637e9e006d69d6b96e33b 0 1633098068000 2 connected
6bcc4e0f5719c1521f98b9c249cd30fb95083dad 192.168.0.162:6381@16381 myself,master - 0 1633098067000 3 connected 10923-16383
e40bd83ab91b8703a0665170e6fd6b386b9ab606 192.168.0.164:6383@16383 slave 063ac3ce7227485cf3958ee076b59ac3e3ada33b 0 1633098069262 1 connected
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 cluster slots
1) 1) (integer) 0
2) (integer) 5460
3) 1) "192.168.0.160"
2) (integer) 6379
3) "063ac3ce7227485cf3958ee076b59ac3e3ada33b"
4) 1) "192.168.0.164"
2) (integer) 6383
3) "e40bd83ab91b8703a0665170e6fd6b386b9ab606"
2) 1) (integer) 5461
2) (integer) 10922
3) 1) "192.168.0.161"
2) (integer) 6380
3) "5b1c8632b07fe88e659637e9e006d69d6b96e33b"
4) 1) "192.168.0.165"
2) (integer) 6384
3) "bab6edba09a7548091b5bcf361cb5b06566df152"
3) 1) (integer) 10923
2) (integer) 16383
3) 1) "192.168.0.162"
2) (integer) 6381
3) "6bcc4e0f5719c1521f98b9c249cd30fb95083dad"
4) 1) "192.168.0.163"
2) (integer) 6382
3) "b6a025e635132ecb8e805c9d6cf30338eb158753"
If I won't change the slots, command like ...
MGET k1 k2 k3
MSET k1 v1 k2 v2 k3 v3
DEL k1 k2 k3
will all fail because of CLUSTER_REDIR_CROSS_SLOT
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 -c mset k1 v1 k2 v2 k3 v3
(error) CROSSSLOT Keys in request don't hash to the same slot
CROSSSLOT, not CROSS'NODE', which means, even I found that k2 k3 belong to node1(0-5460), even split the one command to two, MSET k1 v1 and MSET k2 v2 k3 v3, it won't work either.
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 -c cluster keyslot k1
(integer) 12706
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 -c cluster keyslot k2
(integer) 449
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 -c cluster keyslot k3
(integer) 4576
This code shows that, if I use multiple keys, the keys should be same or the slots of the keys should be same, it won't happen frequently. Seems that MSET MGET can't be used in cluster environment.
/* If it is not the first key, make sure it is exactly
* the same key as the first we saw. */
if (!equalStringObjects(firstkey,thiskey)) {
if (slot != thisslot) {
/* Error: multiple keys from different slots. */
getKeysFreeResult(&result);
if (error_code)
*error_code = CLUSTER_REDIR_CROSS_SLOT;
return NULL;
} else {
/* Flag this request as one with multiple different
* keys. */
multiple_keys = 1;
}
}
Why should be the same SLOT? Why not the slots from SAME NODE? If it supports that slots from SAME NODE, I can sort out the keys in different command requests at lease.
Comment From: itamarhaber
Hello @Ron2014
The cluster is partitioned across N (= 2^14) hash slots. Every key consistently hashes to the same single slot. Any given hash slot is managed by a single node in the cluster. Slots can be migrated from one node to another.
Given the above, CROSSNODE is a runtime error that depends on the current cluster topology. To prevent such errors, the cluster restricts multi-key operations to a single slot.
Comment From: Ron2014
Thanks @itamarhaber for response.
You said, every key consistently hashes to the same single slot. Any given hash slot is managed by a single node in the cluster.
So when I send keys to redis API, redis should know which part of the keys belong to node A, which part belong to node B or other. Just like using '-c' in command, let redis redirect to another node.
# redis-cli -h 192.168.0.162 -p 6381 -a 123456 -c cluster keyslot k1
(integer) 12706
# redis-cli -h 192.168.0.161 -p 6380 -a 123456 set k1 v1
(error) MOVED 12706 192.168.0.162:6381
# redis-cli -h 192.168.0.161 -p 6380 -a 123456 -c set k1 v1
OK
-c tells redis this is cluster-mode. Redirect code is...
/* Check if we need to connect to a different node and reissue the
* request. */
if (config.cluster_mode && reply->type == REDIS_REPLY_ERROR &&
(!strncmp(reply->str,"MOVED",5) || !strcmp(reply->str,"ASK")))
{
char *p = reply->str, *s;
int slot;
output = 0;
/* Comments show the position of the pointer as:
*
* [S] for pointer 's'
* [P] for pointer 'p'
*/
s = strchr(p,' '); /* MOVED[S]3999 127.0.0.1:6381 */
p = strchr(s+1,' '); /* MOVED[S]3999[P]127.0.0.1:6381 */
*p = '\0';
slot = atoi(s+1);
s = strrchr(p+1,':'); /* MOVED 3999[P]127.0.0.1[S]6381 */
*s = '\0';
sdsfree(config.hostip);
config.hostip = sdsnew(p+1);
config.hostport = atoi(s+1);
if (config.interactive)
printf("-> Redirected to slot [%d] located at #%s:%d\n",
slot, config.hostip, config.hostport);
config.cluster_reissue_command = 1;
cliRefreshPrompt();
This design only supports ONE key command redirect. How about making cluster_reissue_command to Array? For multiple keys command, maybe it will redirect to several other nodes at the same time.
That's why I'm confused, -c can make redirect, but not good for multiple keys. Is this feature in TODO list?
And slots can be migrated from one node to another. I got that. Slots are not migrated often. It only happend when I add nodes or delete notes or using hashslot tag. When the slot migration is running, is there A status flag marked in redis? If my connection is asynchronous, I can wait till the migration is over, then redis deals with the multiple keys command, confirming keys' hashslot and node belonging.
If migration status flag and asynchronous delay execution don't work well. I wonder that, why '-c' option with one key command can ignore the slot migration problem. just redirect and no need think any more.
Comment From: itamarhaber
"Very high performance and scalability while preserving weak but reasonable forms of data safety and availability is the main goal of Redis Cluster." (https://redis.io/topics/cluster-spec)
Besides speed, Redis also ensures the atomicity of single operations. A redirect for one key (or multiple keys belonging to the same slot) isn't the same as trying to orchestrate a distributed operation in a cluster. Besides the complexity involved, this will greatly impact the cluster's overall performance once inter-node communication is required to serve multi-slot commands.
Slots are not migrated often
It doesn't matter - because slot assignment is dynamic, you can't say that two clusters are always the same. So, it is possible that in your dev env two slots are on the same node whereas in prod they aren't. Hence the restriction.
Comment From: Ron2014
Hence the restriction.
Which means, if my server want to save datas in one operation, it need to make the hashslot as one, using hashslot tag.
In this server, keys are belonging to the same slot, which means if I want to horizontal scale the servers, make them to server1 server2 server3 ... serverN to deal with requests. There is only way to dispatch the requests -- Consistent Hash.
Even there are several ways of route strategy -- random, mod, poll and so on. If I use redis cluster to save datas, it perfer to use Consistent Hash ONLY to reach load balancing.
Comment From: RedisOptimal
@Ron2014 Here is my code : https://github.com/RedisOptimal/redis-cluster-support-mget That code support mget/mset running in same node, not the same slot. mgetSeries/msetSeries will partition key by node, and run mget/mset command one by one in each node. Finally, combine the result. User dont care about where is the key belong, just use it for speed up your program.