It seems like the server is not able to always compute the correct hash slot when using hash tags
As part of testing redis we tested a python library, which attempts to calculate the hash slot a key should go into, and for some keys it would receive a moved error back with another hash slot than the client had figured out.
It was further investigated to see where the issue is and what could be found was the following: and when we tried to use the CLUSTER KEYSLOT command, which is calculating the hash slot for a given key the result was rather surprising
redis-cli -h host -p 13893 CLUSTER KEYSLOT "a{'1':{'i32':1}}" (integer) 12350 redis-cli -h host -p 13893 CLUSTER KEYSLOT "{'1':{'i32':1}}" (integer) 12350 redis-cli -h host -p 13893 CLUSTER KEYSLOT "'1':{'i32':1" (integer) 5540
We would have assumed that all three should give the same hash slot. The python library and java library agree with 5540
According https://redis.io/topics/cluster-spec it should be allowed to have curly brackets in curly brackets, and how we have understood we should consider they hash tag to be as the input is what ever is between the first starting curly braces and the first ending curly braces.
We also tried another one without curly braces and it worked as expected:
redis-cli -h host -p 13893 CLUSTER KEYSLOT fag{bar} (integer) 5061 redis-cli -h host -p 13893 CLUSTER KEYSLOT bar (integer) 5061
And we tried with another one which did not calculate as expected:
redis-cli -h host -p 13893 CLUSTER KEYSLOT foo{bar}{zap} (integer) 6469 redis-cli -h host -p 13893 CLUSTER KEYSLOT foo{bar} (integer) 5061 redis-cli -h host -p 13893 CLUSTER KEYSLOT foo{zap} (integer) 6469
We would have expected that the foo{bar}{zap} should calculate the same as foo{bar} or bar.
We used the redis cli, we used the redis-py-cluster and lettuce library.
Enterprise redis 5.4.0-19 has been used for the cluster
Comment From: Grokzen
I am acctually getting a different result on my end with a vanilla redis 5.0.3 installation
127.0.0.1:7000> CLUSTER KEYSLOT "a{'1':{'i32':1}}"
(integer) 5540
127.0.0.1:7000> CLUSTER KEYSLOT "{'1':{'i32':1}}"
(integer) 5540
127.0.0.1:7000> CLUSTER KEYSLOT "'1':{'i32':1"
(integer) 5540
127.0.0.1:7000> CLUSTER KEYSLOT foo{bar}{zap}
(integer) 5061
127.0.0.1:7000> CLUSTER KEYSLOT foo{bar}
(integer) 5061
127.0.0.1:7000> CLUSTER KEYSLOT foo{zap}
(integer) 6469
127.0.0.1:7000>
My result is what i expect from vanilla redis based on the cluster spec you posted above.
My bet would be Enterprise redis might have something to do with this problem and should be investigated further or cross verified by someone else that my teest on plain 5.0.3 produces same result as mine and that a Enterprise redis might be working differently or have some other quirk in it.
Comment From: AngusP
@Grokzen I can verify that 5.0.4 gave keyslot 5540 on all three cases for me too, through a UNIX socket and also through a Spiped pipe.
@itamarhaber or one of the other Redis Labs people will have to take a look, this seems very strange indeed...
Comment From: itamarhaber
This is definitely an issue with the Enterprise, and one that can actually be worked around IIRC.
@mat013 please open a ticket with RedisLabs' support - thanks for reporting! @AngusP thanks for flagging me - dunnu how I missed this one :/
Comment From: AngusP
@mat013 can you try again but from an interactive redis-cli instead of as all-in-one shell commands? Just to sanity check that the shell you're using isn't interpreting the contents of "a{'1':{'i32':1}}" and therefore doing something weird and removing bits before passing it to Redis? I did check that zsh, sh and bash for me didn't break it with what you gave as examples, but still it's possible - the only one that worked and gave 5540 for you was also the only one with no paired }. This wouldn't explain the MOVED you said you were seeing from Python but then the occasional MOVED is allowed if the cluster is being adjusted and hash slots are changing nodes..?
e.g.
$ redis-cli -c CLUSTER KEYSLOT \{\'1\':\{\'i32\':1\}\}
(integer) 5540
$ redis-cli -c CLUSTER KEYSLOT {'1':{'i32':1}}
(integer) 3214
Comment From: mat013
@itamarhaber I thought this was the place to report official bugs... I have submitted it to our contact in redis labs... so hopefully they will look at it soon
I am having holiday at the moment, so I have asked one of my colleague to check it again. But if it would be better I can also do it through a library. Because we get the same result.... and there there should not be any ambiguity. Let me know if you prefer. I can use either lettuce in java or redis-py-cluster
Comment From: itamarhaber
@mat013 enjoy your holiday :)
This is indeed the place for issues with the open source Redis project. Enterprise issues, however, need to be reported to support at redislabs dot com and in this case this issue appears to be Enterprise-only and affects all clients.
Please feel free to contact me (itamar at redislabs dot com) and update me with the number of the issue that you've opened with our support so I can monitor it as well.
Comment From: Grokzen
@itamarhaber It has been confirmed in the vanilla redis 5.0.3 by me and AngusP that it at-least is not present in the open source version in that version so we think that it is redis-enterprise only issue. We have not seen or got any other report for any other setup/case where this issue is presented elsewhere.
Comment From: itamarhaber
@Grokzen thanks for verifying!
Comment From: mat013
@AngusP My colleague did do as requested and there was no difference between redis-cli and from the command line.
We are in a dialogue with a consultant from redis lab, and I am trying to obtain the ticket id.
I will post it once i got it
Comment From: madolson
Looks like redis enterprise related, so closing