Redis redis-benchmark cannot work on Redis Cluster

When I use redis-benchmark to fire requests to a master node of the cluster, only partial records can be correctly saved (in the targeted master node we fire to). Other records cannot be successfully sharded to other master nodes and just be dropped.

I wonder that's because redis-benchmark calls redis-cli in a way of without -c specified. Can we enable the feature of sharding on redis-benchmark?

I apologize if this issue has been discussed before. Thanks!

Cheers, Gene

Comment From: mattsta

Can we enable the feature of sharding on redis-benchmark?

The hiredis library doesn't support Redis Cluster yet, so nobody has made a Cluster-aware redis-benchmark. There are other design issues to consider as well: what does it mean to benchmark a cluster? what would the results look like?

Also, Redis Cluster doesn't expand processing power, it just extends your storage space. If you do benchmark a cluster, the only difference from benchmarking a single Redis instance should be the latency between where redis-benchmark runs and your cluster nodes.

Comment From: airpingu

How about sharing? I also hope to benchmark the sharing latency since Redis nodes can be running on different servers.

Comment From: HeartSaVioR

Also, Redis Cluster doesn't expand processing power, it just extends your storage space.

I feel that Redis Cluster can expand processing power with multi-threaded or distrubuted random-heavy-workload jobs, because each Redis instance on cluster only need to execute commands on its own key range. :) But critical part of performance would be up to client, how many client is sending requests to wrong instance and receive MOVED, and retry. So strategy about keeping updated cluster information is very important in point of client's view.

How about sharing? I also hope to benchmark the sharing latency since Redis nodes can be running on different servers.

"Needs citation." AFAIK, only pubsub, and gossip protocol is shared among the Redis instances on cluster. So you don't need to worry about sharing latency unless you're having too much instances on cluster, or there're pubsub usages.

Comment From: airpingu

I apologize for my typo. I mean "sharding" not "sharing"⋯

What I worried about is the extra latency of looking up the slot table and transferring data across Redis nodes/servers in a cluster, which is not an issue in a single Redis node. That's why I attempt to benchmark the sharding effect.

Comment From: badboy

A good client will cache the slot assignment and can calculate which instance to contact for a key without additional round trips. Besides resharding, Cluster nodes will only talk to each other to keep the Cluster state updated. So the performance of a Cluster-aware benchmark will mainly benchmark how good the client deals with this.

Comment From: mattsta

A good client will cache the slot assignment and can calculate which instance to contact for a key without additional round trips.

A better client will query the slot assignment up front (CLUSTER SLOTS) and never have to deal with failed lookups!

transferring data across Redis nodes/servers in a cluster

As mentioned elsewhere, Redis Cluster nodes don't transfer data. Your clients talk to the server holding your data directly.

I feel that Redis Cluster can expand processing power

Yeah, that's a good point. In a Redis Cluster, your throughput should be the throughput of one master multiplied by your total number of masters (assuming you have no key hot spots), so any performance gains shouldn't be a surprise.

the performance of a Cluster-aware benchmark will mainly benchmark how good the client deals with this.

That's really the point here. Any Redis Cluster benchmark is mainly testing a client's implementation of Redis Cluster support. Redis Cluster nodes are just regular Redis servers with additional background processing.

It would be easier to write a cluster benchmark in a language with a currently working Redis Cluster client instead of trying to get it working in redis-benchmark right now.

But then what we really need is a specification for writing Redis Cluster client benchmarks so we can run the exact same tests everywhere and compare client performance across different languages and bindings.

Comment From: antirez

We could support this for now just in a trivial way. 1. Read the map at startup, directly inside redis-benchmark, and assume while we are benchmarking the cluster nobody will reshard it nor failures will occur, so we have just a fixed map. 2. Spawn a thread for each master in the cluster, and start the benchmark in each master using a different hash tag for each master, so that we are sure the keys will not be refused. 3. Use a mutex to update the number of served requests for each thread in the main thread. 4. Handle visualization in the main thread. 5. At the end report both total and each-node performances.