I have the following docker-compose file for setup Redis-Cluster:
version: '3.3'
services:
redis_node1:
hostname: redis_node1
container_name: redis_node1
image: redis:6.0
expose:
- 7000
ports:
- "7000:7000"
- "17000:17000"
volumes:
- ./redis_node1.conf:/etc/redis.conf
command: redis-server /etc/redis.conf
networks:
app_net:
ipv4_address: 173.17.0.2
redis_node2:
hostname: redis_node2
container_name: redis_node2
image: redis:6.0
expose:
- 7001
ports:
- "7001:7001"
- "17001:17001"
volumes:
- ./redis_node2.conf:/etc/redis.conf
command: redis-server /etc/redis.conf
networks:
app_net:
ipv4_address: 173.17.0.3
redis_node3:
hostname: redis_node3
container_name: redis_node3
image: redis:6.0
expose:
- 7002
ports:
- "7002:7002"
- "17002:17002"
volumes:
- ./redis_node3.conf:/etc/redis.conf
command: redis-server /etc/redis.conf
networks:
app_net:
ipv4_address: 173.17.0.4
redis_cluster:
tty: yes
hostname: redis_cluster
container_name: redis_cluster
image: redis:6.0
expose:
- 6382
ports:
- "6382:6379"
depends_on:
- redis_node1
- redis_node2
- redis_node3
command: [ "redis-cli", "--cluster", "create", "173.17.0.2:7000", "173.17.0.3:7001", "173.17.0.4:7002", "--cluster-yes", "--cluster-replicas", "0" ]
networks:
app_net:
ipv4_address: 173.17.0.5
networks:
app_net:
driver: bridge
ipam:
driver: default
config:
- subnet: 173.17.0.0/16
Redis config (port different for each node):
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
C# code for performing write operations via StackExchange.Redis (on the host machine, not in docker):
public static void Main()
{
var connection = ConnectionMultiplexer.Connect(new ConfigurationOptions()
{
ConnectTimeout = 5000,
EndPoints =
{
new IPEndPoint(IPAddress.Parse("127.0.0.1"), 7000),
new IPEndPoint(IPAddress.Parse("127.0.0.1"), 7001),
new IPEndPoint(IPAddress.Parse("127.0.0.1"), 7002),
}
});
if (connection.IsConnected)
{
try
{
var endpoints = connection.GetEndPoints();
var db = connection.GetDatabase();
for (int i = 0; i < 100; i++)
{
db.StringSet(Guid.NewGuid().ToString(), "value");
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
Console.ReadLine();
}
}
Get an error at runtime:
StackExchange.Redis.RedisConnectionException: Endpoint 173.17.0.3:7001 serving hashslot 10166 is not reachable at this point of time. Please check connectTimeout value. If it is low, try increasing it to give the ConnectionMultiplexer a chance to recover from the network disconnect. IOCP: (Busy=0,Free=1000,Min=12,Max=1000), WORKER: (Busy=0,Free=32767,Min=12,Max=32767), Local-CPU: n/a
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2817
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/RedisBase.cs:line 54
at StackExchange.Redis.RedisDatabase.StringSet(RedisKey key, RedisValue value, Nullable`1 expiry, When when, CommandFlags flags) in /_/src/StackExchange.Redis/RedisDatabase.cs:line 2536
at ....Program.Main() in ..../Program.cs:line 32
Not sure if this is a problem driver, looks like var endpoints = connection.GetEndPoints(); return private a public IPs and when trying to get and hash slot for a key is thrown exception.
Could anyone suggest a solution to this problem or describe this behavior? Thanks.
Comment From: mgravell
question: what do you get if you issue, at the command-line from the machine that is running the failing code:
redis-cli -h 173.17.0.3 -p 7001 dbsize
(what I'm trying to ascertain here is: is the server's IP address routable, and is the port accessible - firewalls etc)
Comment From: mgravell
to clarify: the IP addresses that you pass to Connect/ConnectAsync are used to make the initial connection the cluster, but we then ask the cluster what the topology is, and talk to the IP addresses that the cluster advertises. If those addresses happen to be the same, then great - but if they're different: the cluster is taken as the authority. This is so that, for example, you can add/remove/etc nodes from the cluster, and as long as some node is reachable during connect: the entire cluster will light up.
Comment From: rdcm
The host machine does not have redis-cli, only docker API.
Run interactive mode:
docker exec -it redis_node2 /bin/bash
Execute cli command:
redis-cli -h 173.17.0.3 -p 7001 dbsize
(integer) 1
173.17.0.* addresses from docker mapped to localhost and not exposed on host machine
![Redis [QUESTION] Unable write values to Redis Cluster](https://www.nineya.com/upload/2025/1/2009424271035577)
Comment From: mgravell
173.17.0.* addresses from docker mapped to localhost
That is largely irrelevant; the library is going to use the addresses that the cluster advertises via CLUSTER NODES (it should really be updated to use CLUSTER SLOTS, but that didn't exist when the topology code was written, and I haven't yet gone back to it, but it doesn't matter: the data is the same either way)
and not exposed on host machine
But are they reachable from the host machine. If those IPs are private to docker, then: yeah, that isn't going to work. At a push, you could use loopback addresses for both and abuse the fact that you have them port-forwarded, but: that's not really a good idea IMO. The ideal here is that the server advertises IP addresses that are routable and reachable from the client.
If you don't have redis-cli on the host, you can usually use other tools; for example, on Windows you can use telnet:
> telnet
> open 173.17.0.3 7001
> ping
< +PONG
> quit
< +OK
(noting that the ping appears at the top of the screen, because the terminal sucks)
Comment From: rdcm
I'm back.
I solved this issue after moving an application to docker and after this action, the app gets an IP address in the same network. Also for successful cluster setup needed these commands:
#!/bin/bash
redis-cli --cluster create 173.17.0.2:7000 173.17.0.3:7001 173.17.0.4:7002 --cluster-yes --cluster-replicas 0
redis-cli -c -h 173.17.0.2 -p 7000 cluster meet 173.17.0.2 7000
redis-cli -c -h 173.17.0.2 -p 7000 cluster meet 173.17.0.3 7001
redis-cli -c -h 173.17.0.2 -p 7000 cluster meet 173.17.0.4 7002
Same issue here: https://github.com/StackExchange/StackExchange.Redis/issues/634
It works and I can play with the Redis cluster.
But for me, this behavior looks wrong. https://github.com/redis/redis/issues/8485#issuecomment-778159517
Because docker provides a way to set up any storage in any configuration without break connection logic for the application. And with redis image this not possible now, (the host machine knows nothing about docker ips and this fact break connection).
Also redis not support docker nat now: https://redis.io/topics/cluster-tutorial
Redis Cluster and Docker
Currently, Redis Cluster does not support NATted environments and in general, environments where IP addresses or TCP ports are remapped
Would be great, if Redis will have docker nat support.
Comment From: yossigo
@rdcm Did you look at the cluster-announce-* configuration parameters? Seems like the tutorial is not up to date but you should be able to explicitly specify the external address and port of nodes.