Redis Cluster automatically shards data across multiple nodes, but the initial setup and understanding of how it works can be a bit of a black box.
Let’s watch a simple cluster come to life. Imagine we have three physical machines, redis-1, redis-2, and redis-3, all running a fresh Redis 6.2 instance.
First, on each node, we need to tell Redis it’s part of a cluster and which port to use for cluster communication. This is done in redis.conf:
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-announce-ip 192.168.1.101 # Use the actual IP of the node
cluster-announce-port 6379
cluster-announce-bus-port 16379
The cluster-config-file is where Redis stores cluster state for that node. cluster-announce-ip and cluster-announce-port are crucial for nodes to find each other. cluster-announce-bus-port is for the cluster bus, which is cluster-announce-port + 10000.
Now, let’s start the Redis instances on each machine.
# On redis-1
redis-server /etc/redis/redis.conf
# On redis-2
redis-server /etc/redis/redis.conf
# On redis-3
redis-server /etc/redis/redis.conf
With the nodes running, we need to tell them to form a cluster. This is done using the redis-cli --cluster create command. We need to specify the IP and port of at least one node in the cluster, and then list all the nodes that should participate. Let’s say we want three master nodes initially, no replicas for now.
redis-cli --cluster create 192.168.1.101:6379 192.168.1.102:6379 192.168.1.103:6379 --cluster-replicas 0
The --cluster-replicas 0 flag tells redis-cli to create a cluster with only master nodes. If we wanted one replica for each master, we’d use --cluster-replicas 1.
The redis-cli will then propose a configuration, showing which hash slots will be assigned to which master. For example:
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Can I set the above configuration? (type 'yes' to accept):
Type yes and hit enter. The redis-cli will then send commands to each node to configure them as a cluster, assign them hash slots, and establish peer discovery.
After this, if you connect to any node with redis-cli -c (the -c enables cluster mode, which automatically redirects you if you hit the wrong node), you can check the cluster status:
redis-cli -c -p 6379
192.168.1.101:6379> cluster nodes
You’ll see output like this, showing each node and its role:
a1b2c3d4e5f6... 192.168.1.101:6379@16379 master - 0 1678886400000 1 connected 0-5460
f0e1d2c3b4a5... 192.168.1.102:6379@16379 master - 0 1678886400000 2 connected 5461-10922
9a8b7c6d5e4f... 192.168.1.103:6379@16379 master - 0 1678886400000 3 connected 10923-16383
The numbers after connected are the hash slot ranges assigned to each master. Redis uses a hashing algorithm (CRC16) on the key name to determine which hash slot a key belongs to. There are 16384 total hash slots. If a key’s hash slot falls within a range assigned to a specific master, that master is responsible for storing and serving that key.
When you perform a SET mykey value operation, redis-cli (in cluster mode) first calculates the hash slot for mykey. It then checks which node is responsible for that slot. If you happen to be connected to the wrong node, it will send a MOVED redirect to your client, telling it which node to connect to for that specific key.
# Connected to redis-1
192.168.1.101:6379> SET mykey hello
-> Redirected to slot [12182] via -3945394539453945394539453945394539453945:5461
-> Trying to redirect to slot [12182] via 192.168.1.103:6379
192.168.1.103:6379> SET mykey hello
OK
This automatic sharding is the core of Redis Cluster’s scalability. Each master node handles a subset of the hash slots, distributing the load. Adding replicas with --cluster-replicas 1 (or higher) makes the cluster fault-tolerant. If a master fails, one of its replicas can be promoted to take its place.
The cluster-config-file is dynamically updated. If you add a new node or reconfigure the cluster, this file will reflect the changes, so Redis knows its role and peers even after a restart.
The trickiest part is often network configuration. All nodes must be able to reach each other on both their client ports and their cluster bus ports. Firewall rules are common culprits.
The next challenge is understanding how to manage resharding a live cluster without downtime.