Virtual nodes are the secret sauce that makes sharding flexible, allowing you to rebalance your data without disrupting live traffic.
Let’s see this in action. Imagine a simple Redis cluster with two primary shards, 0 and 1. Each shard is responsible for a contiguous range of hash slots.
# Get the current slot distribution
redis-cli -c -p 6379 CLUSTER SLOTS
Output:
1) 1) 0
2) 16383
3) 1) "127.0.0.1"
2) (integer) 6379
2) 1) 16384
2) 16383
3) 1) "127.0.0.1"
2) (integer) 6380
Here, shard 6379 owns slots 0 through 16383, and shard 6380 owns slots 16384 through 16383 (which is actually a typo in the output, it should be 0 to 8191 and 8192 to 16383 if we had two nodes for 16384 slots).
Now, let’s introduce virtual nodes. Instead of assigning a single contiguous range of slots to a physical node, we assign multiple, non-contiguous "virtual" slots. Each physical node can manage many virtual nodes.
# Configure virtual nodes on node 6379
redis-cli -c -p 6379 CLUSTER SET_CONFIG_EPOCH 1
redis-cli -c -p 6379 CLUSTER ADDSLOTSRANGE 0 8191 # Assign first half of slots to node 6379
redis-cli -c -p 6379 CLUSTER SET_CONFIG_EPOCH 2
redis-cli -c -p 6379 CLUSTER SET_CONFIG_EPOCH 3
redis-cli -c -p 6379 CLUSTER SET_CONFIG_EPOCH 4
# Configure virtual nodes on node 6380
redis-cli -c -p 6380 CLUSTER SET_CONFIG_EPOCH 1
redis-cli -c -p 6380 CLUSTER ADDSLOTSRANGE 8192 16383 # Assign second half of slots to node 6380
redis-cli -c -p 6380 CLUSTER SET_CONFIG_EPOCH 2
redis-cli -c -p 6380 CLUSTER SET_CONFIG_EPOCH 3
redis-cli -c -p 6380 CLUSTER SET_CONFIG_EPOCH 4
This is the traditional way. Virtual nodes change how we think about ADDSLOTS. Instead of assigning large ranges, we assign individual slots, and Redis internally distributes these slots across virtual nodes. The number of virtual nodes per physical node is configured using cluster-replicas when you create the cluster, or by adding new nodes and letting redis-trib.rb or redis-cli --cluster create handle the distribution.
Let’s say we have a cluster with 2 physical nodes and we want 100 virtual nodes per physical node. The total number of slots is 16384. So, each physical node will manage 16384 / 2 = 8192 slots. With virtual nodes, we don’t assign ranges anymore. We let the cluster management tool distribute them.
When you add a new physical node to an existing cluster, say node 6381, you don’t manually reassign slots. Instead, you add the new node, and the cluster management tool (like redis-cli --cluster rebalance) automatically migrates a portion of the slots from the existing nodes to the new node. This is where virtual nodes shine. Because each physical node is responsible for many small, non-contiguous sets of slots (its virtual nodes), migrating a subset of these virtual nodes to a new physical node is a much finer-grained operation. It doesn’t require moving entire ranges, which could be massive.
The core idea is that instead of one physical node owning 0-8191, it might own 0, 2, 4, ..., 16382. Another node owns 1, 3, 5, ..., 16383. When you add a third node, it might take 0, 1, 2, 3, ... from both, but in a distributed fashion. This allows for smooth, incremental data movement without impacting availability. The client libraries are aware of the cluster topology and will automatically redirect requests to the correct node, even during rebalancing.
What most people miss is that virtual nodes aren’t a separate Redis feature you enable with a config flag. They are an emergent property of how redis-cli --cluster create and redis-cli --cluster rebalance distribute the 16384 hash slots across the physical nodes, especially when cluster-replicas is greater than 0. The cluster-replicas parameter determines how many additional physical nodes are created for each primary. If you have N primaries and M replicas per primary, you have N * (M+1) total nodes. The redis-cli tool then distributes the 16384 slots among the N primary nodes, and each primary node effectively "owns" a set of virtual nodes, which are simply individual hash slots. The rebalancing command then moves these individual hash slots (virtual nodes) around.
The next concept to grapple with is how client libraries handle cluster topology changes and redirection.