The Redis replication backlog, often called the repl_backlog_size, isn’t just a buffer; it’s the persistent memory of the primary’s write-ahead log (WAL) that replicas use to catch up without a full resynchronization.
Let’s see this in action. Imagine a primary Redis instance (redis-cli -h primary) and a replica (redis-cli -h replica).
On the primary, we can inspect the replication backlog:
redis-cli -h primary INFO replication
You’ll see output like this:
connected_clients:2
client_recent_max_input_buffer:2
client_recent_max_output_buffer:3
blocked_clients:0
replication_backlog_first_byte_offset:1000
replication_backlog_size:1048576
master_replid:a1b2c3d4e5f67890abcdef0123456789
master_replid2:00000000000000000000000000000000
master_port:6379
repl_backlog_active:1
last_repl_stream_offset:1050
repl_backlog_histlen:50
The key players here are replication_backlog_size (total allocated size), replication_backlog_first_byte_offset (where the current backlog started), and master_replid (the primary’s unique ID for this replication stream).
Now, let’s simulate a replica going down and then coming back up. If the backlog is too small, the replica might not be able to find its last known position within it.
When the replica reconnects, it sends its REPLCONF ACK command with the offset it last received. The primary checks if that offset is still within the active backlog. If the replica’s offset is before replication_backlog_first_byte_offset, the primary says, "Sorry, I don’t have that historical data anymore. You’ll need to do a full SYNC."
A full SYNC involves the primary creating a RDB snapshot, sending it over the network to the replica, and the replica then loading that snapshot into memory. This is a heavy operation, blocking the primary and potentially causing downtime or performance issues on the replica.
The replication_backlog_size is configured in redis.conf (or via CONFIG SET) and is the maximum size in bytes that Redis will use for this circular buffer. When the buffer fills up, older data is overwritten. The master_replid2 field is used for a brief window during failover to allow replicas to continue from an older replid if the primary fails and a new primary is elected using a previous replid. However, the repl_backlog_size is the primary mechanism for day-to-day catch-up.
To size it correctly, you need to estimate the write rate of your primary and the maximum acceptable downtime for your replicas. A common formula is:
repl_backlog_size = (primary_write_rate_in_bytes_per_second * max_replica_catchup_time_in_seconds)
Let’s say your primary writes an average of 50 KB/s (51200 bytes/s) and you can tolerate a replica being offline for up to 10 minutes (600 seconds) before it must be able to catch up without a full SYNC.
repl_backlog_size = 51200 bytes/s * 600 s = 30,720,000 bytes
So, you’d set repl_backlog_size 31457280 (using the nearest power of 2, typically 32MB).
You can set this dynamically:
redis-cli -h primary CONFIG SET repl_backlog_size 31457280
Or in redis.conf on the primary:
repl_backlog_size 31457280
After changing this, restart the primary for the redis.conf change to take effect. The backlog is created when the first replica connects.
The repl_backlog_histlen value in the INFO replication output shows the current number of bytes written into the backlog since replication_backlog_first_byte_offset. If repl_backlog_histlen consistently approaches replication_backlog_size, it’s a strong indicator that your backlog is too small for your workload.
A very common mistake is to set repl_backlog_size too small, leading to frequent full SYNC operations during transient network issues or replica restarts. This can create a vicious cycle: a full SYNC puts load on the primary, potentially causing more instability, leading to more SYNCs.
Another factor is the network bandwidth between your primary and replicas. If a full SYNC takes a very long time due to slow network, you might need a larger backlog to accommodate that extended downtime.
The master_replid2 and repl_backlog_histlen are linked. When the backlog wraps around, master_replid2 stores the master_replid from before the wrap, and replication_backlog_first_byte_offset is updated. This allows replicas that connected with the old master_replid to still find their data for a short period.
If you consistently see repl_backlog_histlen equal to replication_backlog_size, it means the backlog is full and old data is being lost. This is the direct signal that you need to increase repl_backlog_size.
The next problem you’ll likely encounter after correctly sizing your replication backlog is managing the network latency and throughput required for efficient RDB snapshot transfers during the initial PSYNC or FULLRESYNC operations.