The Redis RDB snapshotting process is failing because the memory allocated to Redis has become too fragmented.

Common Causes and Fixes

  1. High Fragmentation Ratio: Redis itself reports fragmentation.

    • Diagnosis: Connect to Redis using redis-cli and run INFO memory. Look for the mem_fragmentation_ratio value. A ratio significantly above 1.0 (e.g., > 1.5 or 2.0) indicates fragmentation.
    • Fix: Restart Redis. This is the most straightforward way to defragment memory because Redis will re-allocate memory from the OS in a contiguous block.
      redis-cli shutdown
      # Wait for Redis to stop, then restart it using your usual method (e.g., systemctl start redis)
      
    • Why it works: When Redis restarts, it requests fresh memory from the operating system. The OS typically provides larger, more contiguous chunks of memory, effectively reducing fragmentation.
  2. Large Object Evictions: When Redis needs to evict old data to make space for new data, it might remove large keys. This can leave gaps in memory.

    • Diagnosis: Monitor evicted_keys in INFO memory. If this number is high, and you observe fragmentation, this is a likely cause.
    • Fix: Increase maxmemory and/or tune your eviction policy. If you’re using volatile-lru or allkeys-lru, consider increasing maxmemory to allow more data before eviction is needed. If eviction is unavoidable, a policy like volatile-random or allkeys-random might distribute the impact better, though it’s less predictable.
      • In redis.conf:
        maxmemory 4gb
        maxmemory-policy allkeys-lru
        
      • To apply immediately without restart:
        redis-cli config set maxmemory 4294967296 # 4GB in bytes
        redis-cli config set maxmemory-policy allkeys-lru
        
    • Why it works: A larger maxmemory limit means Redis can hold more data before needing to evict, reducing the frequency of these operations. Changing the policy can alter how eviction happens, potentially leading to less severe fragmentation patterns.
  3. Frequent Deletions of Large Keys: Similar to evictions, deleting large keys can create holes.

    • Diagnosis: Monitor the rate of DEL commands or use Redis Slow Log to identify large keys being deleted frequently.
    • Fix: If possible, avoid deleting very large keys frequently. If necessary, consider using Redis’s UNLINK command (available in Redis 4.0+). UNLINK performs deletion in a background thread, freeing memory incrementally without blocking the main Redis thread.
      redis-cli UNLINK my_very_large_key
      
    • Why it works: UNLINK defers the actual memory deallocation to a background process, preventing large, sudden memory holes and keeping the main Redis thread responsive.
  4. Redis Version Incompatibility or Bugs: Older Redis versions had more significant fragmentation issues.

    • Diagnosis: Check your Redis version: redis-cli --version.
    • Fix: Upgrade to the latest stable Redis version. Redis 4.0 introduced background deallocation (UNLINK), and subsequent versions have continued to improve memory management.
      # Example for updating on Debian/Ubuntu
      sudo apt update
      sudo apt upgrade redis-server
      
    • Why it works: Newer versions incorporate performance and memory management improvements that directly address fragmentation.
  5. Operating System Memory Allocator: Redis uses the system’s memory allocator (like jemalloc or glibc malloc). The allocator’s behavior significantly impacts fragmentation.

    • Diagnosis: Check which allocator is being used. If using jemalloc, its fragmentation statistics can be accessed.
      • If jemalloc is installed:
        # Check if jemalloc is being used (e.g., by checking loaded libraries)
        # Or, if you compiled Redis yourself with jemalloc, check your build flags.
        # If jemalloc is configured and used, you can check its stats.
        # This is more advanced and often requires modifying jemalloc's configuration
        # or using tools like `pmap` and analyzing memory maps.
        
      • A common indicator is if mem_fragmentation_ratio is high but maxmemory isn’t being approached aggressively.
    • Fix: Ensure jemalloc is installed and configured for Redis. jemalloc is generally better at handling fragmentation than the default glibc allocator. Redis often uses jemalloc by default when compiled on systems where it’s available. If you are using glibc, consider switching.
      • Install jemalloc:
        # On Debian/Ubuntu
        sudo apt install libjemalloc-dev
        # On CentOS/RHEL
        sudo yum install jemalloc-devel
        
      • Recompile Redis with jemalloc support if not already using it. If it is already using jemalloc, tune jemalloc’s own configuration parameters (e.g., MALLOC_CONF) which is outside the scope of Redis configuration itself.
    • Why it works: jemalloc is designed with high-performance, concurrent applications in mind and often exhibits lower fragmentation than standard malloc implementations.
  6. Manual Defragmentation (Not Recommended for Production): Redis 4.0+ has a DEBUG command for defragmentation, but it’s blocking and should be used with extreme caution.

    • Diagnosis: mem_fragmentation_ratio is high, and restarts are inconvenient or impossible.
    • Fix: Use the DEBUG command as a last resort, understanding its implications.
      redis-cli DEBUG memory --fragmentation-device /path/to/redis.rdb
      
      Note: This command is experimental, can take a very long time, and will block your Redis instance. It’s generally better to restart.
    • Why it works: This command attempts to rewrite the RDB file in memory, compacting it. However, its blocking nature makes it impractical for live systems.

The next error you’ll likely encounter after fixing fragmentation is related to network connectivity or client timeouts if the RDB saving process was being interrupted.

Want structured learning?

Take the full Redis course →