Redis can often consume more RAM than expected, and optimizing it is crucial for keeping infrastructure costs down and performance up.
Let’s see what a busy Redis instance looks like under the hood. Imagine a simple key-value store where we’re tracking user session data.
# Simulate adding some session data
redis-cli SET user:123:session '{"user_id": 123, "last_login": "2023-10-27T10:00:00Z", "cart_items": 5}'
redis-cli SET user:456:session '{"user_id": 456, "last_login": "2023-10-27T10:05:00Z", "cart_items": 2}'
redis-cli SET user:789:session '{"user_id": 789, "last_login": "2023-10-27T10:10:00Z", "cart_items": 0}'
# Check memory usage for a specific key
redis-cli MEMORY USAGE user:123:session
# Output: 118 (bytes)
# Get overall memory usage
redis-cli INFO memory
# Output (truncated):
# used_memory:12345678
# used_memory_human:11.77M
# used_memory_rss:15678901
# peak_memory:20000000
# peak_memory_human:19.07M
This shows us actual memory in bytes and human-readable formats, along with peak usage, giving us a snapshot of what’s happening.
The core problem Redis memory optimization solves is the gap between what you think Redis is using and what it actually is. This often stems from implicit overheads, inefficient data structures, and unnecessary data. Redis isn’t just storing your keys and values; it’s also managing metadata, internal data structures, and potentially replication buffers, all of which consume RAM.
The mental model for Redis memory management involves three key areas:
- Data Storage: This is the most direct consumption. Each key-value pair occupies memory. The type of value (string, list, hash, set, sorted set) and its serialization format matter significantly.
- Overhead: This includes memory for the Redis server process itself, connection management, replication buffers, AOF/RDB persistence mechanisms, and internal data structures used by Redis to manage keys and data.
- Eviction/Expiration: If you have a TTL on keys or use an eviction policy, Redis needs to track expiration times and manage which keys to remove when memory limits are reached.
To control this, you have levers like:
- Data Structure Choice: Using Hashes for objects with many fields is often more memory-efficient than individual keys for each field.
- Serialization: JSON, MessagePack, or Protocol Buffers can be more compact than plain strings for complex data, but add CPU overhead.
- Key Naming: Shorter, consistent key prefixes can save a small amount of memory per key.
maxmemoryConfiguration: Setting a hard limit to prevent runaway memory usage.- Eviction Policies: Choosing how Redis discards data when
maxmemoryis hit (e.g.,allkeys-lru,volatile-lru). lazyfree-lazy-eviction: Offloading deletion of large keys to a background thread.jemalloc: Using a more memory-efficient allocator than the defaultglibcmalloc.
When you execute redis-cli INFO memory, you’re not just seeing the sum of MEMORY USAGE for all your keys. used_memory is the total bytes allocated by Redis. used_memory_rss is the Resident Set Size, which includes memory allocated by Redis but also potentially shared libraries and pages that the OS might swap out. The difference between these can indicate fragmentation or shared memory usage.
Here’s a common scenario: you’re storing user profiles as individual keys, like user:123:name, user:123:email, user:123:prefs. This leads to significant overhead because each key-value pair incurs Redis’s per-key data structures (hash table entry, pointer, etc.).
Instead, consolidate this into a single Redis Hash:
# Instead of many keys:
# redis-cli SET user:123:name "Alice"
# redis-cli SET user:123:email "alice@example.com"
# redis-cli HSET user:123 name "Alice" email "alice@example.com"
This uses one Redis key (user:123) and stores its fields internally within the Hash data structure, drastically reducing the per-key overhead.
The maxmemory directive in your redis.conf is your primary defense. If you set maxmemory 2gb, Redis will start evicting keys (based on your maxmemory-policy) or return errors when this limit is reached. It’s crucial to understand that maxmemory is a limit on the Redis server’s allocated memory, not necessarily the used_memory_rss.
The maxmemory-policy is equally important. allkeys-lru (Least Recently Used) removes keys that haven’t been accessed recently, regardless of whether they have an expiration set. volatile-lru only considers keys with an expiry set. Choosing the right policy depends entirely on what kind of data you want to keep in memory.
One optimization that people often overlook is the memory allocator. By default, Redis might use the system’s malloc (typically glibc). However, jemalloc is often more memory-efficient, especially for the kind of memory allocation patterns Redis uses. Compiling Redis with jemalloc or ensuring your OS package installs it can lead to a noticeable reduction in used_memory_rss without changing your application code. You can check which allocator Redis is using by looking at the INFO output.
If your Redis instance is serving a high volume of writes and deletions, lazyfree-lazy-eviction can be a lifesaver. When enabled, Redis can unlink and free large keys (like large lists or sets) in a background thread, preventing the main thread from blocking during memory reclamation. This is configured with lazyfree-lazy-eviction yes.
The next step after optimizing memory is often dealing with the performance implications of a very large dataset or complex data structures.