Redis, despite its reputation as an in-memory data store, is surprisingly aggressive about memory efficiency, and its object encoding is a prime example.
Let’s watch Redis in action. Imagine you’ve just started a fresh Redis instance and are adding a few simple keys:
redis-cli
127.0.0.1:6379> SET mykey "hello"
OK
127.0.0.1:6379> RPUSH mylist "a" "b" "c"
(integer) 3
127.0.0.1:6379> HSET myhash field1 "value1" field2 "value2"
(integer) 2
127.0.0.1:6379> ZADD myzset 1 "member1" 2 "member2"
(integer) 2
Now, let’s peek under the hood. The OBJECT ENCODING <key> command tells us what Redis is doing:
127.0.0.1:6379> OBJECT ENCODING mykey
"embstr"
127.0.0.1:6379> OBJECT ENCODING mylist
"ziplist"
127.0.0.1:6379> OBJECT ENCODING myhash
"ziplist"
127.0.0.1:6379> OBJECT ENCODING myzset
"ziplist"
Notice how Redis isn’t just storing raw strings or lists. It’s using specialized, compact encodings. This is the core of "Redis Object Encoding." It’s Redis’s internal strategy to use the smallest possible representation for your data structures based on their size and content. This saves RAM, allowing you to store more data in memory, which is crucial for Redis’s performance.
When you first create a data structure, Redis tries to use the most memory-efficient encoding. For simple string values, it uses EMBSTR if the string is short (up to 44 bytes by default). EMBSTR is special because it allocates a single contiguous memory block for both the Redis object and the string data itself. This is more cache-friendly and avoids an extra memory allocation compared to the more general RAW encoding, which would use a separate memory block for the string.
For collections like Lists, Hashes, and Sorted Sets, Redis initially uses ZIPLIST. A ziplist is a highly optimized, single-column data structure where elements are stored contiguously. It’s incredibly space-efficient for small collections because it avoids the overhead of separate nodes and pointers that a traditional linked list or hash table would require.
But what happens when these collections grow? Redis has thresholds. If a list exceeds a certain number of elements (list-max-ziplist-entries, default 512) or the total size of its elements exceeds a certain threshold (list-max-ziplist-value, default 64KB), Redis will automatically re-encode it into a more scalable, but less memory-compact, representation. For Lists, this typically becomes QUICKLIST. Quicklist is a hybrid: it’s a linked list of ziplists. This gives you the benefits of ziplists for small chunks of data while still allowing the list to grow to arbitrary sizes. For Hashes and Sorted Sets, exceeding the ziplist thresholds will re-encode them into HASH_TABLE and SKIPLIST respectively. These are more traditional data structures that offer better performance for larger collections but consume more memory.
You can also manually trigger re-encoding or inspect encodings. For example, if you add more elements to mylist:
127.0.0.1:6379> RPUSH mylist "d" "e" "f"
(integer) 6
127.0.0.1:6379> OBJECT ENCODING mylist
"ziplist"
If you were to continue pushing thousands of elements, you’d eventually see it change. You can also force a re-encoding to a specific type using OBJECT ENCODING <key> <encoding>. For example, OBJECT ENCODING mykey RAW would convert mykey to the standard string encoding if it was embstr.
The most surprising thing about Redis object encoding is that you don’t usually need to manage it directly. Redis’s autosave feature is remarkably good at picking the right encoding for your data based on its current size and content. The thresholds are tuned to balance memory savings against the performance cost of re-encoding. When a data structure grows beyond the ziplist limits, Redis performs a one-time cost to convert it to a more scalable structure. This is usually a worthwhile trade-off.
The real power of these encodings lies in their ability to adapt. For small, frequently accessed data, embstr and ziplist are incredibly efficient. As your data grows, Redis seamlessly transitions to structures like quicklist, hash_table, and skiplist that provide better performance characteristics for larger datasets, all without requiring you to change your application code.
Understanding these encodings helps you predict memory usage and diagnose performance issues. For instance, if you have a huge number of very small lists, you might be seeing memory inflation if they’ve all been re-encoded away from ziplist. You can tune the list-max-ziplist-entries and list-max-ziplist-value configuration parameters to adjust when these re-encodings happen, potentially keeping more of your small collections as ziplists if memory is a critical constraint and performance on very large collections isn’t paramount.
The next step in understanding Redis memory management is diving into how Redis handles eviction policies when memory limits are reached.