Protobuf evolution is surprisingly easy, but most people break it by treating it like a traditional schema evolution problem.
Let’s see Protobuf evolution in action. Imagine we have a User message with a name field.
syntax = "proto3";
message User {
string name = 1;
}
We deploy this, and it works. Now, we want to add an email field.
syntax = "proto3";
message User {
string name = 1;
string email = 2; // New field
}
If we deploy this change, existing clients that don’t know about email will simply ignore it. New clients will see both name and email. This is the "backward compatibility" Protobuf gives you for free.
But what about "forward compatibility"? If a new client using the User with email field sends data to an old server that only understands name, the server will just ignore the email field. It won’t crash. It will process the name correctly. This is the "forward compatibility" Protobuf offers.
The core problem Protobuf solves is efficient, language-agnostic serialization. It’s not about complex schema evolution with strict rules like XML Schema or JSON Schema. Protobuf’s philosophy is "add fields, don’t change existing ones."
Here’s how it works internally: each field has a unique number. When you serialize, it’s not {"name": "Alice", "email": "alice@example.com"}; it’s more like 1: "Alice", 2: "alice@example.com". When a parser encounters a field number it doesn’t recognize, it skips it. This is the magic.
The mental model is this: your API is a set of messages. When you send a message, you’re sending a serialized byte stream. The receiver deserializes it. If the receiver’s definition of the message has more fields than the sender’s, the extra fields are ignored. If the receiver’s definition has fewer fields than the sender’s, the unknown fields are simply skipped.
The critical lever you control is the field number. Never reuse a field number. If you mark a field as deprecated and remove it from your .proto file, you cannot, under any circumstances, use that field number for a new field in the same message. If you do, old clients sending data with that field number will have their new data parsed into the old field, leading to silent corruption.
The next concept you’ll grapple with is handling breaking changes, like removing a field entirely or changing a field’s type.