Generating JavaScript types from Protobuf definitions is often about more than just having .js files.
Here’s a Protobuf definition for a simple user message:
syntax = "proto3";
package com.example.users;
message User {
string user_id = 1;
string name = 2;
int32 age = 3;
repeated string tags = 4;
}
Running the Protobuf compiler (protoc) with the JavaScript plugin (--js_out) will produce a .js file. Let’s say you’re using a version like 3.19.4. The command might look like this:
protoc --proto_path=. --js_out=import_style=commonjs,binary:./generated ./user.proto
This creates ./generated/user_pb.js. Inside, you’ll find JavaScript code that defines classes corresponding to your Protobuf messages. For our User message, you’d see something like:
// Inside generated/user_pb.js (simplified)
goog.provide('com.example.users.User');
/**
* @generated
* @param {proto.com.example.users.User.Array} array
* @constructor
*/
proto.com.example.users.User = function(array) {
// ... internal initialization ...
};
// ... other methods like toObject, toArray, etc.
This code provides the runtime for working with Protobuf messages in JavaScript. You can instantiate User objects, set their fields, and serialize/deserialize them.
const user_pb = require('./generated/user_pb.js');
const newUser = new user_pb.com.example.users.User();
newUser.setUserId('abc-123');
newUser.setName('Alice');
newUser.setAge(30);
newUser.addTags('developer');
newUser.addTags('javascript');
const bytes = newUser.serializeBinary();
console.log('Serialized bytes:', bytes);
const decodedUser = user_pb.com.example.users.User.deserializeBinary(bytes);
console.log('Decoded User:', decodedUser.toObject());
The crucial insight here is that protoc --js_out generates runtime code, not static type definitions for tools like TypeScript. The generated JavaScript is designed to be executed, providing the logic for encoding and decoding Protobuf messages.
The problem many developers face is that they expect this generated .js file to provide type hints for their IDE or TypeScript compiler. It doesn’t. The JavaScript generated by protoc is designed for runtime operation, not static analysis.
To get actual TypeScript types, you need a different toolchain. The most common approach involves using protobufjs or ts-proto.
Using protobufjs:
protobufjs can parse .proto files and generate both JavaScript runtime code and TypeScript definitions.
First, install it:
npm install -g protobufjs
Then, you can generate files:
pbjs -t static-module --target ts -w commonjs -o ./generated/user_types.js ./user.proto
pbjs -t static --target js -w commonjs -o ./generated/user_runtime.js ./user.proto
This command does two things:
-t static-module --target ts -w commonjs -o ./generated/user_types.js: Generates a single JavaScript file (user_types.js) that also contains TypeScript definitions. This file can berequired orimported, and your TypeScript compiler will pick up the types. Thestatic-moduleandstatictargets embed the Protobuf definitions within the generated code, making it self-contained.-t static --target js -w commonjs -o ./generated/user_runtime.js: Generates a separate JavaScript file containing the runtime code for encoding/decoding.
Your TypeScript code would then look like this:
// Assuming you've configured your tsconfig.json to resolve './generated/user_types.js'
// or are using a bundler that handles it.
import { User } from './generated/user_types'; // This import points to the generated JS file with TS types
// The actual runtime code for encoding/decoding needs to be imported separately
// or is implicitly available if you used a combined generation approach.
// For simplicity, let's assume protobufjs makes the types available directly.
// If using a separate runtime generation:
// import { User as RuntimeUser } from './generated/user_runtime'; // This might be needed depending on generation strategy
const newUser: User = {
userId: 'abc-123',
name: 'Alice',
age: 30,
tags: ['developer', 'javascript'],
};
// In a real scenario, you'd use protobufjs's encode/decode functions
// which are often part of the generated runtime.
// Example using protobufjs's API directly after loading the definitions:
// import * as protobuf from 'protobufjs';
// const root = protobuf.parse(your_proto_file_content_as_string).root;
// const UserMessage = root.lookupType('com.example.users.User');
// const buffer = UserMessage.encode(newUser).finish();
// const decoded = UserMessage.decode(buffer);
console.log('User object:', newUser);
// The actual serialization/deserialization would use protobufjs functions
Using ts-proto:
ts-proto is another popular tool that directly generates TypeScript code from .proto files. It’s often preferred for its cleaner output and better integration with modern TypeScript features.
First, install it:
npm install -g ts-proto
Then, run the generation:
protoc --plugin=protoc-gen-ts_proto=$(which ts-proto) --ts_proto_out=./generated --ts_proto_opt=nestJs=true,useDate=true ./user.proto
This generates generated/user.ts. The nestJs=true option is for integration with NestJS, but ts-proto produces excellent standalone TypeScript.
Your TypeScript code would then be:
// Assuming generated/user.ts is in your TS_NODE_PATH or imported correctly
import { User } from './generated/user'; // This is a pure .ts file
const newUser: User = {
userId: 'abc-123',
name: 'Alice',
age: 30,
tags: ['developer', 'javascript'],
};
// You'll still need a Protobuf runtime library for actual serialization/deserialization.
// ts-proto doesn't generate the runtime; it generates the types and helper functions
// that work *with* a runtime. You'd typically use libraries like @protobuf-ts/plugin
// or the runtime from protobufjs for this.
// Example using @protobuf-ts/runtime:
import { Message, Type, Field, Map, Enum } from "@protobuf-ts/runtime";
import { User as UserProto } from './generated/user'; // From ts-proto generation
// This part is conceptual, as actual runtime integration depends on the chosen runtime library.
// You'd typically import the generated message class and use its methods.
// For example, if ts-proto generated an encoder/decoder class:
// const encoded = UserProto.encode(newUser); // Hypothetical, actual API varies
// const decoded = UserProto.decode(encoded);
console.log('User object:', newUser);
The key difference is that ts-proto focuses on generating idiomatic TypeScript, including interfaces or classes that fully represent your Protobuf messages. You then pair this with a separate Protobuf runtime library (like @protobuf-ts/runtime) for the actual binary encoding and decoding.
The most surprising true thing about generating Protobuf JavaScript types is that the default protoc --js_out command doesn’t give you types for static analysis; it only gives you runtime code for serialization and deserialization.
Consider a scenario where you’re building a complex microservice architecture. Your backend services might be written in Go or Java, using their respective Protobuf code generation. Your frontend, however, is in TypeScript. You need a consistent way to define and use your message structures across these different environments. The challenge is bridging the gap between the binary Protobuf format and the structured, type-safe world of TypeScript.
When you use tools like ts-proto, the generated TypeScript code often looks very much like interfaces or classes you would write yourself.
// Example generated by ts-proto (simplified)
export interface User {
userId: string;
name: string;
age: number;
tags: string[];
}
This generated User interface can then be used throughout your TypeScript codebase. You can create objects that conform to this interface, pass them to functions expecting a User, and your IDE will provide autocompletion and type checking.
However, this generated User interface is just a shape. It doesn’t know how to serialize itself into the Protobuf binary format or deserialize from it. For that, you need a Protobuf runtime library. Libraries like @protobuf-ts/runtime or the runtime provided by protobufjs contain the logic to translate between these TypeScript structures and the Protobuf binary.
The one thing most people don’t know is that the mapping between the Protobuf wire format and your generated TypeScript types isn’t always a direct one-to-one translation for all data types. For instance, Protobuf’s Timestamp type, which is a message itself, is often mapped to JavaScript’s Date object by generation tools. Similarly, Duration might map to an object with seconds and nanos properties, or a specific numeric representation. Understanding these default mappings, and how to customize them if necessary (e.g., using ts-proto options like useDate=true), is key to seamless integration.
The next concept you’ll likely encounter is handling Protobuf oneof fields and their implications for type safety in your generated code.