Fast, code-first protobuf serialization library for TypeScript.
Important: The library uses Buffer API, which is only available in Node.js and recent versions of Deno. It is not compatible with browser environments.
A message can be defined as follows:
import{ProtoField,ProtoMessage}from'@saltify/typeproto';constTestMessage=ProtoMessage.of({uint32Field: ProtoField(1,'uint32'),fixed32Field: ProtoField(2,'fixed32'),sint32Field: ProtoField(3,'sint32'),boolField: ProtoField(4,'bool'),stringField: ProtoField(5,'string'),nestedMessageField: ProtoField(6,{nestedField: ProtoField(1,'uint32'),}),repeatedMessageField: ProtoField(7,{nestedField: ProtoField(1,'uint32'),},'repeated'),repeatedPackedField: ProtoField(8,'uint32','repeated'),repeatedNotPackedField: ProtoField(9,'uint32','repeated',{packed: false}),mappedField: ProtoField(10,['uint32','string']),});... and serialized/deserialized as follows:
// The type of message is inferred automatically// If you break the structure, TypeScript will report compile-time errorsconstencoded=TestMessage.encode({uint32Field: 1,fixed32Field: 2,sint32Field: -1,boolField: true,stringField: 'test',nestedMessageField: {nestedField: 1,},repeatedMessageField: [{nestedField: 1},{},// Even an empty object is valid],repeatedPackedField: [1,2,3],repeatedNotPackedField: [4,5,6],mappedField: newMap([[1,'one'],[2,'two'],]),});// The decoded message also has the correct typeconstdecoded=TestMessage.decode(encoded);Note that all fields are not required when you pass an object to encode. If a non-optional field is not provided, it will be encoded as the default value (zero / empty string / false) of the field type.
You can declare protobuf map fields by passing a tuple to ProtoField:
constTestMessage=ProtoMessage.of({idToName: ProtoField(1,['uint32','string']),});constencoded=TestMessage.encode({idToName: newMap([[1,'Alice'],[2,'Bob'],]),});constdecoded=TestMessage.decode(encoded);// decoded.idToName is Map<number, string>The first element of the tuple is the key type and must be a protobuf-compatible scalar map key type (int32, uint32, int64, uint64, fixed32, fixed64, sfixed32, sfixed64, sint32, sint64, bool, or string). The second element is the value type and can be any scalar type or message type already supported by ProtoField.
You can define a message model and reuse it in multiple places:
constTestMessageA=ProtoMessage.of({uint32Field: ProtoField(1,'uint32'),});constTestMessageB=ProtoMessage.of({reusedField: ProtoField(1,TestMessageA),});You can also provide your model in a lambda:
constTestMessageA=ProtoMessage.of({uint32Field: ProtoField(1,'uint32'),});constTestMessageB=ProtoMessage.of({reusedField: ProtoField(1,()=>TestMessageA),});This could be useful when you want to circularly reference a message model in another file.
The ProtoMessage class has a property model, which is the model object provided to the of method. You can use it to get the model object, which is useful for some advanced use cases.
For example, you can refer to this object to compile the model object to protobuf file or other formats:
constTestMessage=ProtoMessage.of({uint32Field: ProtoField(1,'uint32'),});for(constfieldinTestMessage.model){// Do something with every field}InferProtoModel and InferProtoModelInput are the utility types that can be used to infer output and input types from the model object. When you pass an object to ProtoMessage<T>.encode, it will be checked against InferProtoModelInput<T>; and when you decode a buffer, the result will satisfy the type InferProtoModel<T>. This is useful when you want to use the model in a more generic way.
Assume that you defined a message model, and want to create a list which you can push objects of the model into. You then use the list to generate a list of encoded buffers:
constTestMessage=ProtoMessage.of({uint32Field: ProtoField(1,'uint32'),});constmessages=InferProtoModelInput<typeofTestMessage>[];// this is a shorthand for InferProtoModelInput<typeof TestMessage.model>messages.push({uint32Field: 1});messages.push({});// All fields are optional, so this is also validconstbuffers=messages.map((msg)=>TestMessage.encode(msg));@saltify/typeproto is 3x faster than unoptimized @protobuf-ts/runtime when serializing 100000 same messages in a row, and 1.4x faster when deserializing, according to the benchmark test.
The efficiency of this package comes from the fact that it "compiles" the model written in TypeScript and tends to expand all the logic ahead of use, instead of determining which part of the logic to use at runtime with tons of if and switch statements. The latter approach is the one used by @protobuf-ts/runtime, which is a general-purpose library that can be used in any environment, but it is not optimized for the specific use case of this project. protobuf-ts can also optimize the code by expanding the logic, but only when compiling proto files. This project did not use raw proto files, but instead used DSL-like code to describe the model, so it cannot benefit from the optimizations of protobuf-ts.
The DSL is inspired by @napneko/nap-proto-core, which uses @protobuf-ts/runtime as the backend. The DSL describes the model in an elegant way and is still valid TypeScript code, and can be used directly for type inference. This package fully adopted this style of model representation.
- Related article (Chinese) that explains the design and implementation of this project