Protox

Protocol buffers implementation for Python 3 that generates human-readable code with type hinting support

Quick example

fromenumimportIntEnumfromprotoximportMessagefromprotoximportUInt32, String, EnumFieldclassUser(Message):
classRole(IntEnum):
USER=1ADMIN=2id: int=UInt32(number=1, required=True)
email: str=String(number=2, required=True)
role: Role=EnumField(Role, number=3, required=True, default=Role.USER)
user=User(id=1, email="john@doe.com", role=User.Role.USER)
user_data=user.to_bytes()
print(User.from_bytes(user_data))
# Outputs:# message User# id = 1# email = 'john@doe.com'# role = Role.USER

More examples

• EXAMPLES.md
• Grpclib example

Code generation

Code generator implemented as a protoc plugin so you have to install it first:

Install protoc (Ubuntu)

sudo apt install protobuf-compiler

Install protoc (Mac OS)

brew install protobuf

Install protox

python3 -m pip install protox

Generate messages

protoc \
--proto_path=protobuf_src \
--protox_out=. \
./protobuf_src/user.proto

Generate grpclib service with all required dependencies

The following code generates service ping_pong.proto from the protobuf_src directory into {$PROJECT_ROOT}/app/protobuf.

The protox plugin supports 3 options:

• --base-package=path/to/protobuf/out
• --grpclib # generates grpclib services
• --with-dependencies # generates dependencies for the given protobuf file

The options to the plugin are passed using the --protox_opt="{plugin options here}" command

protoc \
--proto_path=protobuf_src \
--protox_out=. \
--protox_opt="--base-package=app/protobuf --grpclib --with-dependencies" \
./protobuf_src/ping_pong.proto

Core concepts

• Human-readable python3.6+ generated code with type hinting
• Support protobuf 2 and 3 at the same time
• None values instead of zero values in fields for both proto2 and proto3
• Message.has_field() in both proto2 and proto3
• Protocols are easy to describe without code generation
• Messages implemented in more pythonic way: to_bytes() instead of SerializeToString()
• Enums are just enums python int enums

Features

• Messages
• Enums
• Nested messages
• Maps
• Well-known types
• Repeated fields
• Repeated messages
• Repeated enums
• Custom Message.to_python() / from_python() functions
• Group fields (Deprecated by protobuf)

Code generator features

• Protobuf
• Grpclib
• Grpc.io
• Custom python package for protobuf out messages
• Compile protobuf file with dependencies
• Names mangling to avoid reserved names collisions
• Recursive messages/enums support
• Field names to_snake_case support

Difference with google's protobuf implementation

Binary protocol works exactly as google's implementation does.

The difference is in the way messages behave:

• Fields that were not explicitly set are None rather than zero-values
• Methods like SerializeToString() were changed to more pythonic alternatives like to_bytes() / from_bytes()
• Enums are just python int enums

Generated code example

Grpclib service example

fromenumimportIntEnumimportprotoxclassUser(protox.Message):
classStatus(IntEnum):
USER=1ADMIN=2id: intemail: strstatus: 'User.Status'def__init__(
self,
*,
id: int=None,
email: str=None,
status: 'User.Status'=None,
):
super().__init__(
id=id,
email=email,
status=status,
)
protox.define_fields(
User,
id=protox.Int32(
number=1, required=True
),
email=protox.String(
number=2, required=True
),
status=protox.EnumField(
number=3, py_enum=User.Status, default=User.Status.USER, required=True
),
)