scg - Simple Code Generator

A lightweight, high-performance code generator for messages and RPC client/server boilerplate.

SCG provides efficient serialization and RPC performance while maintaining a significantly smaller footprint and simpler codebase than heavyweight alternatives.

Message code is generated for both golang and C++ with JSON and binary serialization.

RPCs are implemented over pluggable transports (WebSocket and TCP currently supported). Client and server code is generated for both golang and C++.

Serialization uses bitpacked variable-length integer encoding with zigzag encoding for signed integers.

Installation:

go install github.com/kbirk/scg/cmd/scg-go@latest
go install github.com/kbirk/scg/cmd/scg-cpp@latest

Dependencies:

Golang:

Websockets: gorilla/websocket
TCP: net

C++:

JSON serialization: nlohmann/json
Websockets: websocketpp and asio
TCP: asio
SSL: openssl

Syntax:

Familiar protobuf-inspired syntax with practical simplifications and enhancements.

package pingpong;
service PingPong {
rpc Ping (PingRequest) returns (PongResponse);
}
message Ping {
int32 count = 0;
}
message Pong {
int32 count = 0;
}
message PingRequest {
Ping ping = 0;
}
message PongResponse {
Pong pong = 0;
}

Containers such as maps and lists use <T> syntax and can be nested:

message OtherStuff {
map<string, float64> map_field = 0;
list<uint64> list_field = 1;
map<int32, list<map<string, list<uint8>>>> what_have_i_done = 2;
}

Generating Go Code:

scg-go --input="./src/dir" --output="./output/dir" --base-package="github.com/yourname/repo"

Generating C++ Code:

scg-cpp --input="./src/dir" --output="./output/dir"

JSON Serialization

JSON serialization for C++ uses nlohmann/json.

#include"pingpong.h"
pingpong::PingRequest src;
src.ping.count = 42;
auto bs = req.toJSON();
pingpong::PingRequest dst;
auto err = dst.fromJSON(bs);
assert(!err && "deserialization failed");

JSON serialization for golang uses encoding/json.

src:= pingpong.PingRequest{
Ping: {
Count: 42,
}
}
bs:=src.ToJSON()
dst:= pingpong.PingRequest{}
err:=dst.FromJSON(bs)
iferr!=nil {
panic(err)
}

Binary Serialization

Binary serialization encodes the data in a portable payload using a single allocation for the destination buffer.

#include"pingpong.h"
pingpong::PingRequest src;
src.ping.count = 42;
auto bs = req.toBytes();
pingpong::PingRequest dst;
auto err = dst.fromBytes(bs);
assert(!err && "deserialization failed");

src:= pingpong.PingRequest{
Ping: {
Count: 42,
}
}
bs:=src.ToBytes()
dst:= pingpong.PingRequest{}
err:=dst.FromBytes(bs)
iferr!=nil {
panic(err)
}

RPCs

The RPC system supports pluggable transports through the Transport interface. Both WebSocket and TCP transports are provided.

Transport Interface

The transport layer is defined by three main interfaces:

// Connection represents a bidirectional communication channeltypeConnectioninterface {
Send(data []byte) errorReceive() ([]byte, error)
Close() error
}
// ServerTransport handles incoming connections for the servertypeServerTransportinterface {
Listen() errorAccept() (Connection, error)
Close() error
}
// ClientTransport handles outgoing connections for the clienttypeClientTransportinterface {
Connect() (Connection, error)
}

Go Server

Both client and server code is generated for golang. The server uses a transport-based configuration:

import (
"github.com/kbirk/scg/pkg/rpc""github.com/kbirk/scg/pkg/rpc/websocket""github.com/yourname/repo/pingpong"
)
// Create server with WebSocket transportserver:=rpc.NewServer(rpc.ServerConfig{
Transport: websocket.NewServerTransport(
websocket.ServerTransportConfig{
Port: 8080,
// Optional: for TLS// CertFile: "server.crt",// KeyFile: "server.key",
}),
ErrHandler: func(errerror) {
log.Printf("Server error: %v", err)
},
})
// Register your service implementationpingpong.RegisterPingPongServer(server, &pingpongServer{})
// Start the serverserver.ListenAndServe()

C++ Server

C++ server code is available for WebSocket and TCP transports:

#include"scg/server.h"
#include"scg/tcp/transport_server.h"
#include"pingpong/pingpong.h"
#include<thread>
#include<chrono>// Implement the service interfaceclassPingPongServerImpl : publicpingpong::PingPongServer {
public:
std::pair<pingpong::PongResponse, scg::error::Error> ping(
const scg::context::Context& ctx,
const pingpong::PingRequest& req) override {
pingpong::PongResponse response;
response.pong.count = req.ping.count + 1;
response.pong.payload = req.ping.payload;
returnstd::make_pair(response, nullptr);
}
};
intmain() {
// Configure TCP transport
scg::tcp::ServerTransportConfig transportConfig;
transportConfig.port = 8080;
// Configure server
scg::rpc::ServerConfig config;
config.transport = std::make_shared<scg::tcp::ServerTransportTCP>(transportConfig);
config.errorHandler = [](const scg::error::Error& err) {
printf("Server error: %s\n", err.message().c_str());
};
// Create serverauto server = std::make_shared<scg::rpc::Server>(config);
// Register service implementationauto impl = std::make_shared<PingPongServerImpl>();
pingpong::registerPingPongServer(server.get(), impl);
// Start server (starts background threads)
server->start();
// Keep main thread alivewhile (true) {
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return0;
}

For TLS connections, use scg::tcp::ServerTransportTCPTLS:

#include"scg/tcp/transport_server_tls.h"
scg::tcp::ServerTransportTLSConfig transportConfig;
transportConfig.port = 443;
transportConfig.certFile = "server.crt";
transportConfig.keyFile = "server.key";
config.transport = std::make_shared<scg::tcp::ServerTransportTCPTLS>(transportConfig);

For WebSocket connections, use scg::ws::ServerTransportWS or scg::ws::ServerTransportWSTLS:

#include"scg/ws/transport_server.h"
scg::tcp::ServerTransportConfig transportConfig;
transportConfig.port = 8080;
transportConfig.logging = logging;
config.transport = std::make_shared<scg::tcp::ServerTransportNoTLS>(transportConfig);

Go Client

The client also uses transport-based configuration:

import (
"context""github.com/kbirk/scg/pkg/rpc""github.com/kbirk/scg/pkg/rpc/websocket""github.com/yourname/repo/pingpong"
)
// Create client with WebSocket transportclient:=rpc.NewClient(rpc.ClientConfig{
Transport: websocket.NewClientTransport(
websocket.ClientTransportConfig{
Host: "localhost",
Port: 8080,
// Optional: for TLS// TLSConfig: &tls.Config{...},
}),
ErrHandler: func(errerror) {
log.Printf("Client error: %v", err)
},
})
c:=pingpong.NewPingPongClient(client)
resp, err:=c.Ping(context.Background(), &pingpong.PingRequest{
Ping: pingpong.Ping{
Count: 0,
},
})
iferr!=nil {
panic(err)
}
fmt.Println(resp.Pong.Count)

Note: A single rpc.Client can be used with multiple services. Service routing is handled automatically by the generated client code:

// Single client for multiple servicesclient:=rpc.NewClient(rpc.ClientConfig{
Transport: websocket.NewClientTransport(
websocket.ClientTransportConfig{
Host: "localhost",
Port: 8080,
}),
})
// Create clients for different services using the same transportserviceAClient:=servicea.NewServiceAClient(client)
serviceBClient:=serviceb.NewServiceBClient(client)
// Each service client automatically routes to the correct servicerespA, _:=serviceAClient.MethodA(ctx, &servicea.RequestA{})
respB, _:=serviceBClient.MethodB(ctx, &serviceb.RequestB{})

Middleware

Both client and server support middleware for cross-cutting concerns:

// Server middlewareserver.Middleware(func(ctx context.Context, next rpc.Handler) rpc.Handler {
returnfunc(ctx context.Context, reqinterface{}) (interface{}, error) {
// Pre-processinglog.Printf("Handling request...")
resp, err:=next(ctx, req)
// Post-processinglog.Printf("Request complete")
returnresp, err
}
})
// Client middlewareclient.Middleware(func(ctx context.Context, next rpc.Handler) rpc.Handler {
returnfunc(ctx context.Context, reqinterface{}) (interface{}, error) {
// Pre-processinglog.Printf("Sending request...")
resp, err:=next(ctx, req)
// Post-processinglog.Printf("Received response")
returnresp, err
}
})

C++ Client

C++ client example using WebSocket transport:

#include<scg/ws/transport_client.h>
#include"pingpong.h"
scg::ws::ClientTransportConfigNoTLS config;
config.host = "localhost";
config.port = 8080;
auto client = std::make_shared<scg::ws::ClientTransportNoTLS>(config);
pingpong::PingPongClient pingPongClient(client);
pingpong::PingRequest req;
req.ping.count = 0;
auto [res, err] = pingPongClient.ping(scg::context::background(), req);
if (err) {
std::cerr << "Request failed: " << err.message() << std::endl;
} else {
std::cout << res.pong.count << std::endl;
}

For TLS connections:

#include<scg/ws/transport_tls.h>
#include"pingpong.h"
scg::ws::ClientTransportConfigTLS config;
config.host = "localhost";
config.port = 443;
auto client = std::make_shared<scg::ws::ClientTransportTLS>(config);
pingpong::PingPongClient pingPongClient(client);
pingpong::PingRequest req;
req.ping.count = 0;
auto [res, err] = pingPongClient.ping(scg::context::background(), req);
if (err) {
std::cerr << "Request failed: " << err.message() << std::endl;
} else {
std::cout << res.pong.count << std::endl;
}

Client Interfaces and Fakes

Alongside each XxxClient, scg generates an abstract XxxApi interface that mirrors the client's call surface one-to-one, plus an XxxFake test double. This gives orchestration logic a mockable seam without hand-writing gateway/forwarding boilerplate — it is the symmetric counterpart to the server-side XxxServer interface.

Only unary rpcs are part of the interface; streaming rpcs stay on the concrete client.

C++ — XxxClient is final and derives from XxxApi; XxxFake exposes one std::function hook per rpc (onMethodName) and returns a default-constructed response with no error when a hook is unset:

// Production code depends on the interface, not the concrete client:voidrunCheckpoint(pingpong::PingPongApi& api) {
pingpong::PingRequest req;
auto [res, err] = api.ping(scg::context::background(), req);
// ...
}
// Wire the real client in production:
pingpong::PingPongClient client(transport);
runCheckpoint(client);
// Substitute a scripted fake in tests:
pingpong::PingPongFake fake;
fake.onPing = [](scg::context::Context& ctx, const pingpong::PingRequest& req) {
pingpong::PongResponse resp;
resp.pong.count = req.ping.count + 1;
returnstd::make_pair(resp, scg::error::Error(nullptr));
};
runCheckpoint(fake);

The fake is deliberately minimal — hooks only, no built-in call recording or thread-safety; tests layer those per their own needs.

Go — XxxApi is an interface satisfied by *XxxClient (enforced with a compile-time var _ XxxApi = (*XxxClient)(nil) assertion). Depend on XxxApi in code you want to fake; since the generated XxxServer interface has the same unary signatures, an in-process test can even hand a server implementation directly to client-side code.

typePingPongApiinterface {
Ping(ctx context.Context, req*PingRequest) (*PongResponse, error)
}

Streaming RPCs

Mark a request and/or response type with the stream qualifier to make an RPC a stream. A stream is opened once and both sides read from and write to it until either side closes it (à la gRPC). Unary RPCs are unaffected and share the same connection and wire format.

service Chat {
# bidirectional: client sends events, server pushes events, until either side closes
rpc Connect (stream ChatMessage) returns (stream ChatMessage);
}
message ChatMessage {
string text = 0;
int32 seq = 1;
}

Go — client:

chat:=pingpong.NewChatClient(client)
stream, err:=chat.Connect(context.Background()) // opens the stream; no message sent yet_=stream.Send(&pingpong.ChatMessage{Text: "hello"})
msg, err:=stream.Recv() // blocks; err == io.EOF on clean close_=stream.CloseSend() // half-close: done sending, still receiving

Go — server:

func (s*chatServer) Connect(stream*pingpong.Chat_ConnectStreamServer) error {
for {
msg, err:=stream.Recv() // io.EOF when the client half-closesiferr==io.EOF { returnnil }
iferr!=nil { returnerr }
_=stream.Send(&pingpong.ChatMessage{Text: "echo:"+msg.Text})
}
}

C++ — client (no exceptions; non-blocking drain suited to a game loop):

pingpong::ChatClient chat(client);
auto [stream, err] = chat.connect(scg::context::background());
err = stream->send(ev); // non-blocking// Per-frame, drain without blocking (recommended for a single-threaded game loop):for (;;) {
auto r = stream->tryRecv(); // never blocksif (r.state == scg::rpc::StreamRecvState::Empty) break; // nothing this frameif (r.state == scg::rpc::StreamRecvState::Closed) break; // r.error is nil on clean closehandle(r.message);
}
err = stream->closeSend();
// auto blocking = stream->recv(); // also available for non-frame-locked consumers

The C++ client receives on the transport's existing background I/O thread and delivers into a bounded per-stream queue; tryRecv()/send() never block, so all stream handling stays on the caller's thread. See streaming.md for the full threading-model rationale and wire protocol.

SCG C++ Serialization Macros

The C++ include/scg/macro.h provides some macros for building serialization overrides for types that are not generated with scg.

There are four macros:

SCG_SERIALIZABLE_PUBLIC: declare public fields as serializable.
SCG_SERIALIZABLE_PRIVATE: declare public and private fields as serializable.
SCG_SERIALIZABLE_DERIVED_PUBLIC: declare a type as derived from another, include any base class serialization logic, along with new public fields.
SCG_SERIALIZABLE_DERIVED_PRIVATE: declare a type as derived from another, and include any base class serialization logic, along with new public and private fields.

// Declare public fields as serializable, note the macro is called _outside_ the struct.structMyStruct {
uint32_t a = 0;
float64_t b = 0;
std::vector<std::string> c;
};
SCG_SERIALIZABLE_PUBLIC(MyStruct, a, b, c);
// Declare declare private fields as serializable, note the macro is called _inside_ the class.classMyClass {
public:MyClass() = default;
MyClass(uint32_t a, float64_t b) : a_(a), b_(b)
{
}
SCG_SERIALIZABLE_PRIVATE(MyClass, a_, b_);
private:uint32_t a_ = 0;
uint64_t b_ = 0;
};
// Declare the base class to derive serialization logic from, note the macro is called _outside_ the struct.structDerivedStruct : MyStruct{
bool d = false;
};
SCG_SERIALIZABLE_DERIVED_PUBLIC(DerivedStruct, MyStruct, d);
// Declare the base class to derive serialization logic from, note the macro is called _inside_ the class.classMyDerivedClass : publicMyClass {
public:MyDerivedClass() = default;
MyDerivedClass(uint32_t a, float64_t b, bool c) : MyClass(a, b), c_(c)
{
}
SCG_SERIALIZABLE_DERIVED_PRIVATE(MyDerivedClass, MyClass, c_);
private:bool c_ = false;
};

Individual serialization overrides can be provided using ADL as follows, for example, here is how to extend it to serialize glm types:

namespaceglm {
template <typename WriterType>
inlinevoidserialize(WriterType& writer, const glm::vec2& value)
{
writer.write(value.x);
writer.write(value.y);
}
template <typename ReaderType>
inline scg::error::Error deserialize(glm::vec2& value, ReaderType& reader)
{
auto err = reader.read(value.x);
if (err) {
return err;
}
return reader.read(value.y);
}
}

Testing:

Generate test files:

./gen-test-code.sh

Generate SSL keys for test server:

./gen-ssl-keys.sh

Download and vendor the third party header files:

cd ./third_party && ./install-deps.sh &&cd ..

Run the tests:

./run-serialize-tests.sh
./run-tcp-tests.sh
./run-websocket-tests.sh

TODO:

Opentracing hooks and context serialization

Navigation Menu

Repository files navigation

scg - Simple Code Generator

Installation:

Dependencies:

Golang:

C++:

Syntax:

Generating Go Code:

Generating C++ Code:

JSON Serialization

Binary Serialization

RPCs

Transport Interface

Go Server

C++ Server

Go Client

Middleware

C++ Client

Client Interfaces and Fakes

Streaming RPCs

SCG C++ Serialization Macros

Testing:

TODO:

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
benchmark		benchmark
cmd		cmd
include/scg		include/scg
internal		internal
pkg		pkg
test		test
third_party		third_party
.editorconfig		.editorconfig
.gitignore		.gitignore
README.md		README.md
gen-benchmark-code.sh		gen-benchmark-code.sh
gen-ssl-keys.sh		gen-ssl-keys.sh
gen-test-code.sh		gen-test-code.sh
go.mod		go.mod
go.sum		go.sum
run-benchmarks.sh		run-benchmarks.sh
run-serialize-tests.sh		run-serialize-tests.sh
run-tcp-tests.sh		run-tcp-tests.sh
run-websocket-tests.sh		run-websocket-tests.sh

Latest commit

History

Folders and files

Repository files navigation

scg - Simple Code Generator

Installation:

Dependencies:

Golang:

C++:

Syntax:

Generating Go Code:

Generating C++ Code:

JSON Serialization

Binary Serialization

RPCs

Transport Interface

Go Server

C++ Server

Go Client

Middleware

C++ Client

Client Interfaces and Fakes

Streaming RPCs

SCG C++ Serialization Macros

Testing:

TODO:

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages