VRPC

VRPC (versioned rpc) is a tool to generate types and service definitions that are versioned e.g. Order_V0, Order_V2 or Post_V1. This is intended to help manage forwards and backwards compatibility and make it easier to evolve software as external interfaces change.

Conceptually you can think of it as working like a version control system, but for the types in your software rather than just files. Changes to your types are stored as a series of change actions e.g. "add a field", "add a service version" rather than as a single static definition file. Each action potentially produces a new type or service version to be generated. Underneath it uses something like a merkle tree, where every change to a type is hashed to ensure that past definitions can't be changed.

Ultimately the goal would be to version all the external interfaces of a service, including things like files and the database.

Status

This is still extremely experimental and there's still a lot of work to do to make this truly useful. For now I've moved on to other things and it might be awhile before I continue work on this.

Example of the concept.

NB: Not all of this is working yet.

The current definitions are stored in a backend file that is used to hold the current state of the definition and the log of changes generated from previous versions of it.

To define a service, first you need to define some types to use for the service:

[
{
"name": "OrderQuery",
"description": "Query orders from the database",
"fields": [
{
"name": "username",
"description": "username of the person ordering",
"_type": "string",
"changeLog": "Front end needs to be able to look up a persons orders"
},
{
"name": "total",
"description": "get orders with a specific value",
"_type": "float",
"optional": true,
"changeLog": "Need this for looking up specific orders by value on the admin page"
}
]
},
{
"name": "OrderQueryResult",
"description": "Result of an OrderQuery",
"fields": [
{
"name": "id",
"description": "Id of the order",
"_type": "integer",
"changeLog": "Useful for display"
},
{
"name": "total",
"description": "Total value of the order",
"_type": "float",
"changeLog": "User needs to see value of the order."
}
]
}
]

Run the commit command to add the definition changes to the backend file:

vrpc update backend.json vrpc_definitions.json

For now this has to be done as a step prior to adding the service definitions, to make sure that the types have been added to the backend.

Add the service definition to the bottom of vrpc_definitions.json using the types defined above:

[
****Snip type definitions****
{
"name": "OrderQueryService",
"description": "Service for querying Orders",
"versions": [
{
"_from": {
"name": "OrderQuery",
"version": 0
},
"to": {
"name": "OrderQueryResult",
"version": 0
},
"changeLog": "Necessary to support the order querying feature"
}
]
}
]

Run the commit command again to add the service changes to the backend:

vrpc update backend.json vrpc_definitions.json

Generate the stubs for the types and the service:

vrpc generate backend.json src/generated

Define your service by passing in an express app to the service definition function along with the definition of the service:

import{OrderQuery,OrderQueryResult,OrderQuery_V0,OrderQueryResult_V0}from'./generated/types';import{OrderQueryServiceExpress}from'./generated/services';importexpressfrom'express';constapp=express();app.use(express.json());OrderQueryServiceExpress(app,{V0: {"OrderQueryResult_V0": async(input: OrderQuery_V0)Promise<OrderQueryResult_V0>=>{letresult;if(input.total){result=query("Select * from orders WHERE username = ? AND total = ?",input.username,input.total);}else{result=query("Select * from orders WHERE username = ?",input.username);}returnnewOrderQueryResult_V0(result.id,result.total);}}});

To query the service you can use the generated client:

import*asclientfrom'./generated/client';asyncfunctionqueryOrders(username: string){returnawaitclient.OrderQueryService.V0.OrderQuery_V0(username);}

TODO

• [] Test CLI.
• [] Package and put on NPM.
• [] Options to control what services and types are generated, rather than just generating them all.
• [] Options to control generating a union type for a group of service inputs.
• [] Handle forward compatibility in client.
• [] Simplify definition of services by spreading input, rather than passing message object.
• [] Service and type deprecation and corresponding changes to generation logic.
• [] Force changelog when updating types and services.
• [] New types:
  • [] arrays
  • [] enums
  • [] intersections
  • [] unions
• [] Add HTTP methods for services, possibly by adding fields to types and services for controlling the generation process.
• [] Fix HTTP codes in generated services, e.g. 400 for misformatted requests, or 410 for deleted versions.
• [] Fix types in generated typescript services.
• [] Remove duplication in code related to handling services and types.
• [] Define services at the same time as the types they depend on.
• [] Generate other languages e.g. protobufs, json schema, python etc.
• [] Metalog format to define generatables and structures as part of log.
• [] Database access framework e.g. for Postgres, Mongo etc.

Authors

• Paul Johnson - pj

License

This project is licensed under the MIT License - see the LICENSE.md file for details