Skip to content
BranchesTags

Repository files navigation

MsgPacketizer

msgpack based serializer / deserializer + packetize for Arduino, ROS, and more

NOTE (>= v0.5.0) : DEPENDENT LIBRARIES REMOVED

If you have already installed this library, please follow:

  • Cloned from GitHub (manually): Please install dependent libraries manually
  • Installed from library manager: re-install this library from library manager
    • Dependent libraries will be installed automatically

Feature

  • one-line [serialize/deserialize] or [publish/subscribe] + packetize + robust [send/receive]
  • [serializer/deserializer] supports almost all standard type of C++ same as msgpack-c
  • support custom class [serialization/deserialization]
  • support one-line manual [serialization/deserialization] to work with any communication interface
  • support working with ArduinoJSON
  • [serializer/deserializer] based on MsgPack
  • packetize based on Packetizer
  • working also in ROS with serial and serial-ros2

Packet Protocol

indexmsgpackcrc8
1 byteN bytes1 byte
  • 1 byte index (packet index can be used to identify packet)
  • N byte serialized msgpack data
  • 1 byte crc8 (for received data check)
  • these bytes are encoded to COBS encoding based on Packetizer

Usage

Direct Data Receive + Data Publishing

#include<MsgPacketizer.h>// input to msgpackint i;
float f;
MsgPack::str_t s; // std::string or String
MsgPack::arr_t<int> v; // std::vector or arx::stdx::vector
MsgPack::map_t<String, float> m; // std::map or arx::stdx::mapuint8_t recv_index = 0x12;
uint8_t send_index = 0x34;
voidsetup() {
Serial.begin(115200);
// update received data directlyMsgPacketizer::subscribe(Serial, recv_index, i, f, s, v, m);
// publish varibales periodically (default 30[times/sec])MsgPacketizer::publish(Serial, send_index, i, f, s, v, m);
}
voidloop() {
// must be called to trigger callback and publish dataMsgPacketizer::update();
}

Callback with Received Objects + One-Line Send

#include<MsgPacketizer.h>uint8_t recv_index = 0x12;
uint8_t send_index = 0x34;
voidsetup() {
Serial.begin(115200);
// handle received data with lambda// which has incoming argument types/dataMsgPacketizer::subscribe(Serial, recv_index,
[](int i, float f, MsgPack::str_t s, MsgPack::arr_t<int> v, MsgPack::map_t<String, float> m)
{
// send received data back in one-lineMsgPacketizer::send(Serial, send_index, i, f, s, v, m);
}
);
}
voidloop() {
// must be called to trigger callbackMsgPacketizer::parse();
}

Nested Data with Custom Class

To serialize / deserialize nested data, defining custom class is recommended. For example, to make {"k1": v, "k2":[i, f, s]}:

structArrayData {
int i; float f; MsgPack::str_t s;
MSGPACK_DEFINE(i, f, s); // [i, f, s]
};
structNestedData {
MsgPack::str_t k1, k2; int v;
ArrayData a;
MSGPACK_DEFINE_MAP(k1, v, k2, a); // {"k1": v, "k2":[i, f, s]}
};

and you can serialize / deserialize your class completely same as other types.

NestedData n;
MsgPacketizer::publish(Serial, send_index, n);
MsgPacketizer::subscribe(Serial, recv_index, n);

Please see examples and MsgPack for more detail.

Manual Encode / Decode with Any Communication I/F

You can just encode / decode data manually to use it with any communication interface. Please note:

  • only one unsupoprted interface (serial, udp, tcp, etc.) is available for manual subscription because MsgPacketizer cannot indetify which data is from which device
  • publish is not available for unsupported data stream
#include<MsgPacketizer.h>constuint8_t recv_index = 0x12;
constuint8_t send_index = 0x34;
voidsetup() {
Serial.begin(115200);
delay(2000);
// subscribe the data for manual operationMsgPacketizer::subscribe(recv_index,
[&](constint i, constfloat f, const String& s) {
// just encode your data manually and get binary packet from MsgPacketizerconstauto& packet = MsgPacketizer::encode(send_index, i, f, s);
// send the packet data with your interface
Serial.write(packet.data.data(), packet.data.size());
}
);
}
voidloop() {
// you should feed the received data manually to MsgPacketizerconstsize_t size = Serial.available();
if (size) {
uint8_t* data = newuint8_t[size];
// you can get binary data from any communication interface
Serial.readBytes(data, size);
// feed your binary data to MsgPacketizer manually// if data has successfully received and decoded, callback will be calledMsgPacketizer::feed(data, size);
delete[] data;
}
}

UDP and TCP Support

TCP Support

  • start client first
  • everything else can be used in the same way
#include<MsgPacketizer.h>
#include<WiFi.h>constuint8_t index = 0x12;
int i; float f; String s;
// WiFi stuff
WiFiClient client;
constchar* host = "192.168.0.10";
constint port = 54321;
voidsetup() {
WiFi.begin("your-ssid", "your-password");
// start client
client.connect(host, port);
// everything else can be used in the same wayMsgPacketizer::publish(client, index, i, f, s)->setFrameRate(1);
MsgPacketizer::subscribe(client, index,
[&](constint i, constfloat f, const String& s) {
// do something with received data
}
);
}
voidloop() {
// do something with your variables i, f, s// must be called to trigger callback and publish dataMsgPacketizer::update();
}

UDP Support

  • start client first
  • set ip and port when you publish or send messages
  • everything else can be used in the same way
#include<MsgPacketizer.h>
#include<WiFi.h>constuint8_t index = 0x12;
int i; float f; String s;
// WiFi stuff
WiFiUDP client;
constchar* host = "192.168.0.10";
constint port = 54321;
voidsetup() {
WiFi.begin("your-ssid", "your-password");
// start client first
client.begin(port);
// set host and port when publishing or sending messagesMsgPacketizer::publish(client, host, port, index, i, f, s)->setFrameRate(1);
MsgPacketizer::subscribe(client, index,
[&](constint i, constfloat f, const String& s) {
// do something with received data
}
);
}
voidloop() {
// do something with your variables i, f, s// must be called to trigger callback and publish dataMsgPacketizer::update();
}

ArduinoJson Support

  • supports only version > 6.x
  • supports StaticJsonDocument<N> and DynamicJsonDocument
  • supports only send, decode, and subscribe with callbacks
    • you can use publish and subscribe directly by reference, but not reccomended
    • please see this official document for the detail
#include<ArduinoJson.h>// include before MsgPacketizer.h
#include<MsgPacketizer.h>
#include<WiFi.h>constuint8_t msg_index = 0x12;
constchar* host = "192.168.0.10";
constint port = 54321;
WiFiUDP client;
voidsetup() {
WiFi.begin("your-ssid", "your-password");
client.begin(port);
MsgPacketizer::subscribe(client, msg_index,
[&](const StaticJsonDocument<200>& doc) {
// do something with your json
}
);
}
voidloop() {
StaticJsonDocument<200> doc;
// make your json hereMsgPacketizer::send(client, host, port, msg_index, doc);
delay(1000);
// must be called to trigger callback and publish dataMsgPacketizer::update();
}

Buffer Size Adjustment when Subscribing DynamicJsonDocument

Currently we cannot calculate the json size from msgpack format before deserialization. Please adjust buffer size by defining following macro before including MsgPacketizer.h. Default is size_of_msgpack_bytes * 3.

#defineMSGPACKETIZER_ARDUINOJSON_DESERIALIZE_BUFFER_SCALE3// default
#include<MsgPacketizer.h>

APIs

Subscriber

namespaceMsgPacketizer {
// ----- for unsupported communication interface with manual operation -----// bind variables directly to specified index packettemplate <typename... Args>
inlinevoidsubscribe_manual(constuint8_t index, Args&&... args);
// bind variables directly to specified index packet with array formattemplate <typename... Args>
inlinevoidsubscribe_manual_arr(constuint8_t index, Args&&... args);
// bind variables directly to specified index packet with map formattemplate <typename... Args>
inlinevoidsubscribe_manual_map(constuint8_t index, Args&&... args);
// bind callback to specified index packettemplate <typename F>
inlinevoidsubscribe_manual(constuint8_t index, F&& callback);
// bind callback which is always called regardless of indextemplate <typename F>
inlinevoidsubscribe_manual(F&& callback);
// unsubscribeinlinevoidunsubscribe_manual(constuint8_t index);
// feed packet manually: must be called to manual decodinginlinevoidfeed(constuint8_t* data, constsize_t size);
// ----- for supported communication interface (Arduino, oF, ROS) -----template <typename S, typename... Args>
inlinevoidsubscribe(S& stream, constuint8_t index, Args&&... args);
template <typename S, typename... Args>
inlinevoidsubscribe_arr(S& stream, constuint8_t index, Args&&... args);
template <typename S, typename... Args>
inlinevoidsubscribe_map(S& stream, constuint8_t index, Args&&... args);
template <typename S, typename F>
inlinevoidsubscribe(S& stream, constuint8_t index, F&& callback);
template <typename S, typename F>
inlinevoidsubscribe(S& stream, F&& callback);
template <typename S>
inlinevoidunsubscribe(const S& stream, constuint8_t index);
template <typename S>
inlinevoidunsubscribe(const S& stream);
template <typename S>
// get UnpackerRef = std::shared_ptr<MsgPack::Unpacker> of stream and handle it manuallyinline UnpackerRef getUnpackerRef(const S& stream);
// get map of unpackers and handle it manuallyinline UnpackerMap& getUnpackerMap();
// must be called to receive packetsinlinevoidparse(bool b_exec_cb = true);
inlinevoidupdate(bool b_exec_cb = true);
}

Publisher

namespaceMsgPacketizer {
// ----- for unsupported communication interface with manual operation -----// encode arguments directly with variable typestemplate <typename... Args>
inlineconst Packetizer::Packet& encode(constuint8_t index, Args&&... args);
// encode binary datainlineconst Packetizer::Packet& encode(constuint8_t index, constuint8_t* data, constuint8_t size);
// encode manually packed datainlineconst Packetizer::Packet& encode(constuint8_t index);
// encode args as array formattemplate <typename... Args>
inlineconst Packetizer::Packet& encode_arr(constuint8_t index, Args&&... args);
// encode args as map formattemplate <typename... Args>
inlineconst Packetizer::Packet& encode_map(constuint8_t index, Args&&... args);
// ----- for supported communication interface (Arduino, oF, ROS) -----// send arguments dilectly with variable typestemplate <typename S, typename... Args>
inlinevoidsend(S& stream, constuint8_t index, Args&&... args);
// send binary datatemplate <typename S>
inlinevoidsend(S& stream, constuint8_t index, constuint8_t* data, constuint8_t size);
// send manually packed datatemplate <typename S>
inlinevoidsend(S& stream, constuint8_t index);
// send args as array formattemplate <typename S, typename... Args>
inlinevoidsend_arr(S& stream, constuint8_t index, Args&&... args);
// send args as map formattemplate <typename S, typename... Args>
inlinevoidsend_map(S& stream, constuint8_t index, Args&&... args);
// UDP version of sendtemplate <typename... Args>
inlinevoidsend(UDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
inlinevoidsend(UDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, constuint8_t* data, constuint8_t size);
inlinevoidsend(UDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index);
template <typename... Args>
inlinevoidsend_arr(UDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
template <typename... Args>
inlinevoidsend_map(UDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
// publish arguments periodicallytemplate <typename S, typename... Args>
inline PublishElementRef publish(const S& stream, constuint8_t index, Args&&... args);
// publish arguments periodically as array formattemplate <typename S, typename... Args>
inline PublishElementRef publish_arr(const S& stream, constuint8_t index, Args&&... args);
// publish arguments periodically as map formattemplate <typename S, typename... Args>
inline PublishElementRef publish_map(const S& stream, constuint8_t index, Args&&... args);
// unpublishtemplate <typename S>
inlinevoidunpublish(const S& stream, constuint8_t index);
// get registerd publish element classtemplate <typename S>
inline PublishElementRef getPublishElementRef(const S& stream, constuint8_t index);
// UDP version of publishtemplate <typename... Args>
inline PublishElementRef publish(constUDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
template <typename... Args>
inline PublishElementRef publish_arr(constUDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
template <typename... Args>
inline PublishElementRef publish_map(constUDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index, Args&&... args);
inlinevoidunpublish(constUDP& stream, conststr_t& ip, constuint16_t port, constuint8_t index);
inline PublishElementRef getPublishElementRef(constUDP& stream, constuint8_t index);
// must be called to publish datainlinevoidpost();
// get MsgPack::Packer and handle it manuallyinlineconst MsgPack::Packer& getPacker();
}

Other Options

#defineMSGPACKETIZER_DEBUGLOG_ENABLE

For NO-STL Boards

For following archtectures, several storage size for packets are limited.

  • AVR
  • megaAVR

Memory Management (only for NO-STL Boards)

As mentioned above, for such boards like Arduino Uno, the storage sizes are limited. And of course you can manage them by defining following macros. But these default values are optimized for such boards, please be careful not to excess your boards storage/memory. These macros have no effect for STL enabled boards.

MsgPacketizer

// max publishing elemtnt size in one destination
#defineMSGPACKETIZER_MAX_PUBLISH_ELEMENT_SIZE5// max destinations to publish
#defineMSGPACKETIZER_MAX_PUBLISH_DESTINATION_SIZE1

MsgPack

// msgpack serialized binary size
#defineMSGPACK_MAX_PACKET_BYTE_SIZE96// max size of MsgPack::arr_t
#defineMSGPACK_MAX_ARRAY_SIZE3// max size of MsgPack::map_t
#defineMSGPACK_MAX_MAP_SIZE3// msgpack objects size in one packet
#defineMSGPACK_MAX_OBJECT_SIZE16

Packetizer

// max number of decoded packet queues
#definePACKETIZER_MAX_PACKET_QUEUE_SIZE1// max data bytes in packet
#definePACKETIZER_MAX_PACKET_BINARY_SIZE96// max number of callback for one stream
#definePACKETIZER_MAX_CALLBACK_QUEUE_SIZE3// max number of streams
#definePACKETIZER_MAX_STREAM_MAP_SIZE1

Dependent Libraries

License

MIT

About

msgpack based serializer / deserializer + packetize for Arduino, ROS, and more

Topics

arduinocppmessagepackopenframeworks

Resources

Readme
MIT license
Activity

Stars

88 stars

Watchers

7 watching

Forks

11 forks
Report repository

Releases

Packages

Used by

Contributors

Languages