Base Driver

Overview

base-driver is a framework for creating drivers for complex interfaces or data transformation tasks. This guide will walk you through the framework concepts, components, and their purpose.

Source

Github Repository

Concepts

Platform

Platform is not part of the base-driver package, but is rather implemented as part of individual drivers to support communication with the native data source.

• Methods inside Platform implementations faciliate communication with the data source via a local Machine instance reference.
• Platform methods are invoked by a Collector instance.
• No data source transformation logic should occur inside the Platform implementation.
• Veneer classes are responsible for transformation logic.

Example: factoryio-driver

Retrieve all tags from FactoryIO instance.

publicasyncTask<dynamic>GetTagsAsync(){varrequest=newRestRequest("tags",DataFormat.Json);varresponse=await_machine.Client.ExecuteGetAsync(request);returnJArray.Parse(response.Content);}

Write tags to FactoryIO instance.

publicasyncTask<dynamic>WriteTagsByNameAsync(stringtag_array){varrequest=newRestRequest("tag/values/by-name",Method.PUT,DataFormat.Json);request.AddParameter("application/json",tag_array,ParameterType.RequestBody);varresponse=await_machine.Client.ExecuteAsync(request);returnJArray.Parse(response.Content);}

Example: opcxmlda-driver

Retrieve multiple items from an OPC XML-DA server.

publicdynamicReadMultipleTags(List<dynamic>descriptors){DAVtqResult[]results=null;NativeDispatchReturnndr=nativeDispatch(()=>{results=_machine.Client.ReadMultipleItems(newServerDescriptor{UrlString=_machine.OpcxmldaEndpoint.URI},Array.ConvertAll(descriptors.ToArray(),newConverter<dynamic,DAItemDescriptor>(item =>newDAItemDescriptor((string)((Dictionary<object,object>.KeyCollection)item.Keys).ElementAt(0)))));returntrue;});varnr=new{invocationMs=ndr.ElapsedMilliseconds,request=new{read_multiple_tags=new{descriptors}},response=new{read_multiple_tags=new{results}}};_logger.Trace($"[{_machine.Id}] Platform invocation result:\n{JObject.FromObject(nr).ToString()}");returnnr;}

Example: fanuc-driver

Retrieve machine identifier from Fanuc controller.

publicdynamicCNCId(){uint[]cncid=newuint[4];NativeDispatchReturnndr=nativeDispatch(()=>{return(Focas.focas_ret)Focas.cnc_rdcncid(_handle,cncid);});varnr=new{method="cnc_rdcncid",invocationMs=ndr.ElapsedMilliseconds,doc="https://ladder99.github.io/fanuc-driver/focas/SpecE/Misc/cnc_rdcncid",success=ndr.RC==Focas.EW_OK,rc=ndr.RC,request=new{cnc_rdcncid=new{}},response=new{cnc_rdcncid=new{cncid}}};_logger.Trace($"[{_machine.Id}] Platform invocation result:\n{JObject.FromObject(nr).ToString()}");returnnr;}

Machine

A Machine instance includes:

• native connectivity information and setup (from config.yaml)
• data source access (Platform)
• data output post-processor (Handler)
• data collection strategy (Collector)
• data transformations (Veneer)

Drivers extend the Machine class to provide functionality unique to the data source.

A concrete Machine type, including assembly name, is referenced in the configuration at path machines[id].type. Example: l99.driver.fanuc.FanucMachine, fanuc.

Example: factoryio-driver

Communication protocol configuration and HTTP client are managed by the Machine instance.

publicFactoryioRemoteMachine(Machines machines,bool enabled,string id,object config):base(machines,enabled,id,config){dynamiccfg=(dynamic)config;this["cfg"]=cfg;this["platform"]=newPlatform(this);_factoryioRemoteEndpoint=newFactoryioRemoteEndpoint(cfg.type["net_uri"],(short)cfg.type["net_timeout_s"]);_client=newRestClient($"{cfg.type["net_uri"]}/api");_client.Timeout=cfg.type["net_timeout_s"]*1000;}

Example: opcxmlda-driver

Communication protocol configuration and QuickOPC DA client are managed by the Machine instance.

publicOpcxmldaMachine(Machines machines,bool enabled,string id,object config):base(machines,enabled,id,config){dynamiccfg=(dynamic)config;this["cfg"]=cfg;this["data"]=cfg.type["data"];this["platform"]=newPlatform(this);_opcxmldaEndpoint=newOpcxmldaEndpoint(cfg.type["net_uri"],(short)cfg.type["net_timeout_s"]);_client=newEasyDAClient();}

Example: fanuc-driver

Communciation protocol configuration is managed by the Machine instance.

publicFanucMachine(Machines machines,bool enabled,string id,object config):base(machines,enabled,id,config){dynamiccfg=(dynamic)config;_focasEndpoint=newFocasEndpoint(cfg.type["net_ip"],(ushort)cfg.type["net_port"],(short)cfg.type["net_timeout_s"]);this["platform"]=newPlatform(this);}

Veneer

veneer : to overlay or plate (a surface, as of a common sort of wood) with a thin layer of finer wood for outer finish or decoration

peel : to break away from a group or formation —often used with off

Collected data requires processing before it can be relayed to another system. Veneer instances are responsible for:

• simple or complex source data transformations
• making source data human readable
• creating richer (more valuable to the consumer) data from multiple data points
• tracking data changes
• creating data hierarchies

Veneer instances are "applied" over native data points and "peeled" during the data collection cycle to reveal modified data structures.

Veneers can be applied/peeled as a whole. Veneers can be sliced and applied/peeled across logical boundaries (e.g. path, axis, spindle). Atomic values should be used for slicing veneers. Sliced veneers must be marked before peeling in order to convey the logical hierarchy of the observation to downstream systems.

Example: opcxmlda-driver | intermediate format

Transformation into an intermediate format.

protectedoverrideasyncTask<dynamic>AnyAsync(dynamicinput,paramsdynamic?[]additionalInputs){varcurrent_value=new{name=additionalInputs[0],type=input.Vtq?.ValueType.Name,value=input.Vtq?.Value,good=input.Vtq?.Quality.IsGood};awaitonDataArrivedAsync(input,current_value);if(current_value.IsDifferentString((object)lastChangedValue)){awaitonDataChangedAsync(input,current_value);}returnnew{veneer=this};}

Example: fanuc-driver | human readable

Transformation into an human readable format.

protectedoverrideasyncTask<dynamic>AnyAsync(dynamicinput,paramsdynamic?[]additionalInputs){if(input.success){varcurrent_value=new{cncid=string.Join("-",((uint[])input.response.cnc_rdcncid.cncid).Select(x =>x.ToString("X")).ToArray())};awaitonDataArrivedAsync(input,current_value);if(!current_value.Equals(lastChangedValue)){awaitonDataChangedAsync(input,current_value);}}else{awaitonErrorAsync(input);}returnnew{veneer=this};}

Example: fanuc-driver | enriched

Tracking executed G-code blocks.

protectedoverrideasyncTask<dynamic>AnyAsync(dynamicinput,paramsdynamic?[]additionalInputs){if(input.success&&additionalInputs[0].success&&additionalInputs[1].success){_blocks.Add(input.response.cnc_rdblkcount.prog_bc,additionalInputs[0].response.cnc_rdactpt.blk_no,additionalInputs[1].response.cnc_rdexecprog.data);varcurrent_value=new{blocks=_blocks.ExecutedBlocks};awaitonDataArrivedAsync(input,current_value);varlast_keys=((List<gcode.Block>)lastChangedValue.blocks).Select(x =>x.BlockNumber);varcurrent_keys=((List<gcode.Block>)current_value.blocks).Select(x =>x.BlockNumber);if(last_keys.Except(current_keys).Count()+current_keys.Except(last_keys).Count()>0){awaitonDataChangedAsync(input,current_value);}}else{awaitonErrorAsync(input);}returnnew{veneer=this};}

Example: fanuc-driver | performance tracking

Tracking driver performance.

protectedoverrideasyncTask<dynamic>AnyAsync(dynamicinput,paramsdynamic?[]additionalInputs){varmax=((List<dynamic>)input.focas_invocations).MaxBy(o =>o.invocationMs).First();varmin=((List<dynamic>)input.focas_invocations).MinBy(o =>o.invocationMs).First();varavg=(int)((List<dynamic>)input.focas_invocations).Average(o =>(int)o.invocationMs);varsum=((List<dynamic>)input.focas_invocations).Sum(o =>(int)o.invocationMs);varfailedMethods=((List<dynamic>)input.focas_invocations).Where(o =>o.rc!=0).Select(o =>new{o.method,o.rc});varcurrent_value=new{input.sweepMs,invocation=new{count=input.focas_invocations.Count,maxMethod=max.method,maxMs=max.invocationMs,minMs=min.invocationMs,avgMs=avg,sumMs=sum,failedMethods}};;awaitonDataArrivedAsync(input,current_value);returnnew{veneer=this};}

Example: fanuc-driver | boundary marker

Example of a generated observation marker for spindle 'S' on execution path '1'.

"marker": [
{
"path_no": 1
},
{
"name": "S",
"suff1": "",
"suff2": ""
}
]

Collector

Collector is a data collection strategy and an interface to the data source. The Collector is responsible for:

• establishing and tearing down connection with the data source via a Platform implementation
• applying Veneer over collected data
• invoking Veneer over collected data
• tracking data source connectivity failures

A concrete Collector type, including assembly name, is referenced in the configuration at path machines[id].strategy. Example: l99.driver.fanuc.BlockTracker, fanuc.

Example: opcxmlda-driver

Initialization, "applying veneers".

publicoverrideasyncTask<dynamic?>InitializeAsync(){try{foreach(dynamicdescriptorinmachine["data"]){machine.ApplyVeneer(typeof(opcxmlda.veneers.Tag),getDataKey(descriptor));}machine.VeneersApplied=true;}catch(Exceptionex){logger.Error(ex,$"[{machine.Id}] Collector initialization failed.");}returnnull;}

Collection cycle, "peeling veneers".

publicoverrideasyncTask<dynamic?>CollectAsync(){try{dynamictags=awaitmachine["platform"].ReadMultipleTagsAsync(machine["data"]);for(inti=0;i<tags.response.read_multiple_tags.results.Length;i++){vartag=tags.response.read_multiple_tags.results[i];stringdescriptor=getDataKey(i);awaitmachine.PeelVeneerAsync(descriptor,tag,descriptor);}LastSuccess=true;}catch(Exceptionex){logger.Error(ex,$"[{machine.Id}] Collector sweep failed.");}returnnull;}

Example: fanuc-driver

Initialization, "applying veneers".

publicoverrideasyncTask<dynamic?>InitializeAsync(){try{while(!machine.VeneersApplied){dynamicconnect=awaitmachine["platform"].ConnectAsync();if(connect.success){machine.ApplyVeneer(typeof(fanuc.veneers.Connect),"connect");machine.ApplyVeneer(typeof(fanuc.veneers.CNCId),"cnc_id");machine.ApplyVeneer(typeof(fanuc.veneers.RdParamLData),"power_on_time");machine.ApplyVeneer(typeof(fanuc.veneers.SysInfo),"sys_info");machine.ApplyVeneer(typeof(fanuc.veneers.GetPath),"get_path");dynamicdisconnect=awaitmachine["platform"].DisconnectAsync();machine.VeneersApplied=true;}else{awaitTask.Delay(sweepMs);}}}catch(Exceptionex){logger.Error(ex,$"[{machine.Id}] Collector initialization failed.");}returnnull;}

Collection cycle, "peeling veneers".

publicoverrideasyncTask<dynamic?>CollectAsync(){try{dynamicconnect=awaitmachine["platform"].ConnectAsync();awaitmachine.PeelVeneerAsync("connect",connect);if(connect.success){dynamiccncid=awaitmachine["platform"].CNCIdAsync();awaitmachine.PeelVeneerAsync("cnc_id",cncid);dynamicpoweron=awaitmachine["platform"].RdParamDoubleWordNoAxisAsync(6750);awaitmachine.PeelVeneerAsync("power_on_time",poweron);dynamicinfo=awaitmachine["platform"].SysInfoAsync();awaitmachine.PeelVeneerAsync("sys_info",info);dynamicpaths=awaitmachine["platform"].GetPathAsync();awaitmachine.PeelVeneerAsync("get_path",paths);dynamicdisconnect=awaitmachine["platform"].DisconnectAsync();}LastSuccess=connect.success;}catch(Exceptionex){logger.Error(ex,$"[{machine.Id}] Collector sweep failed.");}returnnull;}

Handler

Handler is an observation post-processor and interface to target systems. Data gathered via a Collector is transformed through a Veneer and acted upon the processing stages of:

• data arrival
• data change
• data source errors
• data collection cycle completion

A concrete Handler type, including assembly name, is referenced in the configuration at path machines[id].handler. Example: l99.driver.fanuc.handlers.SparkplugB, fanuc.

Example: opcxmlda-driver

The handler prepares changed data into Splunk metric format.

publicoverrideasyncTask<dynamic?>OnDataChangeAsync(Veneersveneers,Veneerveneer,dynamic?beforeChange){varpayload=new{time=newDateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds(),@event="metric",host=veneers.Machine.Id,fields=new{metric_name=veneer.LastArrivedValue.name,_value=veneer.LastArrivedValue.value,type=veneer.LastArrivedValue.type,good=veneer.LastArrivedValue.good}};returnpayload;}

The Splunk metric payload is then published to an MQTT broker. Alternatively, an HTTP request to the Splunk HEC endpoint could be executed.

protectedoverrideasyncTaskafterDataChangeAsync(Veneersveneers,Veneerveneer,dynamic?onChange){if(onChange==null)return;vartopic=$"opcxmlda/{veneers.Machine.Id}/splunk/{veneer.Name}";stringpayload=JObject.FromObject(onChange).ToString();awaitveneers.Machine.Broker.PublishChangeAsync(topic,payload);}

Example: fanuc-driver

The handler prepares changed data into InfluxDb line format.

publicoverrideasyncTask<dynamic?>OnDataChangeAsync(Veneersveneers,Veneerveneer,dynamic?beforeChange){if(veneer.Name=="axis_data"){varpayload=newLineProtocolWriter(Precision.Milliseconds).Measurement(veneer.Name).Tag("machine_id",veneers.Machine.Id).Tag("path_no",veneer.Marker[0].path_no.ToString()).Tag("axis_name",(veneer.Marker[1].name+veneer.Marker[1].suff).ToString()).Field("position",(float)veneer.LastArrivedValue.pos.absolute).Field("feed",(float)veneer.LastArrivedValue.actf);returnpayload;}returnnull;}

The InluxDb line payload is then published to an MQTT broker.

protectedoverrideasyncTaskafterDataChangeAsync(Veneersveneers,Veneerveneer,dynamic?onChange){if(onChange==null){return;}vartopic=$"fanuc/{veneers.Machine.Id}/influx";stringpayload=JObject.FromObject(onChange).ToString();awaitveneers.Machine.Broker.PublishChangeAsync(topic,payload);}

Lifecycle

Overview

graph LR
start:::in --> id1(parse args) --> id2(parse config) --> id3(create machines) --> id4(execute) --> id5(shutdown) --> stop:::out
classDef in fill:lightgreen;
classDef out fill:red;

Loading

staticasyncTaskMain(string[]args){dynamicconfig=awaitBootstrap.Start(args);Machinesmachines=awaitMachines.CreateMachines(config);awaitmachines.RunAsync();awaitBootstrap.Stop();}

Arguments

Logging

Relative or absolute path to logging configuration file.

Argument: --nlog

Default: nlog.config

Example: nlog.config, /etc/fanuc/nlog.config

WARNING: Target log file is defined inside nlog.config

NLog Documentation

Configuration

Relative or absolute path to driver configuration file.

Argument: --config

Default: config.yml

Example: config.yml, /etc/fanuc/config.yml

Terminating

Data collection will continue to run until the application is stopped or Shutdown() is invoked on all Machine instances.

Configuration

Driver configuration is maintained in the config.yml file. You can read more about YAML structure here. Multiple machines can be added and are differentiated by their id key.

Machine

Parameters relevant to driver initialization.

id : Unique machine identifier.

enabled : Toggles the active collection state. Disabled machines are not initialized upon startup.

type: Machine class type initialized on startup.

strategy : Collector class type initialized on startup.

handler : Handler class type initialized on startup.

Examples:

machines:
- id: bender01enabled: !!bool truetype: l99.driver.opcxmlda.OpcxmldaMachine, opcxmldastrategy: l99.driver.opcxmlda.collectors.Basic01, opcxmldahandler: l99.driver.opcxmlda.handlers.SHDR, opcxmlda

machines:
- id: cnc01enabled: !!bool truetype: l99.driver.fanuc.FanucMachine, fanucstrategy: l99.driver.fanuc.collectors.NLuaRunner, fanuchandler: l99.driver.fanuc.handlers.Native, fanuc

Broker

Parameters relevant to built-in MQTT client.

enabled : Toggles the active client state. Disabled clients are not initialized upon startup.

net_ip : Broker IP address.

net_port : Broker TCP port.

auto_connect : Automatically connect to broker on startup. In the example of SparkplugB, this parameter should be false.

publish_status : Publish status, typically at the end of sweep.

publish_arrivals : Publish all data every sweep.

publish_changes : Publish data only when it changes.

publish_disco : Publish machine information to discovery topic.

disco_base_topic : Topic used for discovery.

anonymous : Connect anonymously or use credentials.

user : Broker user.

password : Broker password.

Examples:

machines:
- id: bender01...broker:
enabled: !!bool truenet_ip: 10.20.30.40net_port: !!int 1883auto_connect: !!bool truepublish_status: !!bool truepublish_arrivals: !!bool truepublish_changes: !!bool truepublish_disco: !!bool truedisco_base_topic: opcxmldaanonymous: !!bool falseuser: client1password: secret123

type

machine[id].type specific configuration.

Examples:

machines:
- id: demotype: l99.driver.opcxmlda.OpcxmldaMachine, opcxmlda...l99.driver.opcxmlda.OpcxmldaMachine, opcxmlda:
sweep_ms: !!int 1000net_uri: http://opcxml.demo-this.com/XmlDaSampleServer/Service.asmxnet_timeout_s: !!int 3data:
- Dynamic/Analog Types/Double:
- Dynamic/Analog Types/Int:
- Dynamic/Analog Types/Double[]:
- Static/Simple Types/String:
- Static/Simple Types/DateTime:
- Static/ArrayTypes/Object[]:
- Dynamic/Analog Types/Fools/Guildenstern:
- Dynamic/Enumerated Types/Gems:
- SomeUnknownItem:

machines:
- id: cnc01type: l99.driver.fanuc.FanucMachine, fanuc...l99.driver.fanuc.FanucMachine, fanuc:
sweep_ms: !!int 1000net_ip: 10.20.30.50net_port: !!int 8193net_timeout_s: !!int 3

machines:
- id: fio01type: l99.driver.factoryio.FactoryioLocalMachine, factoryio...l99.driver.factoryio.FactoryioLocalMachine, factoryio:
sweep_ms: !!int 100

machines:
- id: fio02type: l99.driver.factoryio.FactoryioRemoteMachine, factoryio...l99.driver.factoryio.FactoryioRemoteMachine, factoryio:
sweep_ms: !!int 1000net_uri: http://10.20.32.6:7410net_timeout_s: !!int 3

strategy

machine[id].strategy specific configuration.

machines:
- id: cnc02strategy: l99.driver.fanuc.collectors.NLuaRunner, fanuc...l99.driver.fanuc.collectors.NLuaRunner, fanuc:
script: lua/test1.lua

machines:
- id: fio01strategy: l99.driver.factoryio.collectors.BasicLocal01, factoryio...l99.driver.factoryio.collectors.BasicLocal01, factoryio:
sub_topic: factoryio/fio01/io

handler

machine[id].handler specific configuration.

machines:
- id: bender01handler: l99.driver.opcxmlda.handlers.SHDR, opcxmlda...l99.driver.opcxmlda.handlers.SHDR, opcxmlda:
port: !!int 7878verbose: !!bool truelua_head: | luanet.load_assembly 'System'; luanet.load_assembly 'Newtonsoft.Json'; JObject = luanet.import_type 'Newtonsoft.Json.Linq.JObject';data:
- avail:
shdr:
name: availcategory: eventeval: | return "AVAILABLE";