Workflow manager

Workflow manager is a suit of tools to provide workflow control in javascript. It has:

• Manager to manager workflow
• Producer (abstract class) to produce data
• Relation to descripe relationship between each producer

with features:

• Multiple input / output for all producers
• Asynchronized data stream
• Relations can use javascript code to allow full or part data transfer from one producer to another

For example, in this case, run sequence should be ROOT -> 1, 2-1, 3 -> 2-2 -> 4.

How to use

npm install @ekifvk/workflow

1. Create a WorkflowManager.

constmanager=newWorkflowManager();

2. Create some producers.

consta=newSomeProducer();constb=newSomeProducer2();

3. Add relations.

// last param is the condition, use 'input' to access input data, like 'return input.a === "a"'.Relation.create(a,b,'return input.a === a');Relation.create(a,b,input=>input.a===a);// or use function

4. Register entrace.

manager.entrance=a;manager.output=b;// Optional. Set output will only return output's result for memory optimization.

5. Run workflow.

if(manager.unreachableNodes.length>0){// Check if workflow's DAG has unreachable nodes.thrownewTypeError(`Has unreachable node!`);}else{manager.run(/* input array */,/* Optional environment parameter */,/* Optional options */).then(...).catch(...);// e.g.manager.run([0]);// With environment variablemanager.run([0],{skip: 0});// With env and optionsmanager.run(0,{skip: 0},{singleInput: true,returnLast: true});// Way to receive any producer's data, not requiredmanager.resultObserver=data=> ...;}

Stop, pause and resume

Workflow cannot pause/stop current running producer, but it can pause/stop before process next producer.

mamager.pause().then(()=>{console.log('Paused!');});setTimeout(()=>{manager.resume();},3000);manager.stop().then(()=>{console.log('Stopped!');});

How to write producer

All producers must extend Producer, which is an abstract class, they can have these function implementations:

publicabstractintroduce(): string;publicabstractparameterStructure(): IParameterDescriptor;publicabstractproduce(input: any[],params: ParameterTable,context: WorkflowContext): any[]|Promise<any[]>;protectedcheckParameters(params: {[key: string]: any }): {[key: string]: any};

Remember all producers are in multi-input (input is array) multi-output (output should be array or array inside promise) mode.

The checkParameters function should check the parameter's type and change them if needed. The introduce function should return producer's description. The parameterStructure function should return a list of Parameter which defined initialize's parameter structure (still it has no use). The _produce function should produce the data and return new data.

One more thing, producers can access their parameter by using params. Of course they can handle parameters by their own, but using params will have an automatic cache & replace if inject by relation(s) is active when running.

Producer should use params.get(/* name */) to get parameter in purpose to support parameter injection. Or it can use this.parameters.get(/* name */) to access parameters without injection.

returninput+params.get<number>('number1');

Producer can also access workflow's current state by using third parameter context, like cancel current workflow:

context.cancelled=true

Or find other producer's state:

constisOtherFinished=context.finished.includes('Other ID')

Or access environment parameters (if have):

constskip=context.environment.skip;

For example, a producer that returns { key, value } pairs of any object can be like this:

exportclassKeyValuePairProducerextendsProducer{publicintroduce(): string{return'Read input object\'s key and value and return { key: key, value: value } array';}publicparameterStructure(): IParameterDescriptor{return{};// No parameter}publicproduce(input: any[],params: ParameterTable,context: WorkflowContext): any[]|Promise<any[]>{constresult: any[]=[];input.forEach(data=>{// Map the dataconstkeys=Object.keys(data).forEach(key=>{result.push({key: key,value: data[key]});});});returnresult;}}

Pre-defined producers

Under '@ekifvk/workflow/dist/producers'.

Empty producer

Producer that does nothing.

Wrap producer

This producer can create a temporary producer using given code.

constwrap=newWrapProducer();wrap.initialize({handler: (input,params)=>input// just like writing a producer});// orwrap.initialize({handler: async(input,params)=>input// Promise is also supported});

Data pick producer / Structured data pick producer

Pick data from json object or array, see JPQuery's document for more information.

constpicker=newDataPickerProducer();picker.initialize({query: '/times/success[-1 -> -2]'})constpicker2=newStructuredDataPickProducer():
picker2.initialize({query: {lastTime: '/times/success[-1]',rawTime: '/times',lastTwoTimes: ['/times/success[-2]','/times/success[-1]']}});

Value convert producer

Use given structure to focus on input data's specific places, then using rules to convert the value. See this producer's parameterStructure() for more information.

constconverter=newValueConvertProducer();converter.initialize({rules: [default: true,value: i=>i*2],structure: {data1: {data2: [{data3: true}]}}});

Subworkflow producer / Circulate subworkflow producer

Contains a fully functional workflow, use that workflow's result as producer's return data. The first one only run workflow once, while other one run workflow multiple times.

constcirculate=newCirculateSubWorkflowProducer();constsubWorkflow=newWorkflowManager();// Another workflowsubWorkflow.entrance=newLogProducer('sub1');// Enrance pointcirculate.initialize({definition: subWorkflow,// Use that workflowenv: {skip: 0},// EnvironmentonResult: (input: IWorkflowResult[])=>{// Function that generate resultreturninput.map(v=>v.data[0].data[0]);},onLoop: (env: {[key: string]: any},context: WorkflowContext,output: IWorkflowResult)=>{// Change environmentenv.skip++;// While's condition. It will run loop two times.returnenv.skip<3;}});

Static workflow definition

A JSON object can be created to define a workflow or part of workflow (see IWorkflow):

{
"producers": [], // Optional in each file"relations": [], // Optional in each file"entrance": "", // Optional in each file, must has one in all files. Entrance producer's ID."output": ""// Optional. Output producer's ID of workflow.
}

Elements in producers should follow this structure (see IProducer):

{
"id": "String. ID of this producer",
"type": "String. the type of this producer. Normally if producer class's name is <name>Producer, then <name> is the type of that producer.",
"parameters": "IParameterDescriptor. This producer's parameters.",
"description": "String. Optional. Description of this producer.",
"runningDelay": "Number. Optional. Delayed millisecond before run producer.",
"replyDelay": "Number. Optional. Delayed millisecond before return producer's result.",
"proceed": "Function/Function's content in string. Optional. Function runs after proceed data (after applied delay time), it takes one param (input: any[]) and return an array.",
"errorHandler": "Function/Function's content in string. Optional. Function that handles error. If this function still returns error, workflow will be terminated. Must returns array."
}

Elements in relations should follow this structure (see IRelation):

{
"from": "String. Parent producer's ID.",
"to": "String. Child producer's ID.",
"inject": "String. Optional. Inject parameter name. Inject parameter means data transfered by this relation will be inject to producer as a temporaty \"initialize\" parameter only for this round of produce.",
"condition": "Function/Function's content in string. Optional. It takes one param (input: any) and return true/false. Condition to judge the data that pass through this relation (in JavaScript)."
}

Call WorkflowManager.fromDefinitions() and provide all definition objects to get a workflow. The first paramater should be a function, which has a string param as type to return the constructor of Producer. Rest params will be combined to one, please notice that one and only one of them must has entrance property.

WorkflowManager.fromDefinitions(type=>{switch(type){case'empty':
returnEmptyProducer;case'datapick':
returnDataPickProducer;default:
thrownewTypeError(`Unknow type ${type}`);}}, ...someDefinitions);

Example

varworkflow=require('@ekifvk/workflow');varmanager=newworkflow.WorkflowManager();classLogProducerextendsworkflow.Producer{introduce(){return'';}parameterStructure(){return{log: {type: workflow.ParameterType.String,optional: true,default: '',description: 'Log content'}};}checkParameters(params: {[key: string]: any}){params.log=params.log||'';returnparams;}produce(input,activeParams){constcontent=activeParams.get('log');console.log(content);returninput;}}varentrance=newLogProducer('entrance');vartest1=newLogProducer('test1');vartest2=newLogProducer('test2');vartest3=newLogProducer('test3');vartest4=newLogProducer('test4');vartest5=newLogProducer('test5');entrance.initialize({log: 'entrance'});test1.initialize({log: '1'});test2.initialize({log: '2'});test3.initialize({log: '3'});test4.initialize({log: '4'});entrance.relation(newworkflow.Relation(entrance,test1));test1.relation(newworkflow.Relation(test1,test2));test2.relation(newworkflow.Relation(test2,test3));test3.relation(newworkflow.Relation(test3,test4));manager.entrance=entrance;manager.output=test3;// Check errorif(manager.unreachableNodes.length>0){// unreachableNodes is computed field, call it as less as possiblethrownewTypeError(`Has unreachable node!`);}// Run workflowmanager.run(0).then(v=>console.log(v)).catch(e=>console.log('error: '+e));// Pause and resumemanager.pause().then(()=>{setTimeout(()=>{manager.resume();},3000);});