Fiddler

Fiddler is a stream processor built in rust that uses a yaml based configuration file syntax to map together inputs, processors, and outputs. Fiddler takes a module approach to inputs, processors, and outputs allowing users to take advantage of incremental releases of new modules or develop their own to run within the fiddler runtime. Whether you are looking to utilize fiddler for it's processing capabilities through the CLI or integrate directly in your own applications, head over to our [Documentation](https://rc1405.github.io/fiddler/latest) to get started.

Getting Started

Running as a CLI

Installation

1. Grab a release for your OS here
2. Install with Cargo cargo install fiddler
3. Build with Cargocargo build --release --features all

Usage

Usage: fiddler <COMMAND>
Commands:
lint Data Stream processor CLI written in rust
run Data Stream processor CLI written in rust
test Data Stream processor CLI written in rust
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help

Run

Data Stream processor CLI written in rust
Usage: fiddler run [OPTIONS]
Options:
-c, --config <CONFIG>
-l, --log-level <LOG_LEVEL> [default: none] [possible values: info, debug, trace, error, none]
-h, --help Print help
-V, --version Print version

1. fiddler run -c <path_to_config_file> [ -c ... ]

Lint

Data Stream processor CLI written in rust
Usage: fiddler lint [OPTIONS]
Options:
-c, --config <CONFIG>
-h, --help Print help
-V, --version Print version

1. fiddler lint -c <path_to_config_file> [ -c ... ]

Test

Data Stream processor CLI written in rust
Usage: fiddler test [OPTIONS]
Options:
-c, --config <CONFIG>
-l, --log-level <LOG_LEVEL> [default: none] [possible values: info, debug, trace, error, none]
-h, --help Print help
-V, --version Print version

Tests within fiddler are designed to test the processing pipeline configuration. The expected naming syntax is <filename>_test.yaml. I.e. if you have a configuration input.yaml. The expected filename is input_test.yaml.

The test file syntax is as follows:

- name: name_of_testinputs:
- list of expected input stringsexpected_outputs:
- list of expected output strings

For example:

- name: input_testinputs:
- Hello Worldexpected_outputs: - Hello World

1. fiddler-cli test -c <path_to_configuration>.yaml

Integration

Building own CLI with Additional modules

Creating your own fiddler-cli requires a binary crate with at least two dependencies, fiddler_cmd and fiddler.

cargo new my-cli --bin
cargo add fiddler
cargo add fiddler_cmd

Next create your runtime, by default in src/main.rs

use fiddler_cmd::run;use fiddler::Error;use fiddler::config::{ConfigSpec, register_plugin};#[tokio::main]asyncfnmain() -> Result<(),Error>{// insert registration of custom modules hereregister_plugin("mock".into(),ItemType::Input, conf_spec, create_mock_input)?;// run the normal CLIrun().await}

Integrating the runtme into your own application

Head on over to docs.rs to view the latest API reference for utilizing fiddler runtime.

Contributing

Contributions are welcome and encouraged! Contributions come in many forms. You could:

1. Submit a feature request or bug report as an issue.
2. Ask for improved documentation as an issue.
3. Comment on issues that require feedback.
4. Contribute code via pull requests.

Note that unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in fiddler by you shall be licensed under the Apache License, Version 2.0, without any additional terms or conditions.