Skip to content

Development Guide

Jump to bottom
Jason Garber edited this page Apr 3, 2025 · 5 revisions

Welcome to the toolkit development guide!

The Kenna toolkit is a consolidated toolkit of functions for security professionals using Kenna and intended to be run in a docker container in production. During development, however, most developers prefer to set up an environment on their local system.

Getting the code and dependencies

Local Ruby development

  1. you should have a ruby setup, preferably set up with rbenv. The current Ruby version used by the connector is 3.2.2.
  2. in the toolkit directory, install bundler using the command gem install bundler
  3. install the dependencies using bundler bundle install

Using the container

docker build . -t toolkit:latest
docker run -v $(pwd):/opt/app/toolkit --rm -it --entrypoint bash toolkit

First steps

A good place to start with development on the toolkit is by looking at the example task. Two methods matter:

  • self.metadata: a method on the Task class, which self-describes how the task can be used, and what options it takes
  • run: the method that operates on the data it's passed.

One easy thing to do for new developers is to copy the toolkit/tasks/connectors/_sample_task folder to a new folder and adjust the name of the folder and the task (which should always match) to something like 'hello'. This creates a new task that is automatically available in the toolkit and can be called simply by running (in the root) bundle exec ruby ./toolkit.rb task=hello

Code Organization

We recommend splitting the code into 2 sections: the Task itself and an APIClient. The APIClient is responsible only for interactions with the service (scanner) to obtain the data needed by the task. The APIClient can also format the obtained data to ease the Task process. The Task is responsible for the creation of Kenna objects, upload, and execution of Kenna processes.

The common folder layout is this:

|-- sample_task
| |-- lib
| | |-- sample_api_client.rb
| | |-- sample_custom_helpers.rb
| |-- sample_readme.md
| |-- sample_task.rb
  1. sample_task.rb is expected to contain your Task class which defines the run and self.metadata methods.
  2. sample_readme.md is the connector's documentation markdown file.
  3. The lib folder is where client classes and helper modules live.

Main run method

We recommend balancing the code in several methods avoiding putting too much in the main run method.

The following is a simplified and fully commented code snippet of the entire process and can be used as guideline:

defruninitialize_options# Process set options from command line parametersclient=Client.new(user_id,user_token)# Instantiate the client using options passed as parameterspage=1loopdopage_data=client.get_page(page)# Get data from the client in batchespage_data.eachdo |issue| # For each resulting issue ...asset=extract_asset(issue)# Builds an asset for Kennafinding=extract_finding(issue)# Builds the finding (issue) object for Kennadefinition=extract_definition(issue)# Builds the issue definition (unique definitions)create_kdi_asset_finding(asset,finding)# Creates the association in current Kenna batchcreate_kdi_vuln_def(definition)# Creates the definition (deduplicated) in current Kenna batchend# Below line uploads current batch to Kennakdi_upload(@output_directory,"report_#{page}.json",@kenna_connector_id,@kenna_api_host,@kenna_api_key,@skip_autoclose,@retries,@kdi_version)breakifpage_data.empty?# Stop loop if there is no more datapage += 1end# Below line starts the import process in Kenna for all uploaded batcheskdi_connector_kickoff(@kenna_connector_id,@kenna_api_host,@kenna_api_key)rescueApiError=>e# Api exception handler for the entire processfail_taske.messageend

In the example above the methods extract_asset, extract_finding and extract_definition should return a hash with JSON data in the format specified by the KDI Json Format.

Depending on the final destination for the data upload, you need to use one of create_kdi_asset_finding or create_kdi_asset_vuln helper methods.

Please, refer to the provided sample for specific details on Client implementation, exception handling, and log tracing.

Batching

The connector process runs in a constrained environment and must wisely use the memory and processor resources, making use of batching or pagination techniques.

Usually batch_size and/or page_size are added to the connector parameters with common defaults, normally a number between 100 and 500. In the code, these parameters should be used to split API calls to the source application as well as Kenna, contributing this way to memory usage reduction on both ends.

Helpers

The Toolkit provides several helper methods included in the Kenna::Toolkit::BaseTask class, which is the common ancestor for Task classes.

classBaseTaskincludeKenna::Toolkit::HelpersincludeKenna::Toolkit::KdiHelpers
...
end

You can find the source code in toolkit/lib/helpers.rb and toolkit/lib/kdi/kdi_helpers.rb.

Most commonly used methods in Helpers are:

  • print(message = nil) # Log useful information
  • print_good(message = nil) # Log something was successfull
  • print_error(message = nil) # Log and error (doesn't break task execution)
  • print_debug(message = nil) # Log message only if @options[:debug] == true
  • fail_task(message) # Log message as error and terminate execution returning a non zero exit code
  • remove_html_tags(string) # Return the argument string whithout any html tags

Most commonly used methods in KdiHelpers are:

  • create_kdi_asset(asset_hash) # Create kdi asset with the hash argument and keep it in memory up to next upload
  • create_kdi_asset_vuln(asset_hash, vuln_hash) # Create kdi asset with associated vuln
  • create_kdi_asset_finding(asset_hash, finding_hash) # Create kdi asset with associated finding
  • kdi_upload(output_dir, filename, kenna_connector_id, ...) # Upload the current batch of assets, vulns and findings to Kenna
  • kdi_connector_kickoff(kenna_connector_id, kenna_api_host, ...) # Instruct Kenna to process all previous pending uploads

Submitting a Pull Request

Follow instructions in the README.md to submit a pull request. Your PR will get merged faster if all checks pass! Please ensure you've covered your change sufficiently with specs.

Spec writing tips

Clone this wiki locally