CI4TOSCA Prototype

The CI4TOSCA Prototype is a Go-based tool for generating CI/CD pipelines from TOSCA build descriptions. It serves as the reference implementation of the CI4TOSCA framework, which introduces a declarative build stage to unify artifact creation within a TOSCA model.

This prototype supports Jenkins, GitHub Actions, and GitLab CI pipelines, integrates with Artifactory and MinIO for artifact storage, and features a plugin-based architecture for extensibility.

---

Overview

Traditional DevOps workflows maintain separate scripts for build and deployment, leading to redundancy, complexity, and vendor lock-in. CI4TOSCA addresses this by embedding build instructions directly in the TOSCA model and transforming them into executable pipelines, reducing configuration effort and increasing maintainability.

This prototype was evaluated through 576 pipeline executions and demonstrated:

• An 83% reduction in CI/CD configuration files
• A 40% reduction in lines of configuration code
• Full tool-switching support with no model changes

For evaluation details, see:
=> CI4TOSCA Evaluation

---

Features

• Declarative Build Descriptions: Define build logic directly within TOSCA service templates.
• Pipeline Generation: Generate pipelines for Jenkins, GitHub Actions, and GitLab CI.
• Plugin Architecture: Easily add new artifact types, pipeline tools, and storage backends.
• Artifact Storage Support: Uploads to Artifactory, MinIO, S3, and other backends.
• TOSCA Version Support: Supports TOSCA 1.3, 2.0, and future versions via YAML templates.
• Preview Mode: View generated pipelines without writing them to disk.
• Git Integration: Automatically clones repositories referenced in TOSCA inputs.
• Component-Level Configuration: Assign individual pipeline tools and outputs per service.

---

Installation

Prerequisites

• Go 1.20 or later
• Valid TOSCA build description file (v1.3 or v2.0)

Steps

git clone https://github.com/CI4TOSCA/Prototype
cd Prototype
go mod tidy
go build -o prototype

---

Usage

Generate a Single Pipeline

go run orchestrator.go \
--tosca path/to/tosca.yaml \
--tool Jenkins \
--output Jenkinsfile \
--storage https://artifact-store.example.com \
--storageUser user \
--storagePassword pass

Generate Multiple Pipelines

go run orchestrator.go \
--tosca path/to/tosca.yaml \
--tool comp1=GitHub,comp2=Jenkins \
--output comp1=github.yaml,comp2=jenkinsfile \
--storage https://artifact-store.example.com \
--storageUser user \
--storagePassword pass

Preview a Pipeline

go run orchestrator.go \
--tosca path/to/tosca.yaml \
--tool GitHub \
--storage https://artifact-store.example.com \
--storageUser user \
--storagePassword pass \
--preview

Testing

Run all tests:

go test ./... -v

---

Command-Line Options

Flag	Description
--tosca	Path to the TOSCA input file.
--tool	Pipeline tool (global or per-component).
--output	Output file(s) (global or per-component).
--storage	Artifact storage URL.
--storageUser	User name for the storage backend.
--storagePassword	Password for the storage backend.
--preview	Preview the pipeline without saving it.
--executeURL	URL of pipeline tool.
--executeUser	User for pipeline tool.
--executeToken	Token for pipeline tool.
--notificationMethod	Notification type (e.g. slack).
--notificationWebHook	Webhook URL for notifications.

---

Extending the CI4TOSCA Prototype

The prototype is structured for extensibility. Each plugin type implements a common interface and is registered dynamically.

Add a New Pipeline Tool

1. Implement the Pipeline interface in pipelines/:

typeMyCustomPipelinestruct {
BasePipeline
}
func (p*MyCustomPipeline) transform(stages []models.Stage) (string, error) { ... }
func (p*MyCustomPipeline) execute() (string, error) { ... }
func (p*MyCustomPipeline) validate() error { ... }

2. Register it:

funcinit() {
pipelines.RegisterPipeline("MyCustomPipeline", func(base pipelines.BasePipeline) pipelines.Pipeline {
return&MyCustomPipeline{BasePipeline: base}
})
}

---

Add a New Artifact Type

1. Implement the Artifact interface in artifacts/:

typeMyCustomArtifactstruct {
BaseArtifact
}
func (m*MyCustomArtifact) GetType() string {
returnm.Type
}
func (m*MyCustomArtifact) GenerateStages(buildDescription models.BuildDescription, artifactStorageURLstring) ([]models.Stage, error) { ... }
func (m*MyCustomArtifact) GetBuildDescription(artifactTypestring, componentNamestring, artifactInputsmap[string]interface{}) models.BuildDescription { ... }

2. Register it:

funcinit() {
artifacts.RegisterArtifact("MyCustomArtifact", func(base artifacts.BaseArtifact) artifacts.Artifact {
return&MyCustomArtifact{BaseArtifact: base}
})
}

---

Add a New Storage Backend

1. Implement the ArtifactStorage interface in storage/:

typeMyCustomStoragestruct { ... }
funcNewCustomStorage(artifactStorageURLstring) (*MyCustomStorage, error) { ... }
func (s*MyCustomStorage) GenerateUploadCommand(filePathstring, repoPathstring) (string, error) { ... }
func (s*MyCustomStorage) GetArtifactStorageURLs(filePathstring, repoPathstring) string { ... }

2. Update the switch logic in storage_handler/NewStorageHandler(...) to support your backend via URL scheme.

---

Add a New Notifier

1. Implement the Notifier interface in notifications/:

typeMyNotifierstruct {
WebhookURLstring
}
func (m*MyNotifier) Send(messagestring) error { ... }

2. Register it:

funcinit() {
RegisterNotifier("MyNotifier", NewMyNotifier)
}
funcNewMyNotifier(urlstring) Notifier {
return&MyNotifier{WebhookURL: url}
}

---

Example TOSCA Files

TOSCA 1.3 Example

tosca_definitions_version: tosca_simple_yaml_1_3node_types:
my_application:
derived_from: tosca.nodes.SoftwareComponentmetadata:
name: my_applicationversion: 1.0.0artifacts:
my_image_source:
file: "./source.tar.gz"type: CI4TOSCA.Sourcesmy_application_binary: # For Binary imagesfile: my_imagetype: CI4TOSCA.Binary.Imageinterfaces:
Standard:
build:
inputs:
architecture:
type: "x86_64"directory: "src/"build_steps:
- step: compilecommand: "gcc -o my_app main.c"
- step: packagecommand: "tar -czf my_app.tar.gz my_app"my_application_docker: # For Docker imagesfile: "https://github.com/CI4TOSCA/Evaluation/3_pipeline_files/dockerfiles/VueRailsDockerfile"type: CI4TOSCA.Docker.Imageproperties:
image_name: "vue-rails"my_application_npm: # For NPM imagesfile: "https://github.com/khaledosman/angular-realworld-example-app"type: CI4TOSCA.NPM.Imageproperties:
build_command: "run"# optionalmy_application_maven: # For Maven imagesfile: "https://github.com/spring-projects/spring-petclinic"type: CI4TOSCA.Jar.Imagemy_application_extern: # For images build by external pipelinesfile: "https://github.com/my-org/my-repo/deploy.yml"type: CI4TOSCA.ExternalPipeline.Imageproperties:
api_token: "XYZ"body: "my_body"# optionaltopology_template:
node_templates:
my_application:
type: my_applicationrequirements:
- host:
node: my_hostmy_host:
type: tosca.nodes.Compute

TOSCA 2.0 Example

tosca_definitions_version: tosca_2_0service_template:
node_types:
my_application:
derived_from: tosca.nodes.SoftwareComponentmetadata:
name: my_applicationversion: 1.0.0artifacts:
my_image_source:
file: "./source.tar.gz"type: CI4TOSCA.Sourcesmy_application_binary: # For Binary imagesfile: my_imagetype: CI4TOSCA.Binary.Imageinterfaces:
Standard:
build:
inputs:
architecture:
type: "x86_64"directory: "src/"build_steps:
- step: compilecommand: "gcc -o my_app main.c"
- step: packagecommand: "tar -czf my_app.tar.gz my_app"my_application_docker: # For Docker imagesfile: "https://github.com/CI4TOSCA/Evaluation/3_pipeline_files/dockerfiles/VueRailsDockerfile"type: CI4TOSCA.Docker.Imageproperties:
image_name: "vue-rails"my_application_npm: # For NPM imagesfile: "https://github.com/khaledosman/angular-realworld-example-app"type: CI4TOSCA.NPM.Imageproperties:
build_command: "run"# optionalmy_application_maven: # For Maven imagesfile: "https://github.com/spring-projects/spring-petclinic"type: CI4TOSCA.Jar.Imagemy_application_extern: # For images build by external pipelinesfile: "https://github.com/my-org/my-repo/deploy.yml"type: CI4TOSCA.ExternalPipeline.Imageproperties:
api_token: "XYZ"body: "my_body"# optionalnode_templates:
my_application:
type: my_applicationrequirements:
- host:
node: my_hostmy_host:
type: tosca.nodes.Compute

---

License

This project is licensed under the MIT License. See the LICENSE file for details.