Vignet

The missing GitOps piece: expose a Git repository behind an authenticated API to perform updates with authorization.

Where does it fit in?

GitOps tools already handle many aspects of syncing infrastructure resources (e.g. in Kubernetes) from Git repositories. To fully integrate into the delivery workflow, updates to the Git repo should be able to be performed via an API for automation (e.g. set a new image tag for a release). Since a Git repo can store the complete infrastructure, updates should be protected - an application pipeline should only be allowed to update its own declaration.

This is why we created Vignet:

• It runs as a standalone service in your infrastructure
• It will get access to your GitOps repositories
• It exposes an authenticated Rest API for patching YAML declarations via commands
• It integrates flexible authorization via OPA (Open Policy Agent) rules to decide if a command should be allowed
• It is easy to integrate into GitLab CI, GitHub Actions and other systems
• Works perfectly with Flux, ArgoCD or other GitOps tools

Design principles

• Vignet is stateless, repositories and authorization are configured via configuration files
• Policies are customizable via Open Policy Agent (OPA) rules

Current state

It is in the early stages of development, but it should already be usable for integration in GitLab CI. Configuration and API is subject to change. Use in production at your own risk.

Command reference

NAME:
vignet - The missing GitOps piece: expose Git repositories for automation via an authenticated HTTP API
USAGE:
vignet [global options] command [command options] [arguments...]
DESCRIPTION:
The default command starts an HTTP server that handles commands.
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help (default: false)
authorization
--policy value Path to an OPA policy bundle path, uses the built-in by default [$VIGNET_POLICY]
configuration
--config value, -c value Path to the configuration file (default: "config.yaml") [$VIGNET_CONFIG]
http
--address value Address for HTTP server to listen on (default: ":8080") [$VIGNET_ADDRESS]
logging
--force-logfmt Force logging to use logfmt (default: false) [$VIGNET_FORCE_LOGFMT]
--verbose Enable verbose logging (default: false) [$VIGNET_VERBOSE]

Configuration

Vignet is configured via flags or env vars and a YAML configuration file. The configuration file makes it easier to manage multiple repository configurations. A custom Open Policy Agent policy bundle can be used to customize authorization via the policy flag.

Example configuration

authenticationProvider:
# Use a GitLab job token authentication providertype: gitlab# Configuration for the GitLab authentication providergitlab:
# URL to the GitLab instanceurl: https://gitlab.example.com# Configure repositories that can be accessed by Vignetrepositories:
# Repository namemy-project:
# URL to the repositoryurl: https://gitlab.example.com/my-group/my-project.gitbasicAuth:
# Username doesn't matter for GitLabusername: gitlab# Use an access token with scopes "read_repository", "write_repository"password: an-access-tokencommit:
# Default message to use for a commit if none is specified in a requestdefaultMessage: "Automated update"# Default author to use for a commit if none is specified in a requestdefaultAuthor:
name: Git autopilotemail: bot@example.com

Rest API

POST /patch/{repository}

Pulls the repository, patches files according to commands, creates a commit and pushes to the repository.

Responds with status code 200 on success.

Body

• commitobject Commit options (optional)
  • messagestring Commit message (optional)
  • committerobject Committer for the commit (optional)
    • namestring
    • emailstring
  • authorobject Author for the commit (optional)
    • namestring
    • emailstring
• commandsarray Commands to perform, one of setField and n.n. must be set
  • pathstring Path to the file to patch (relative from repository root)
  • setFieldobject Perform a set field command (optional)
    • fieldstring Field to set with dot path syntax, JSONPath features are supported (see examples)
    • valuemixed Value to set the field to
    • createboolean Create the field (and intermediate path) if it doesn't exist (optional, defaults to false)
  • createFileobject Perform a create file command to create a new file (optional)
    • contentstring Content of the file to create
  • deleteFileobject Perform a delete file command to delete a file (optional)

Examples

Setting a field in a YAML file

POST http://localhost:8080/patch/infra-testAuthorization: Bearer [CI_JOB_JWT]Content-Type: application/json
{
"commit": {
"message": "Bump image to 1.2.5"
},
"commands": [
{
"path": "my-group/my-project/release.yml",
"setField": {
"field": "spec.values.image.tag",
"value": "1.2.5"
}
}
]
}

Using JSONPath

JSONPath can be used to reference a field by array index, filter expression or other features:

POST http://localhost:8080/patch/infra-testAuthorization: Bearer [CI_JOB_JWT]Content-Type: application/json
{
"commit": {
"message": "Bump image to 1.2.5, update BUILD_ID"
},
"commands": [
{
"path": "my-group/my-project/deployment.yml",
"setField": {
"field": "spec.template.spec.containers[0].image",
"value": "registry.example.com/my/image:1.2.5"
}
},
{
"path": "my-group/my-project/deployment.yml",
"setField": {
"field": "spec.template.spec.containers[0].env[?(@.name == 'BUILD_ID')].value",
"value": "987654"
}
}
]
}

Using Curl is a convenient way to integrate Vignet into GitLab CI:

curl -s --fail-with-body -H "Authorization: Bearer $CI_JOB_JWT" -H "Content-Type: application/json" -d @- http://localhost:8080/patch/infra-test <<JSON{ "commit": { "message": "${CI_PROJECT_PATH}: Release ${CI_REGISTRY_TAG} to ${CI_ENVIRONMENT_SLUG}" }, "commands": [ { "path": "projects/${CI_PROJECT_PATH}/release-${CI_ENVIRONMENT_SLUG}.yaml", "setField": { "field": "spec.values.app.image.tag", "value": "$CI_REGISTRY_TAG" } } ]}JSON

Writing a new file

POST http://localhost:8080/patch/infra-testAuthorization: Bearer [CI_JOB_JWT]Content-Type: application/json
{
"commit": {
"message": "Bump image to 1.2.5"
},
"commands": [
{
"path": "my-group/my-project/new.yml",
"createFile": {
"content": "---\nversion: 1.2.3\n"
}
}
]
}

Curl can be paired with jq to read an existing file and convert it to a JSON string:

# Prepare a YAML file for the new release before this command
yaml_file=new-release.yml
curl --fail-with-body -H "Authorization: Bearer $CI_JOB_JWT" http://localhost:8080/patch/infra-test -H "Content-Type: application/json" -d \
@<(jq -n --arg yaml "$(jq -sR .$yaml_file)""$(cat <<JSON{ "commit": { "message": "${CI_PROJECT_PATH}: Release ${CI_REGISTRY_TAG} to ${CI_ENVIRONMENT_SLUG}" }, "commands": [ { "path": "projects/${CI_PROJECT_PATH}/new-${CI_ENVIRONMENT_SLUG}.yaml", "createFile": { "content": \$yaml } } ]}JSON)")

Authentication

GitLab

• GitLab CI generates a job token env var CI_JOB_JWT for each job. It contains claims about the user, project and repository.
• This token needs to be passed via Authorization: Bearer [CI_JOB_JWT] header to Vignet.
• Requests are denied if the token is invalid or missing.
• Claims in the token are passed to the authorization policy to check if the request should be allowed.

Authorization

Vignet will pass the authentication context and request information to the policy for decision.

Default policy

Patch request

• path Accepts only .yml and .yaml files

The further policy behavior depends on the authentication provider:

GitLab

• path Requires a prefix of the GitLab project path (of the job passing the job token).

  E.g. a job token with project_path: "my-group/my-project" will only authorize requests for my-group/my-project/**/*.{yml,yaml}.

Known limitations

• Currently, only authentication via a GitLab job token is supported
• There is only a setField command for now

License

MIT