Crossplane Composition Functions using KCL

Introduction

Crossplane KCL function allows developers to use KCL (a DSL) to write composite logic without the need for repeated packaging of crossplane functions, and we support package management and the KRM KCL specification, which allows for OCI/Git source and the reuse of KCL's module ecosystem.

Check out these following blogs to learn more.

Here's a simple example:

apiVersion: apiextensions.crossplane.io/v1kind: Compositionmetadata:
name: examplespec:
compositeTypeRef:
apiVersion: example.crossplane.io/v1beta1kind: XRmode: Pipelinepipeline:
- step: basicfunctionRef:
name: function-kclinput:
apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Read the XR oxr = option("params").oxr # Patch the XR with the status field dxr = { **option("params").dxr status.dummy = "cool-status" } # Construct a bucket bucket = { apiVersion = "s3.aws.upbound.io/v1beta1" kind = "Bucket" metadata.annotations: { "krm.kcl.dev/composition-resource-name" = "bucket" } spec.forProvider.region = option("oxr").spec.region } # Return the bucket and patched XR items = [bucket, dxr] - step: automatically-detect-ready-composed-resourcesfunctionRef:
name: function-auto-ready

Install the KCL Function to Cluster

cat <<EOF | kubectl apply -f -apiVersion: pkg.crossplane.io/v1kind: Functionmetadata:
name: kcl-functionspec:
package: xpkg.upbound.io/crossplane-contrib/function-kcl:latestEOF

Using this Function

Source Support

To use a KCLInput as the function config, the KCL source must be specified in the source field. Additional parameters can be specified in the params field. The params field supports any complex data structure as long as it can be represented in YAML. Besides, the function can load KCL codes from inline source, OCI source, Git source and FileSystem source.

Inline source example

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | { apiVersion = "s3.aws.upbound.io/v1beta1" kind = "Bucket" metadata.annotations: { "krm.kcl.dev/composition-resource-name" = "bucket" } spec.forProvider.region = option("oxr").spec.region }

OCI source example

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-function

Run a Composition with OCI tags

For production-like workflows, publish the KCL module as a versioned OCI artifact and pin the tag in Composition instead of embedding large inline templates.

apiVersion: apiextensions.crossplane.io/v1kind: Compositionmetadata:
name: machine-deploymentspec:
mode: Pipelinepipeline:
- step: render-machine-deploymentfunctionRef:
name: function-kclinput:
apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: oci://registry.example.com/machine-deployment?tag=0.2.0

Build and publish the module:

kcl mod pkg --vendor --target build
kcl mod push --vendor "oci://registry.example.com/machine-deployment?tag=0.2.0"

Render the composition locally:

crossplane render xr.yaml composition.yaml functions.yaml

Git source example

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: github.com/kcl-lang/modules/crossplane-xnetwork-kcl-function

FileSystem source example

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: ./path/to/kcl/file.k

Note

You can't run the FileSystem example using crossplane render because it loads templates from a ConfigMap in the cluster. You can create a ConfigMap with the templates using the following command.

kubectl create configmap templates --from-file=templates.k -n crossplane-system

This ConfigMap will be mounted to the function pod and the templates will be available in the /templates directory. See the following function config for details.

---
apiVersion: pkg.crossplane.io/v1kind: Functionmetadata:
name: function-kclspec:
package: xpkg.upbound.io/crossplane-contrib/function-kcl:latestruntimeConfigRef:
name: mount-templates
---
apiVersion: pkg.crossplane.io/v1beta1kind: DeploymentRuntimeConfigmetadata:
name: mount-templatesspec:
deploymentTemplate:
spec:
selector: {}template:
spec:
containers:
- name: package-runtimevolumeMounts:
- mountPath: /templatesname: templatesreadOnly: truevolumes:
- name: templatesconfigMap:
name: templates

Use as a Base Image

This function can also be used as a base image to build complex functions in KCL. To do this, add your KCL code to the image and set the FUNCTION_KCL_DEFAULT_SOURCE environment variable to the path where you put your code.

For example, if you have the following in main.k:

# Read the XR
oxr = option("params").oxr
# Patch the XR with the status field
dxr = {
**option("params").dxr
status.dummy = "cool-status"
}
# Construct a bucket
bucket = {
apiVersion = "s3.aws.upbound.io/v1beta1"
kind = "Bucket"
metadata.annotations: {
"krm.kcl.dev/composition-resource-name" = "bucket"
}
spec.forProvider.region = option("oxr").spec.region
}
# Return the bucket and patched XR
items = [bucket, dxr]

You can use the following Dockerfile to build a function that runs the code above and does not require any input:

FROM xpkg.upbound.io/crossplane-contrib/function-kcl:latest
ADD main.k /src/main.k
ENV FUNCTION_KCL_DEFAULT_SOURCE=/src/main.k

You may also wish to replace the /package.yaml metadata file to give your new function a unique name and remove or replace the input CRD.

Including modules in the base image

You may include KCL modules in a custom base image. First, copy the module directories into the image:

ADD kcl /kcl

Next, add a file specifying the dependencies to be added to all functions, using the syntax of the [dependencies] section of kcl.mod, for example:

ADD dependencies /dependencies

where the file dependencies contains:

example = { path = "/kcl/example" }

Finally, specify the location of the dependencies file in the ENTRYPOINT:

CMD ["--dependencies=/dependencies"]

Read the Function Requests and Values through the `option` Function

Read the ObservedCompositeResource from option("params").oxr.
Read the ObservedComposedResources from option("params").ocds.
Read the DesiredCompositeResource from option("params").dxr.
Read the DesiredComposedResources from option("params").dcds.
Read the function pipeline's context from option("params").ctx.
Return an error using assert {condition}, {error_message}.
Log variable values using the function print(variable) and it will be output to the stdout of the function pod.
Read the PATH variables. e.g. option("PATH").
Read the environment variables. e.g. option("env").

Custom Parameters

You can define your custom parameters in the params field and use option("params").custom_key to get the custom_value.

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
params:
custom_key: custom_valuesource: oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-function

Source Credentials

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
params:
annotations:
krm.kcl.dev/allow-insecure-source: "true"# For localhost OCI registrysource: oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-functioncredentials: # If private OCI registryurl: https://<oci-host-url> # or KCL_SRC_URL environment variableusername: <username> # or KCL_SRC_USERNAME environment variablepassword: <password> # or KCL_SRC_PASSWORD environment variable

You can provide credentials in a Secret to your pipeline step under the name kcl-registry.

# composition.yamlapiVersion: apiextensions.crossplane.io/v1kind: Compositionmetadata:
name: examplespec:
compositeTypeRef:
apiVersion: example.crossplane.io/v1beta1kind: XRmode: Pipelinepipeline:
- step: basicfunctionRef:
name: function-kclinput:
apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Read the XR oxr = option("params").oxr # Patch the XR with the status field dxr = { **option("params").dxr status.dummy = "cool-status" } # Construct a bucket bucket = { apiVersion = "s3.aws.upbound.io/v1beta1" kind = "Bucket" metadata.annotations: { "krm.kcl.dev/composition-resource-name" = "bucket" } spec.forProvider.region = option("oxr").spec.region } # Return the bucket and patched XR items = [bucket, dxr]credentials: # If private OCI registry
- name: kcl-registrysource: SecretsecretRef:
namespace: defaultname: default

And your secret:

apiVersion: v1kind: Secretmetadata:
name: defaultnamespace: defaultdata:
username: dXNlcm5hbWU=password: cGFzc3dvcmQ=url: aHR0cHM6Ly9leGFtcGxlLmNvbQ==

You can use these credentials with crossplane render --function-credentials=secret.yaml xr.yaml composition.yaml functions.yaml.

Run Config

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-functionconfig: # See [pkg/api/ConfigSpec]vendor: truesortKeys: truedisableNone: true# omit other fields

Dependencies

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
# Set the dependencies are the external dependencies for the KCL code.# The format of the `dependencies` field is same as the [dependencies]` in the `kcl.mod` filedependencies:
k8s = "1.31"source: | import k8s.api.core.v1 as k8core k8core.Pod { spec: k8core.PodSpec{ containers: [{ name = "main" }] } }

Expect Output

A KRM YAML list means that each document must have an apiVersion, kind through the items field or a single YAML output.

Using the items field

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | items = [{ apiVersion: "ec2.aws.upbound.io/v1beta1" kind: "Instance" metadata.name = "instance1" spec.forProvider.region: "us-east-2" }, { apiVersion: "ec2.aws.upbound.io/v1beta1" kind: "Instance" metadata.name = "instance2" spec.forProvider.region: "us-east-2" }]

Single YAML output

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | { apiVersion: "ec2.aws.upbound.io/v1beta1" kind: "Instance" metadata.name = "instance" spec.forProvider.ami: "ami-0d9858aa3c6322f73" spec.forProvider.instanceType: "t2.micro" spec.forProvider.region: "us-east-2" }

Note

When returning multiple resources, we need to set different metadata.name or metadata.annotations."krm.kcl.dev/composition-resource-name" to distinguish between different resources in the composition functions.

Target Support

The KCL function can target various types of objects:

Default: create new resources and set fields on the XR.
Resources: create new resources.
PatchDesired: set fields on existing DesiredComposed Resources.
PatchResources: set fields on existing resources fields. These resources will then be added to the desired resources map.
XR: set fields on the XR.

This is controlled by fields on the KCInput

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
# default: Defaulttarget: Default | PatchDesired | PatchResources | Resources | XRsource: | # Omit the source field ...

Extract Data from a Specific Composed Resource

To extract data from a specific composed resource by using the resource name, we can use the option("params").ocds variable, ocds is a mapping that its key is the resource name and its value is the observed composed resource like the example.

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | { metadata.name = "ocds" spec.ocds = option("params").ocds spec.user_kind = option("params").ocds["test-user"]?.Resource.Kind spec.user_metadata = option("params").ocds["test-user"]?.Resource.metadata spec.user_status = option("params").ocds["test-user"]?.Resource.status }

Composite Resource Connection Details

v1 Composite Resources (Legacy)

For legacy v1 XRs only, you can return desired composite resource connection details by including a KCL config that produces the special CompositeConnectionDetails resource:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputmetadata:
name: basicspec:
source: | details = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "CompositeConnectionDetails" data: { "connection-secret-key": "connection-secret-value" } } # Omit other composite logics. # Input the details resource into the return resource list. items = [ details # Omit other return resources. ]

Note

The value of the connection secret value must be base64 encoded. This is already the case if you are referencing a key from a managed resource's connectionDetails field. However, if you want to include a connection secret value from somewhere else, you will need to use the base64.encode function:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | import base64 # Omit other logic ocds = option("params").ocds details = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "CompositeConnectionDetails" data: { "server-endpoint" = base64.encode(ocds["my-server"].Resource.status.atProvider.endpoint) } }

v2 Composite Resources

For v2 composite resources, the CompositeConnectionDetails resource is not supported. Instead, you should compose an explicit Kubernetes Secret resource that aggregates connection details from the other composed resources.

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | import base64 oxr = option("params").oxr ocds = option("params").ocds secret = { apiVersion: "v1" kind: "Secret" metadata: { name: oxr.spec.writeConnectionSecretToRef?.name or "" annotations: { "krm.kcl.dev/composition-resource-name": "connection-secret" } } if "my-server" in ocds: data: { "server-endpoint" = base64.encode(ocds["my-server"].Resource.status.atProvider.endpoint) } else: data: {} } items = [secret]

For a detailed walkthrough and full example, please see the Connection Details Compositions guide in the Crossplane docs.

Readiness

To mark a desired composed resource as ready, use the krm.kcl.dev/ready annotation:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Omit other logic user = { apiVersion: "iam.aws.upbound.io/v1beta1" kind: "User" metadata.name = "test-user" metadata.annotations: { "krm.kcl.dev/ready": "True" } }

Required resources

By defining one or more "required resources", you can ask Crossplane to retrieve additional resources from the local cluster and make them available to your templates. See the docs for more information.

This feature only works with Crossplane v2. Crossplane v1 must use Extra Resources, described in the section below.

There are two ways to request required resources:

One, you can list the resources to retrieve in the requirements.requiredResources field of the pipeline step:

apiVersion: apiextensions.crossplane.io/v1kind: Compositionmetadata:
name: examplespec:
compositeTypeRef:
apiVersion: example.crossplane.io/v1beta1kind: XRmode: Pipelinepipeline:
- step: basicfunctionRef:
name: function-kclrequirements:
requiredResources:
- requirementName: fooapiVersion: example.com/v1beta1kind: FoomatchLabels:
foo: bar
- requirementName: barapiVersion: example.com/v1beta1kind: Barname: my-bar
- requirementName: bazapiVersion: example.m.com/v1beta1kind: Barname: my-barnamespace: my-baz-ns
- requirementName: quuxapiVersion: example.m.com/v1beta1kind: QuuxmatchLabels:
baz: quuxnamespace: my-quux-nsinput:
apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: "..."

Two, the composition can dynamically request resources by returning a special RequiredResources item:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Omit other logic details = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "RequiredResources" requirements = { foo = { apiVersion: "example.com/v1beta1", kind: "Foo", matchLabels: { "foo": "bar" } }, bar = { apiVersion: "example.com/v1beta1", kind: "Bar", name: "my-bar" }, baz = { apiVersion: "example.m.com/v1beta1", kind: "Bar", name: "my-bar" namespace: "my-baz-ns" }, quux = { apiVersion: "example.m.com/v1beta1", kind: "Quux", matchLabels: { "baz": "quux" } namespace: "my-quux-ns" } } } # Omit other composite logics. items = [ details # Omit other return resources. ]

Either way will result in Crossplane retrieving the requested resources and making them available with the following format:

foo:
- Resource:
apiVersion: example.com/v1beta1kind: Foometadata:
labels:
foo: bar# Omitted for brevity
- Resource:
apiVersion: example.com/v1beta1kind: Foometadata:
labels:
foo: bar# Omit for brevitybar:
- Resource:
apiVersion: example.com/v1beta1kind: Barmetadata:
name: my-bar# Omitted for brevity

You can access the retrieved resources in your code like this:

Note

Crossplane performs an additional reconciliation pass for dynamic required resources. Consequently, during the initial execution, these resources might not be present. It is essential to implement checks to handle this scenario.

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | er = option("params")?.requiredResources if er?.bar: name = er?.bar[0]?.Resource?.metadata?.name or "" # Omit other logic

Extra resources

Extra resources are Crossplane v1's mechanism for retrieving additional resources from the local cluster. It is deprecated in Crossplane v2.

Note

With ExtraResources, you can fetch cluster-scoped resources, but not namespaced resources such as claims. If you need to get a composite resource via its claim name you can use matchLabels with crossplane.io/claim-name: <claimname>. Namespace scoped resources can be queried with the matchNamespace field. Leaving the matchNamespace field empty or not defining it will query a cluster scoped resource.

Unlike required resources, there is no mechanism for requesting extra resources in the pipeline step definition. They can only be requested dynamically, by returning a special ExtraResources item:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Omit other logic details = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "ExtraResources" requirements = { foo = { apiVersion: "example.com/v1beta1", kind: "Foo", matchLabels: { "foo": "bar" } }, bar = { apiVersion: "example.com/v1beta1", kind: "Bar", matchName: "my-bar" }, baz = { apiVersion: "example.m.com/v1beta1", kind: "Bar", matchName: "my-bar" matchNamespace: "my-baz-ns" }, quux = { apiVersion: "example.m.com/v1beta1", kind: "Quux", matchLabels: { "baz": "quux" } matchNamespace: "my-quux-ns" } } } # Omit other composite logics. items = [ details # Omit other return resources. ]

You can retrieve the extra resources either via labels with matchLabels or via name with matchName: somename.

See the docs for more information.

This will result in Crossplane receiving the requested resources and making them available using the following format:

foo:
- Resource:
apiVersion: example.com/v1beta1kind: Foometadata:
labels:
foo: bar# Omitted for brevity
- Resource:
apiVersion: example.com/v1beta1kind: Foometadata:
labels:
foo: bar# Omit for brevitybar:
- Resource:
apiVersion: example.com/v1beta1kind: Barmetadata:
name: my-bar# Omitted for brevity

You can access the retrieved resources in your code like this:

Note

Crossplane performs an additional reconciliation pass for extra resources. Consequently, during the initial execution, these resources might not be present. It is essential to implement checks to handle this scenario.

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | er = option("params")?.extraResources if er?.bar: name = er?.bar[0]?.Resource?.metadata?.name or "" # Omit other logic

Patching the XR status field

You can read the XR, patch it with the status field and return the new patched XR in the item result like this

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputspec:
source: | # Read the XR dxr = option("params").dxr # Patch the XR with the status field dxr.status.dummy = "cool-status" items = [dxr] # Omit other resources

Settings conditions and events

Note

This feature requires Crossplane v1.17 or newer.

You can set conditions and events directly from KCL, either in the composite resource or both the composite and claim resources. To set one or more conditions, use the following approach:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputmetadata:
annotations:
"krm.kcl.dev/default_ready": "True"spec:
source: | oxr = option("params").oxr dxr = { **oxr } conditions = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "Conditions" conditions = [ { target: "CompositeAndClaim" force: False condition = { type: "DatabaseReady" status: "False" reason: "FailedToCreate" message: "Encountered an error creating the database" } } ] } items = [ conditions dxr ]

target: Specifies whether the condition should be present in the composite resource or both the composite and claim resources. Possible values are CompositeAndClaim and Composite
force: Forces the overwrite of existing conditions. If a condition with the same type already exists, it will not be overwritten by default. Setting force to True will overwrite the first condition.

You can also set events as follows:

apiVersion: krm.kcl.dev/v1alpha1kind: KCLInputmetadata:
annotations:
"krm.kcl.dev/default_ready": "True"spec:
source: | oxr = option("params").oxr dxr = { **oxr } events = { apiVersion: "meta.krm.kcl.dev/v1alpha1" kind: "Events" events = [ { target: "CompositeAndClaim" event = { type: "Warning" reason: "ResourceLimitExceeded" message: "The resource limit has been exceeded" } } ] } items = [ events dxr ]

Library

You can directly use KCL standard libraries such as regex.match, math.log.

Tutorial

See here to study more features such as conditions and loops in KCL.

Examples

More examples can be found here

Composition design guide

For a production-oriented structure (typed schemas, main.k/params.k/asserts.k/dxr.k , tests, and OCI tag rollout), see Proposed Crossplane KCL Composition Structure.

Debugging the KCL Function in Cluster

Logs are emitted to the Function's pod logs. Look for the Function pod in crossplane-system.

Levels

Info # default
Debug # run with --debug flag

Developing

# Run code generation - see input/generate.go
$ go generate ./...
# Run tests - see fn_test.go
$ go test ./...
# Build the function's runtime image - see Dockerfile
$ docker build . --tag=kcllang/crossplane-kcl
# Build a function package - see package/crossplane.yaml
$ crossplane xpkg build -f package --embed-runtime-image=kcllang/crossplane-kcl
# Push a function package to the registry
$ crossplane --verbose xpkg push -f package/*.xpkg xpkg.upbound.io/crossplane-contrib/function-kcl:latest

Name		Name	Last commit message	Last commit date
.devcontainer		.devcontainer
.github		.github
docs		docs
examples		examples
examples_kcl/eks		examples_kcl/eks
input		input
manifests		manifests
package		package
pkg/resource		pkg/resource
scripts		scripts
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
OWNERS.md		OWNERS.md
README.md		README.md
cspell.json		cspell.json
fn.go		fn.go
fn_test.go		fn_test.go
go.mod		go.mod
go.sum		go.sum
main.go		main.go
recycle.go		recycle.go
recycle_test.go		recycle_test.go
render.go		render.go
render_test.go		render_test.go
rendercache.go		rendercache.go
rendercache_test.go		rendercache_test.go
renovate.json		renovate.json

Navigation Menu

Uh oh!

Repository files navigation

Crossplane Composition Functions using KCL

Introduction

Install the KCL Function to Cluster

Using this Function

Source Support

Run a Composition with OCI tags

Use as a Base Image

Including modules in the base image

Read the Function Requests and Values through the `option` Function

Custom Parameters

Source Credentials

Run Config

Dependencies

Expect Output

Target Support

Extract Data from a Specific Composed Resource

Composite Resource Connection Details

v1 Composite Resources (Legacy)

v2 Composite Resources

Readiness

Required resources

Extra resources

Patching the XR status field

Settings conditions and events

Library

Tutorial

Examples

Composition design guide

Debugging the KCL Function in Cluster

Levels

Developing

About

Releases

Packages

Used by

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Crossplane Composition Functions using KCL

Introduction

Install the KCL Function to Cluster

Using this Function

Source Support

Run a Composition with OCI tags

Use as a Base Image

Including modules in the base image

Read the Function Requests and Values through the option Function

Custom Parameters

Source Credentials

Run Config

Dependencies

Expect Output

Target Support

Extract Data from a Specific Composed Resource

Composite Resource Connection Details

v1 Composite Resources (Legacy)

v2 Composite Resources

Readiness

Required resources

Extra resources

Patching the XR status field

Settings conditions and events

Library

Tutorial

Examples

Composition design guide

Debugging the KCL Function in Cluster

Levels

Developing

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Read the Function Requests and Values through the `option` Function