Task (WALLIX fork)

Fork of go-task/task (v3.49.1) with opinionated changes focused on build-system reliability: deterministic fingerprinting, distributed caching and locking, and setup tasks.

Source: github.com/wallix/task

Changes from upstream

Removed

• Remote taskfiles -- http:// and git:// includes are no longer supported. Related CLI flags (--download, --offline, --insecure, --timeout, --clear-cache, --trusted-hosts, --expiry, --remote-cache-dir, --cacert, --cert, --cert-key) have been removed.
• Timestamp fingerprinting -- only checksum-based fingerprinting remains. The method field on tasks is removed.
• none fingerprint method -- tasks either use checksum fingerprinting or have no sources.

Added

Setup tasks

A new setup field runs tasks unconditionally and sequentially before deps and fingerprint checks. Unlike deps, setup tasks always run regardless of whether the parent is up-to-date, and they do not affect the parent's fingerprint. Use run: once to avoid re-executing shared setup tasks.

tasks:
enforce-version:
run: oncecmds:
- date +%Y-%m-%d > version.txtbuild:
setup:
- enforce-versionsources:
- version.txt
- src/**/*.gogenerates:
- bin/appcmds:
- go build -ldflags "-X main.buildDate=$(cat version.txt)" -o bin/app .

Fingerprint-based generates

For large generated directories where hashing every file is expensive, a generates entry can specify a fingerprint file -- a single representative file used for checksum-based up-to-date detection instead of hashing every file matched by the glob. The full glob is still used for cache operations (save/restore), so all files are archived correctly.

Four YAML forms are supported in sources and generates:

generates:
# Scalar: simple glob pattern (hashes all matched files)
- "build/**/*"# Exclude: negated pattern
- exclude: "build/tmp/**"# Glob + fingerprint: the glob defines the full set of files for caching,# while fingerprint names a single file for up-to-date checks.
- glob: "node_modules/**/*"fingerprint: "node_modules/.yarn-state.yml"# From: inherit entries from related tasks (see "Inherited sources/generates")
- from: deps

Example: yarn install with fingerprint

tasks:
install:
sources:
- package.json
- yarn.lockgenerates:
- glob: "node_modules/**/*"fingerprint: "node_modules/.yarn-state.yml"cmds:
- yarn install --immutable

Here node_modules/ may contain thousands of files, but only .yarn-state.yml is hashed for staleness checks. When caching is enabled, the full node_modules/**/* glob (plus the fingerprint dotfile) is archived.

Example: mixed generates with caching

tasks:
build:
sources:
- src/**/*.ts
- package.jsongenerates:
- "dist/**/*"
- glob: "node_modules/**/*"fingerprint: "node_modules/.yarn-state.yml"
- exclude: "dist/tmp/**"cache:
url: 'file:///tmp/cache/build-{{.CHECKSUM}}.zip'cmds:
- npm run build

Inherited sources/generates (from: deps and from: cmds)

Wrapper tasks can inherit sources and generates from their dependencies or cmd task-calls using the from: directive. This avoids duplicating glob patterns across tasks and ensures cache keys reflect the full input/output set. Entries are deduplicated automatically.

from: deps — copies entries from all direct dependencies:

tasks:
all:
sources:
- from: depsgenerates:
- from: depscache:
url: 'file:///tmp/cache/all-{{.CHECKSUM}}.zip'deps:
- build-a
- build-bbuild-a:
sources: [src/a/**/*.go]generates: [bin/a]cmds: [go build -o bin/a ./cmd/a]build-b:
sources: [src/b/**/*.go]generates: [bin/b]cmds: [go build -o bin/b ./cmd/b]

from: cmds — copies entries from all cmd task-calls:

tasks:
build:
sources:
- from: cmdsgenerates:
- from: cmdscmds:
- task: compile
- task: linkcompile:
sources: [src/**/*.c]generates: [build/**/*.o]cmds: [make compile]link:
sources: [build/**/*.o]generates: [bin/app]cmds: [make link]

Literal globs and from: entries can be mixed freely:

sources:
- config.yml # own source
- from: deps # plus all dep sources

Per-task cache block (file://, redis:// and oci:// backends)

Cache generated files so that subsequent runs (or other machines) can skip execution entirely. The url and lock fields are Go templates with access to all task variables plus {{.CHECKSUM}} (SHA256 of sources, commands, and generates).

tasks:
build:
sources:
- src/**/*.gogenerates:
- bin/appcache:
enabled: '{{ne .REDIS_URL ""}}'# optional, template boolurl: 'file:///tmp/cache/build-{{.CHECKSUM}}.zip'lock: 'redis://{{.REDIS_URL}}/lock:build-{{.CHECKSUM}}'cmds:
- go build -o bin/app .

OCI registry backend. With an oci:// URL the entry is stored as an OCI artifact: files are cut into content-defined chunks (FastCDC), compressed with zstd, and pushed as individual blobs that the registry deduplicates by digest. Saving a slightly changed node_modules or VM image only uploads the new chunks; pulls go through a local chunk store so repeated restores are incremental. Entries expire through the registry's retention policy (no TTL).

cache:
url: 'oci://registry.example.com/task-cache:{{urlsafe .TASK}}-{{.CHECKSUM}}?ca=/etc/ssl/registry-ca.crt'

The URL shape is oci://[user:password@]host/repo:tag[?ca=<file>][&cas=<dir>][&plainhttp=1] (the tag carries the cache key and is limited to [A-Za-z0-9._-]). Credentials and trust can also come from the environment — TASK_CACHE_OCI_USER, TASK_CACHE_OCI_PASSWORD, TASK_CACHE_OCI_CA and TASK_CACHE_OCI_CAS_DIR — keeping secrets out of the Taskfile.

See docs/cache-server.md for setting up the server side (a Harbor registry for the cache entries plus a Redis for the distributed locks).

Filesystem-based locking

Tasks with sources and generates automatically acquire a POSIX advisory file lock (stored in .task/). The lock key is taskname:sourcehash, so different source states don't contend on the same lock.

Redis-based distributed locking

When cache.lock evaluates to a redis:// URL, locking is distributed across machines using Redis SET NX EX with TTL-based heartbeat renewal.

urlsafe template function

{{urlsafe .TASK}} percent-encodes a string for use in URLs, replacing special characters like colons from namespaced task names. Useful in cache URLs:

cache:
url: echo "redis://$REDIS_URL/cache:{{urlsafe .TASK}}/$TASK_CACHE_HASH"

--status flag

Show fingerprint status of tasks without running them:

task --status build # human-readable
task --status --json build # machine-readable

--export-cache and --import-cache

Portable fingerprint state for CI/CD pipelines:

# On build machine
task --export-cache state.zip build test# On CI machine
task --import-cache state.zip

Exports checksum state and generated files for up-to-date tasks as a ZIP archive.

Changed

• --force is now gentle by default -- --force only forces the directly called task; dependent tasks still check their status. Use --force-all to force everything (previously the default --force behavior). The TASK_X_GENTLE_FORCE experiment flag has been removed.

Improved

• Richer fingerprints -- checksums now include serialized commands and variable data, not just file contents.
• Separate staleness reporting -- sources and generates staleness is tracked and reported independently.

Execution pipeline

setup tasks (unconditional, sequential)
-> acquire lock (file or redis)
-> run deps (parallel)
-> check fingerprint (sources + generates, including from: resolution)
-> try restore from cache (file://, redis:// or oci://)
-> if miss: execute task, then save to cache
-> release lock