pilot-app — publish an existing API or CLI on the Pilot app store

pilot-app turns a declarative pilot.app.yaml into a complete, signed, publishable Pilot Protocol app-store adapter — the same shape as the hand-written reference app io.pilot.cosift, generated in seconds.

You bring an existing HTTP API (or a CLI); you describe its methods in one YAML file; pilot-app emits a buildable Go adapter, a manifest.json with the right grants, a Makefile that builds → sha256-pins → signs → packages, and the exact release + catalogue-PR steps. No hand-written Go, no IPC boilerplate.

Why an adapter at all

A Pilot app is a thin, stateless adapter + a signed manifest. The daemon fetches the bundle from the catalogue, re-verifies its signature and binary sha256 on every spawn, hands it a unix socket, and brokers JSON-in/JSON-out IPC calls. The heavy backend (your API) lives wherever it already lives; the adapter just forwards each IPC method to it. That forwarding layer is ~99% boilerplate — which is exactly what this generates.

Quickstart

go build -o pilot-app ./cmd/pilot-app
go install github.com/pilot-protocol/app-template/cmd/pilot-app@latest
pilot-app example > pilot.app.yaml # starter spec, fully annotated$EDITOR pilot.app.yaml # point it at your API, list your methods
pilot-app validate # catch spec errors early
pilot-app init -o ./my-app # scaffold the adapter projectcd my-app
make gen-key # one-time ed25519 publisher key (gitignored)
make package # build -> pin -> sign -> tarball
pilot-app verify io.pilot.<id>-<ver>.tar.gz # optional: run the gate locally# fork this repo, then publish through the single front door:
pilot-app submit -C . --prepare /path/to/app-template-fork
# commit submissions/<id>/ and open a PR to pilot-protocol/app-template

CI verifies the bundle and a maintainer reviews; on merge, automation releases it on pilot-protocol/catalog and opens the catalogue.json data entry on TeoSlayer/pilotprotocol. Then anyone can pilotctl appstore install <your.id>. You only ever touch one repo (app-template); see submissions/README.md and docs/APP-PUBLISHING-SPEC.md.

New to publishing? Start with docs/PUBLISHING.md — the field-level required-vs-optional reference, the two publish paths and why they're at parity, and a cli-app worked example.

Shipping a real app end to end? Follow docs/PUBLISHING-PLAYBOOK.md — the complete runbook across every backend (http/cli) and auth mode (byo/managed), native-binary delivery, testing, and the submission → catalogue → website-card steps, with edge cases and a pre-flight checklist.

Every submission carries a product demo. Author a product_demo in submission.json — a compact, example-first, skill-file-shaped usage guide shown at install, injected as a SKILL.md, and rendered on the website as the "Full usage demo". It drives correct first usage for autonomous agents. See docs/PRODUCT-DEMOS.md.

Every submission carries a next-steps graph. Author a next_steps in submission.json — the recommended next commands pilotctl prints after everyappstore call, on success and on failure (402 → how to top up, needs_signup → the gateway, bad params → a corrected call). The demo drives install→first-call; the graph drives first-call→actual usage. See docs/NEXT-STEPS-GRAPHS.md.

Two ways to publish — same required fields, same result

You can publish either way; the inputs and the output are identical:

1. The website form at pilotprotocol.network/publish, which POSTs the rich Submission JSON to the publish-api (cmd/publish-server). The server scaffolds + cross-compiles + signs the adapter for you.
2. A PR to this repo with a submissions/<id>/ produced by pilot-app submit --prepare — pilot-app scaffolds + builds the same bundle on your machine, and you commit the signed bundle + a small pointer submission.json.

Both run the same scaffold pipeline and the same validation; the only difference is where the build runs (our server vs. your laptop). Field-by-field parity is in docs/PUBLISHING.md.

Two rules that trip people up:

• The adapter is scaffolded by the pipeline — never hand-built. Do not write your own adapter and submit a pre-built binary as the bundle; the pipeline generates the adapter Go, manifest, and binaries from your spec.
• Binaries must be the full per-platform set or a true universal binary — never single-platform. Every app cross-compiles to darwin × linux × arm64 × amd64 automatically; a lone linux/amd64 bundle is a build-host accident that refuses to spawn elsewhere.

The spec (pilot.app.yaml)

id: io.pilot.weatherapp_version: 0.1.0description: "Current conditions and forecasts."backend:
type: httpbase_url: https://weather.example.com # baked in as the default; no config neededmethods:
- name: weather.currentsummary: "Current conditions for a lat/lon."duration: fast # fast|med|slow -> per-call timeout + help classhttp: {verb: GET, path: /current} # GET: payload -> query stringparams: {lat: "string (required)", lon: "string (required)"}
- name: weather.reportsummary: "Synthesized briefing."duration: slowhttp: {verb: POST, path: /report} # POST: payload -> JSON body

pilot-app example prints the full annotated version, including the cli backend form. Run pilot-app validate for fast feedback.

What gets generated

my-app/
cmd/<binary>/main.go # six lifecycle flags, socket serve loop, dispatcher,
# per-method handlers, config resolution, <ns>.help
internal/backend/client.go # tuned HTTP client (http backend)
internal/backend/exec.go # subprocess runner (cli backend)
manifest.json # id, exposes (every method + <ns>.help), grants, store
Makefile # gen-key / bundle / package / verify / publish-help
README.md go.mod .gitignore

The generated adapter has zero dependencies beyond the pinned app-store/pkg/ipc, ships pointing at production (no config step for users), and auto-exposes a <namespace>.help discovery method describing every method's params, kind, and latency class.

Backends

• http — works on the platform today. Maps each method to a backend HTTP endpoint (GET → query string, POST → JSON body). This is what the reference app io.pilot.cosift does, by hand; pilot-app generates the equivalent.
• cli — generates a working subprocess adapter (curated routes and/or a passthrough that fronts the whole tool). Provide backend.command + methods + the CLI binary as assets[]; the pipeline scaffolds and cross-compiles the adapter. Installing it through the catalogue needs a proc.exec capability the platform is rolling out — see docs/CLI-ADAPTER.md and the worked example in docs/PUBLISHING.md. Until proc.exec lands on your hosts, a CLI can also ship today fronted by a small HTTP shim published as an http adapter.

Authentication: BYO key vs. managed key

backend.auth chooses how the adapter authenticates to the API:

• byo (default) — each user supplies their own API key at install via ${TOKEN} headers (env or $APP/secrets.json). The key is never baked into the bundle. Use this when each user has their own account with the partner.
• managed — Pilot holds one master key and meters it per user. The generated adapter is keyless: it points at the Pilot broker, signs each request with the per-app identity the daemon provisions, and the broker verifies the caller, enforces a per-user quota, injects the master key, and forwards to the partner. Use this when the partner gives Pilot one shared key (e.g. an enrichment or data partner).

backend:
type: httpbase_url: https://api.example.com # registered with the broker, not shipped to usersauth: managed

Publishing is identical either way — same pilot.app.yaml, same one-repo flow. The full design (security model + an ELI5) is in docs/MANAGED-KEY.md. The broker lives in this repo (cmd/broker, internal/broker); run the prod-like stack from deploy/docker.

Repository layout

cmd/pilot-app the scaffolder CLI (init / validate / verify / submit)
cmd/publish-server submission API + admin dashboard (the VM service)
cmd/broker the managed-key gateway (holds master keys, meters per user)
cmd/broker-sign dev/ops helper: sign a broker request as a caller
cmd/ipc-call dev/ops helper: call a running adapter over its IPC socket
internal/scaffold pilot.app.yaml -> adapter project (templates/)
internal/publish submission, build, sign, case store, broker registration
internal/broker identity verify, registry, auth inject, store, breaker
internal/catalogue review-gate checks (SPEC §7.1)
deploy/ GCE startup script (publish + broker units) + docker/ stack
docs/ publishing spec, managed-key design, adapter archetypes
scripts/ e2e-broker.sh, e2e-managed.sh, install-git-hooks.sh

Architecture

Two flows, one repo. Publish (build + sign + register an app) and runtime (a user calls a managed app). The website form is the only piece outside this repo; everything else — scaffold, build, sign, broker — lives here.

 PUBLISH (the admin board triggers everything)
─────────────────────────────────────────────────────────────────────────────
developer website form publish-server (cmd/publish-server)
┌────────┐ POST ┌──────────┐ /api/submit ┌───────────────────────────────┐
│ author │ ──────► │ form │ ───────────► │ Validate → scaffold.Generate │
└────────┘ └──────────┘ │ → go build → sign → bundle │
│ (case: pending) │
admin clicks Approve │ │
───────────────────────► │ /admin/approve: │
│ 1. register managed app ─────┼──► apps.json
│ with the broker (route) │ (BROKER_REGISTRY)
│ 2. TriggerPublish (catalog) ──┼──► pilot-protocol/catalog
└───────────────────────────────┘ + catalogue.json PR
→ pilotctl install
RUNTIME (managed app: one master key, metered per user)
─────────────────────────────────────────────────────────────────────────────
user's pilot daemon broker (cmd/broker, internal/broker) partner
┌───────────────────┐ ┌───────────────────────────────────┐ ┌─────────┐
│ keyless adapter │ sign │ 1 verify caller (ed25519) → 401 │ key │ API │
│ (generated) │ ───► │ 2 known app? → 404 │ ───► │ │
│ signs with the │ HTTP │ 3 method allow-listed? → 403 │ ◄─── │ │
│ --identity key │ ◄─── │ 4 breaker + per-caller quota → 429 │ resp └─────────┘
└───────────────────┘ JSON │ 5 inject master key, forward, METER│
▲ daemon brokers IPC └───────────────────────────────────┘
│ from the calling app │ durable per-(app,caller) usage → /gw/usage
┌────┴─────┐
│ agent │ pilotctl appstore call io.pilot.<app> <method> '{...}'
└──────────┘

BYO-key apps skip the broker entirely: the adapter calls the partner directly with the user's own ${TOKEN}. Same scaffold, same publish flow — only backend.auth differs. Deep dive: docs/MANAGED-KEY.md.

Verified end to end

The generated adapter has been installed against the real pilot daemon (via a file:// catalogue), auto-spawned, and called against the live cosift backend — help, health, and search all return correct results. The generated cli archetype produces valid, compiling Go (covered by tests).

The managed-key path is covered by a real-process end-to-end (scripts/e2e-managed.sh): it drives the admin board to build + register a managed app, runs the actual generated adapter binary (signing with a daemon-format identity.json), and asserts the call flows through the broker to the partner, is metered, and is rate-limited — plus a multi-user broker e2e (scripts/e2e-broker.sh).

Testing

make test# unit tests
make race # race detector + coverage profile (writes cover.out)
make cover # total statement coverage (one line)
make e2e # real-process broker end-to-end
make e2e-managed # full submit → build → register → broker → meter e2e

The badge above reports total statement coverage across the internal/ core packages. CI hard-gates the security-critical and high-traffic packages so coverage can't silently regress:

package	role	floor
internal/broker	holds master keys, verifies caller identity	78%
internal/catalogue	the objective review gate (VerifyEntry)	80%
internal/publish	build → sign → submit pipeline	70%
internal/scaffold	pilot.app.yaml → adapter generator	68%

Reproduce the number behind the badge with make cover.