feat(mcp): add ToolHive operator and mcp-gateway workloads

[xnoto]

Summary

Adds a Kubernetes MCP gateway (Stacklok ToolHive) to the cluster and wires the opencode web service to it, giving every agentic session one universal, aggregated MCP backend.

operators/toolhive/ — ToolHive operator 0.44.0 via two ArgoCD OCI-chart Applications (toolhive-operator-crds wave 0, toolhive-operator wave 1). Chart resource defaults are emptied per the single-node policy; update check / usage metrics disabled.

workloads/mcp-gateway/ (new mcp namespace):

• MCPGroup/gateway + VirtualMCPServer/gateway → aggregated endpoint http://vmcp-gateway.mcp.svc:4483/mcp (service name/port verified against the 0.44.0 operator source)
• MCPServer/kubernetes — containers/kubernetes-mcp-server:v0.0.66 (digest-pinned) with --read-only and a dedicated ClusterRole that excludes Secrets (KSOPS-decrypted secrets must not become readable through the gateway)
• MCPRemoteProxy/context7 — public, unauthenticated
• MCPRemoteProxy/github — api.githubcopilot.com with MCPExternalAuthConfig headerInjection; the PAT is injected server-side from the KSOPS secret mcp-github (clients never hold it)
• VirtualMCPServerincomingAuth: anonymous is deliberate: ClusterIP-only, flannel doesn't enforce NetworkPolicy, and edge auth is Cloudflare Access (see paired tfroot-cloudflare PR)

opencode — ConfigMap gains an mcp block pointing at vmcp-gateway. Remote-only by design: the opencode image (Alpine + static Bun binary, no python/uv/node) cannot run stdio MCPs, and its pod hardening forbids runtime installs.

wait-for-crds — added the five ToolHive CRDs to the PreSync gate.

Authentication model

1. Edge (enforcement): Cloudflare Access on mcp.makeitwork.cloud — service-token (non_identity) for agent clients + admins GitHub SSO for debug. Unauthenticated traffic dies at the edge.
2. In-cluster: anonymous on ClusterIP (see above).
3. Credentials: GitHub PAT lives only as a KSOPS secret; k8s backend is RBAC-read-only + --read-only.

Required sequence — read before merging

1. First: merge + apply the paired tfroot-cloudflare PR (creates the mcp CNAME and the Access applications). Do not merge this PR's TunnelBinding before that apply — the TunnelBinding is deliberately excluded from this PR to avoid the documented DNS race (81053); it lands as a small follow-up PR after the CNAME exists.
2. Merge this PR. ArgoCD syncs: CRDs → operator → group/backends → vMCP.
3. Set the real GitHub PAT:sops workloads/mcp-gateway/github-secret.yaml (value must be Bearer <pat>), commit, push. Until then only the github proxy is degraded; context7 + kubernetes work.
4. Follow-up PR: workloads/mcp-gateway/tunnel-binding.yaml → mcp.makeitwork.cloud → http://vmcp-gateway.mcp.svc:4483.
5. Verify: kubectl --context k3s get mcpservers,virtualmcpservers,mcpremoteproxies -n mcp, then opencode mcp debug mcp-gateway from a session.

Local client config (after steps 1–4)

"mcp-gateway": {
"type": "remote",
"url": "https://mcp.makeitwork.cloud/mcp",
"enabled": true,
"oauth": false,
"headers": {
"CF-Access-Client-Id": "{env:CF_ACCESS_CLIENT_ID}",
"CF-Access-Client-Secret": "{env:CF_ACCESS_CLIENT_SECRET}"
}
}

Validation

• kustomize build operators/toolhive ✅ (2 Applications), kustomize build workloads/apps ✅ (10 objects)
• workloads/mcp-gateway local build blocked by KSOPS exec plugin — pre-existing repo limitation, identical on main (verified via stash); structure linted instead
• pre-commit run on all changed files ✅ (check-yaml, KubeLinter, detect-private-key, secrets)
• Secret scan of the diff ✅ (placeholder PAT is SOPS-encrypted)

Not done (explicit)

• No TunnelBinding (step 4 above)
• No tofu plan/apply — paired tfroot PR, applies by repo owner
• vMCP OIDC against Dex — documented upgrade path, deferred

[xnoto]

Paired tfroot-cloudflare PR: makeitworkcloud/tfroot-cloudflare#26 (must merge + apply first — see "Required sequence" above).