diff --git a/rfc/0014-release-stability/README.md b/rfc/0014-release-stability/README.md new file mode 100644 index 0000000000..3df68ba5fc --- /dev/null +++ b/rfc/0014-release-stability/README.md @@ -0,0 +1,225 @@ +--- +authors: + - "@drew" +state: review +links: + - https://github.com/NVIDIA/OpenShell/pull/2148 + - https://github.com/NVIDIA/OpenShell/pull/2695 +--- + +# RFC 0014 - Alpha Exit Criteria and Stable Release Policy + +## Summary + +The goal of this RFC is to define OpenShell's exit from alpha and establish a +predictable release cycle for production users and ecosystem developers. + +We propose + +- Development releases for every commit to `main`, nightly pre-releases, and + qualified stable releases every Tuesday. +- Stable and Experimental API maturity, compatibility, and versioning + rules, and maintenance for the latest and N-1 minor release lines. +- A release qualification pipeline covering conformance, upgrades, + API changes, and security reviews across the supported release matrix. + +## Motivation + +The goal is to exit alpha without slowing OpenShell's development. Releases +should remain frequent and automated, and Experimental APIs should be able to +evolve quickly enough to keep pace with the ecosystem. At the same time, users +need Stable interfaces they can confidently build on. + +Starting with `0.1.0`, OpenShell provides both: a defined compatibility contract +for Stable interfaces and room to evolve Experimental APIs and features. +Releases are suitable for production use within the published support matrix +only after passing conformance, upgrade, compatibility, artifact, and security +checks. + +## Proposal + +### Release cadence + +Starting with `0.1.0`, OpenShell publishes stable tagged releases intended for production use within +the published support matrix. Stable releases occur every Tuesday and increment +the patch version by default, for example `0.1.1` followed by `0.1.2`. A release +may instead increment the minor version when it introduces notable new features +or documented breaking changes. A stable tag is published only when there are +changes and every blocking qualification suite passes. + +OpenShell publishes a development release for every commit to `main`. Each +development release identifies its source commit and artifact manifest, and the +floating `dev` alias points to the newest one. Development releases enable all +development compilation flags and features. They give feature authors, early +adopters, and integration owners a way to consume upcoming OpenShell changes +between stable releases. This also allows development of a large feature to +span multiple weeks behind a compile-time feature flag without including the +unfinished feature in the stable release track. Development releases have +passed normal CI, but have not passed release qualification and are not +intended for production use. + +OpenShell builds a pre-release nightly for the next expected stable release +when `main` has changed and normal CI passes. After `0.1.1`, pre-releases are +numbered `0.1.2-pre.1`, `0.1.2-pre.2`, and so on. Features may land between +pre-releases. The first pre-release locks the version for that weekly release +train. Once a patch train starts, breaking changes that require a minor release +are staged for the following week; the train never switches from patch to minor. + +Pre-releases give the automated release qualification and QA systems an +immutable artifact set to evaluate. They are also available to maintainers and +integration owners who need to validate the prospective stable release. A +pre-release uses the same release feature set as stable and may still fail +qualification; it is not intended for production use. Building one nightly +allows failures to be fixed and reevaluated before Tuesday. + +| Release | Intended users | Contents and expectations | +| --- | --- | --- | +| Development | Feature authors, early adopters, and integration owners testing upcoming functionality | Published for every commit to `main`; enables development features; passes normal CI but not release qualification | +| Pre-release | Automated qualification, maintainers, and integration owners validating the next stable release | Built nightly from eligible `main`; uses the stable feature set; immutable but not yet qualified for production | +| Stable | Production users and downstream integrations that require the published compatibility and support contract | Published Tuesday from a pre-release that passed every blocking qualification suite | + +### Project version and compatibility contract + +Version `0.1.0` begins OpenShell's supported compatibility contract. For +the `0.x` series: + +- Patch releases may contain bug fixes and additive, backward-compatible + functionality. They do not intentionally break a stable interface. +- Minor releases may represent notable new features even when they remain + backward-compatible. They may also contain documented breaking changes to + stable interfaces, for example `0.1.x` to `0.2.0`. + +The [release version selection supplement](release-version-selection.md) +defines how Conventional Commits select the next patch or minor release +pre-release. + +### API and feature maturity + +Public APIs are Stable by default. APIs that are expected to change frequently +must be explicitly designated Experimental. Protobuf packages encode the +designation in their version name. SDKs use language-appropriate package, +module, namespace, or symbol naming to expose the same designation. + +| Maturity | Protobuf naming | SDK naming | Compatibility | +| --- | --- | --- | --- | +| Stable | `v1` or `v2` | Default Stable package, module, namespace, or symbol | Backward-compatible across patch releases; may change in a minor release with notice and migration guidance | +| Experimental | `v1experimental` | Language-specific Experimental package, module, namespace, or symbol | May change or be removed in a patch release without notice | + +Every API and feature is Stable or Experimental, regardless of whether it +appears in stable, pre-release, or development artifacts. Each SDK must +document its language-specific naming convention, and Experimental interfaces +must not appear to be Stable. + +Experimental APIs are intended for rapid iteration and may change in place +without a compatibility guarantee. Graduation adds a Stable `v1` package +instead of renaming the Experimental package in place. + +Unreleased features use named compile-time flags such as `unstable-`, +collected under a `dev` compilation flag. Development releases enable them; stable +releases and pre-releases exclude them from service binaries, the CLI, +configuration, and documentation. For now, SDK distributions may include their +generated types and client methods when the SDK naming convention communicates +their maturity. + +### Breaking changes and API versioning + +A breaking change to a Stable API or other stable OpenShell surface requires a +minor release with notice and migration guidance. This applies to Protobuf, +SDK, CLI, configuration, policy, Helm, and state interfaces. A breaking +Experimental API change may ship in a patch release without notice when it does +not break a Stable interface. + +Breaking-change detection runs during code review and again during pre-release +qualification. Stable protobuf packages use Buf's `FILE` rules +against the latest stable baseline and any additional supported baseline needed +for the N-1 maintenance promise. Language-specific API checks or agent review +skills cover the public SDKs. + +The following examples illustrate how these rules affect a pre-release: + +| Example | Concrete change | Release treatment | +| --- | --- | --- | +| Breaking Stable Protobuf contract | `string policy = 7` to `PolicyReference policy = 7` | Blocks a patch release and requires a minor OpenShell release. | +| Breaking Experimental Python SDK method | `create_sandbox(timeout=30)` to `create_sandbox(deadline=...)` | May ship in a patch release without notice. | +| Breaking stable policy document | `endpoints:` to `destinations:` | Blocks a patch release unless both fields remain supported. | +| Breaking stable CLI contract | `--policy policy.yaml` to `--policy-file policy.yaml` | Requires retaining the old flag as an alias or shipping a minor release. | + +### Pre-release qualification and stable publication + +The release system separates pre-release creation, qualification, and stable +publication: + +```mermaid +flowchart LR + A["Commit to main"] --> B["Dev release
dev features enabled"] + A --> C["Normal CI passes"] + C --> D["Tag and build nightly pre-release
release feature set"] + D --> E["Pre-release qualification"] + E -->|"pass"| F["Eligible Tuesday pre-release"] + E -->|"fail"| G["No stable release"] + F --> H["Publish stable artifacts"] + H --> I["Final artifact checks"] + I -->|"pass"| J["Create tag and publish"] + I -->|"fail"| G +``` + +Every pre-release is tagged before qualification and produces a manifest with +its version, source commit, build inputs, artifact digests, SBOM, provenance, +and qualification results. Qualification exercises those artifacts rather than +a substitute source build. Pre-releases are stored in artifact storage instead +of published as GitHub Releases, following the +[Bazel rolling release model](https://bazel.build/release/rolling). Any change +creates a new pre-release. + +Release qualification consists of four suites defined in the +[release qualification supplement](release-qualification.md): + +- **Conformance** runs across supported driver and gateway configurations. It + verifies core sandbox behavior, policy enforcement, and extension contracts. +- **Upgrade** runs once per supported installation package and verifies its + upgrade path, state migration, post-upgrade health, and rollback where + promised. +- **Breaking API change review** runs once per pre-release and compares stable + protobuf and SDK interfaces with every applicable compatibility baseline. +- **Security review** runs once per pre-release and verifies security scan + results, reviews changes to security-sensitive boundaries, and confirms that + every finding has the required disposition. + +The initial release targets are defined in the [build matrix](build-matrix.md), +and blocking coverage is defined in the +[release qualification supplement](release-qualification.md). + +The weekly release tags the newest eligible pre-release commit with the next +stable version and publishes the stable artifacts. + +### Maintenance and backports + +OpenShell maintains two release lines: the latest minor and N-1. Support +applies to the newest patch on each line. Users on an older patch update to the +new maintenance patch rather than receiving a separate fix for every historical +patch. + +Maintenance releases contain critical reliability fixes and security fixes. +They do not backport features. A fix is developed on the appropriate primary +branch and backported to a `release/.` branch when the older line +is affected. Each backport passes the compatibility, regression, packaging, +and supported upgrade qualification appropriate to that line. + +For example, if `0.3.1` is current and a vulnerability also affects the 0.2 +line, OpenShell publishes the next available `0.2.x` patch from the maintained +0.2 branch. A security release may occur immediately rather than waiting for +the next Tuesday release. + +## Implementation plan + +1. **Keep per-commit dev releases and build 0.1.0 pre-releases nightly.** Build + every commit to `main` with all development features, and publish sequential + `0.1.0-pre.N` artifacts with the release feature set leading to 0.1.0. +2. **Make the necessary breaking API changes.** Use the pre-0.1.0 window to + finalize Stable interfaces, move evolving APIs to Experimental packages, + and establish the compatibility baseline. +3. **Build qualification tests and release machinery.** Automate compatibility + detection, conformance, upgrade, breaking API change review, security + qualification, artifact validation, and release publication gates. +4. **Release 0.1.0.** Select a qualified pre-release, publish the stable + artifacts and support guidance, and begin the weekly release cadence. diff --git a/rfc/0014-release-stability/build-matrix.md b/rfc/0014-release-stability/build-matrix.md new file mode 100644 index 0000000000..43ca9f768e --- /dev/null +++ b/rfc/0014-release-stability/build-matrix.md @@ -0,0 +1,70 @@ +# RFC 0014 Supplement - Build Matrix + +This supplement defines the build targets for the release system described by +RFC 0014. The targets follow the structure of the +[proposed support matrix](https://github.com/NVIDIA/OpenShell/blob/d575cf85f8c520774cc3e307ec1bcdabf57cd682/docs/reference/support-matrix.mdx) +and describe the intended final state. + +These tables enumerate build and capability targets. They do not imply that +every cross-product of platform, architecture, installation method, driver, +topology, and capability is supported. The +[release qualification supplement](release-qualification.md) defines the +minimal blocking release gate. It contains one conformance workflow per compute +driver, one Kubernetes conformance workflow per supported gateway topology, and +one upgrade workflow per installation package, plus one breaking API change +review and one security review per pre-release. Other supported dimensions are +exercised as subcases or separate CI controls rather than as a Cartesian product +of release jobs. + +## Platforms + +| Platform | Installation methods | Requirements | +| --- | --- | --- | +| macOS (Apple Silicon) | Homebrew | macOS 13.3 or later | +| Linux (x86_64, arm64) | APT/DEB, RPM, Snap | glibc 2.28 or later | +| Windows (x86_64, arm64) | MSI, WinGet | Documented minimum Windows version and MSVC toolchain | + +Package installers include the CLI, TUI, gateway, and supported local drivers. +Standalone CLI and TUI binaries are also published for remote gateway access. + +## Compute drivers + +| Driver | Supported hosts | Minimum version | Requirements | +| --- | --- | --- | --- | +| Docker | macOS, Linux, Windows | 28.0.4 or later | Docker Engine or Docker Desktop | +| Podman | macOS, Linux, Windows | 5.x | Podman socket, rootless networking, and cgroups v2 | +| MicroVM | macOS, Linux | macOS 13.3 or later; KVM on Linux | Host virtualization, Hypervisor.framework on macOS, and KVM on Linux | +| Kubernetes | Kubernetes clusters | 1.29 or later | Helm 3.x and a compatible Agent Sandbox controller and CRDs | + +## GPU support + +| Compute driver | Supported environment | Device interface | Requirements and limits | +| --- | --- | --- | --- | +| Docker | Linux; Windows through WSL2 | NVIDIA CDI | CDI-enabled runtime with visible NVIDIA devices; default and counted GPU requests | +| Podman | Linux; Windows through WSL2 | NVIDIA CDI | Visible NVIDIA CDI devices; default and counted GPU requests | +| MicroVM | Linux | QEMU/VFIO PCI passthrough | IOMMU, VFIO, root privileges, compatible sandbox image, and one GPU per sandbox | +| Kubernetes | Linux GPU nodes | `nvidia.com/gpu` extended resources | NVIDIA GPU Device Plugin and a compatible sandbox image | + +Release qualification accepts current CUDA drivers and validates against the +Tesla Recommended Driver branches. + +## Kubernetes + +| Component | Supported version | Notes | +| --- | --- | --- | +| Kubernetes | 1.29 or later | Required for Helm deployments and sandbox scheduling | +| Helm | 3.x | Required to install and upgrade the OpenShell chart | +| Agent Sandbox controller and CRDs | Compatible release | Required before installing the OpenShell chart | +| User namespaces | 1.33 or later | Optional; enables `hostUsers: false` for UID remapping | + +GKE Standard, GKE Autopilot, and OpenShift 4.x are supported. OpenShift uses +the OpenShift-specific SCC binding and chart configuration. + +## SDKs + +| SDK | Minimum version | +| --- | --- | +| Python | 3.12 | +| TypeScript | 5.7 | +| Rust | 1.90 | +| Go | 1.24 | diff --git a/rfc/0014-release-stability/release-qualification.md b/rfc/0014-release-stability/release-qualification.md new file mode 100644 index 0000000000..243f6468df --- /dev/null +++ b/rfc/0014-release-stability/release-qualification.md @@ -0,0 +1,115 @@ +# RFC 0014 Supplement - Release Qualification + +A pre-release is eligible for stable publication only after its artifacts pass +every required [conformance test](#conformance-tests) and +[upgrade test](#upgrade-tests), its +[breaking API change review](#breaking-api-change-review), and its +[security review](#security-review). + +## Conformance tests + +Conformance tests run for different OpenShell driver, gateway, and host +configurations. Each workflow installs the pre-release in one representative +environment and verifies the supported OpenShell behavior for that +configuration. + +For each driver, the suite: + +1. Installs the pre-release artifacts and starts the gateway and selected runtime. +2. Verifies gateway health, version reporting, TLS, authentication, and + authorization. +3. Creates a sandbox, connects to it, executes commands, and exercises stop, + restart, and restore behavior where supported. +4. Exercises filesystem, process, network, credential, and inference policy + enforcement, including expected denial behavior. +5. Verifies the configured compute driver, credential driver, interceptor, and + middleware contracts, including capability discovery, validation, errors, + restart reconciliation, and cleanup. +6. Deletes the sandbox and confirms that runtime, network, credential, and + persisted resources are removed. + +Conformance passes only when every driver workflow completes without an +undeclared skip or retry-dependent result. A configuration may opt out of a +specific conformance test when it does not support the capability under test. +The opt-out and unsupported capability must be declared in the test +configuration before qualification runs; any other skip fails qualification. +A workflow may exercise relevant GPU, authentication, credential-driver, +interceptor, middleware, and client/server skew variants as subcases without +creating separate workflows. Unsupported capabilities must return documented +errors rather than fail silently. + +Each workflow uses one representative environment for the configuration under test. It +may run multiple capability subcases inside that environment without creating +additional workflows. + +### Configurations to test + +The goal is to provide test coverage for each documented driver, topology, and environment configuration. + +| ID | Compute driver | Representative environment | Gateway configuration | +| --- | --- | --- | --- | +| C01 | Docker | Ubuntu x86_64 with Docker 28.0.4 | Local gateway; TLS and sandbox mTLS; credentials; CDI GPU when stable | +| C02 | Podman | Fedora x86_64 with Podman 5.x rootless | Local gateway; TLS and sandbox mTLS; credentials; CDI GPU when stable | +| C03 | MicroVM | Linux x86_64 with KVM and IOMMU | Local gateway; libkrun CPU; TLS and sandbox mTLS; QEMU/VFIO GPU when stable | +| C04 | Kubernetes | Kubernetes 1.29 | Sidecar, three replicas; external PostgreSQL; Kubernetes Secrets; TLS and OIDC; GPU when stable | +| C05 | Kubernetes | Kubernetes 1.29 | Combined, three replicas; external PostgreSQL; Kubernetes Secrets; TLS and OIDC; GPU when stable | +| C06 | Docker | Fedora x86_64 with Docker 28.0.4 and SELinux enforcing | Local gateway; TLS and sandbox mTLS; credentials | +| C07 | Kubernetes | Supported OpenShift release | Combined, three replicas; external PostgreSQL; Kubernetes Secrets; TLS and OIDC | + +## Upgrade tests + +Upgrade tests run once per supported product installation package. They verify +that users can move from every supported source release to the pre-release +without reinstalling or losing supported state. + +For each installation package, the suite: + +1. Installs the source release and creates representative gateways, sandboxes, + policies, providers, credentials, and persisted state. +2. Exercises the source installation to establish a known-good baseline. +3. Performs the documented in-place upgrade using the pre-release + artifacts. +4. Verifies schema and state migrations, gateway health, existing resources, + policy behavior, and new sandbox creation. +5. Runs a post-upgrade smoke test that verifies existing resources and creates + and deletes a new sandbox. +6. Exercises rollback when rollback is part of the supported upgrade contract. + +### Configurations to test + +| ID | Installation package | Representative environment | Drivers | Upgrade path | +| --- | --- | --- | --- | --- | +| U01 | Homebrew formula | macOS on Apple Silicon | MicroVM | Previous stable formula to pre-release formula | +| U02 | DEB through APT | Ubuntu x86_64 | Docker | Previous stable repository package to pre-release package | +| U03 | RPM | Fedora x86_64 | Podman | Previous stable repository package to pre-release package | +| U04 | Snap | Ubuntu x86_64 | Docker | Previous stable revision to pre-release revision | +| U05 | Windows MSI through WinGet | Windows x86_64 | Docker Desktop | Previous stable MSI to pre-release MSI | +| U06 | Helm chart | Kubernetes 1.29 | Kubernetes | Previous stable chart and images to pre-release chart and image digests | + +## Breaking API change review + +Breaking API change review runs once per pre-release. It compares its +stable protobuf and public SDK interfaces with the latest stable baseline and +any additional baseline required by the N-1 maintenance promise. The review: + +1. Runs protobuf compatibility checks against the supported descriptor + baselines. +2. Runs language-specific compatibility checks for generated and hand-written + SDK or configuration interfaces. An agent may assess this using a breaking + API detection skill. + +Breaking API change review passes when no unaddressed breaking change affects a +stable API baseline and every intentional minor-release break satisfies the +versioning and migration requirements. + +## Security review + +Security review runs once per pre-release and will: + +1. Produce vulnerability, dependency, container, and infrastructure scan + results for the pre-release artifacts. +2. Run an agent-based security scanner, such as Codex Security, against changes + since the previous stable release. + +Security review passes when no unresolved critical or high-severity finding +affects a supported configuration. diff --git a/rfc/0014-release-stability/release-version-selection.md b/rfc/0014-release-stability/release-version-selection.md new file mode 100644 index 0000000000..d21c8d9211 --- /dev/null +++ b/rfc/0014-release-stability/release-version-selection.md @@ -0,0 +1,55 @@ +# RFC 0014 Supplement - Release Version Selection + +This supplement defines how OpenShell selects the next pre-release version +after a stable release during the `0.x` series. + +## Algorithm + +1. Find the latest stable tag and collect commits after it on the target branch. +2. Classify commits using Conventional Commits: + - `feat:` requests a minor release. + - A commit marked with `!` or a `BREAKING CHANGE` footer requests a minor + release during `0.x`. + - `fix:` and `deps:` request a patch release. + - Other commit types do not request a release by default. +3. Select the highest requested bump. Minor takes precedence over patch. If no + commit requests a release, do not create a pre-release. +4. Increment the stable version to obtain the pre-release base version. From + `0.2.0`, a patch becomes `0.2.1` and a minor becomes `0.3.0`. +5. Create a `-pre.1` Git tag before building and qualifying a new base version. + If that base already has a pre-release, increment the pre-release number. +6. Treat that base version as fixed for the weekly train. If a patch train has + started, stage any later breaking change that requires a minor release for + the following week's train. +7. Fail the branch check if the pre-release base version is not greater than the + latest stable version. + +| Latest stable | Commits since stable | Next pre-release | +| --- | --- | --- | +| `0.2.0` | `fix:` | `0.2.1-pre.1` | +| `0.2.0` | `feat:` | `0.3.0-pre.1` | +| `0.2.0` | `feat!:` | `0.3.0-pre.1` | +| `0.2.0` | Only `docs:` or `chore:` | No pre-release | + +Git tags are the version source of truth; release versions are never committed +to source. If a commit has multiple tags, select stable over pre-release, then +use Semantic Version order; use the commit SHA when neither exists. After +tagging the selected commit with its stable version, delete its `-pre.N` tags. +Stored pre-release artifacts retain their provenance metadata. + +## Release Please + +[Release Please](https://github.com/googleapis/release-please-action) can +implement the commit classification and base-version calculation. During +`0.x`, its manifest configuration should include: + +```json +{ + "bump-minor-pre-major": true, + "bump-patch-for-minor-pre-major": false +} +``` + +The release workflow owns the `-pre.N` suffix and stable publication. An explicit +`Release-As: x.y.z` footer may override the calculation with maintainer +approval.