Skip to content

feat(compute): add standalone first-party drivers - #2822

Merged
drew merged 5 commits into
mainfrom
refactor/2417-external-driver-binaries/drew
Aug 21, 2026
Merged

feat(compute): add standalone first-party drivers#2822
drew merged 5 commits into
mainfrom
refactor/2417-external-driver-binaries/drew

Conversation

@drew

@drewdrew commented Aug 20, 2026•
edited by elezar
Loading

Copy link
Copy Markdown
Collaborator

Summary

Add standalone Docker, Podman, Kubernetes, and VM compute-driver binaries and exercise them through the public compute-driver RPC contract. This isolates external-driver parity from the later gateway/server composition move.

Related Issue

Part of #2417.

Stacked on #2786. Followed by #2823.

Changes

  • add standalone public-RPC entry points for Docker, Podman, and Kubernetes; VM already exposes the same service
  • add shared external-driver socket and operator allowlist utilities
  • add a driver-free gateway build path for public-API validation
  • add external-driver E2E tasks for Docker, Podman, Kubernetes, and VM
  • include the final VM default-image handling at introduction
  • reuse the Kubernetes supervisor image and isolate the external sidecar with a test-only post-renderer
  • scope and stabilize the external Podman smoke job

External Kubernetes drivers currently support shared and managed workspace modes. operator mode is rejected at startup because its live namespace allowlist is not yet synchronized across the process boundary; #2839 tracks support.

No production Helm configuration surface is introduced for the Kubernetes test setup.

Testing

  • mise run pre-commit
  • mise run e2e:gateway:no-compute-drivers
  • Docker external-driver E2E not rerun during the history split; covered by branch CI
  • Podman external-driver E2E requires Podman; covered by branch CI
  • Kubernetes external-driver E2E requires k3d; covered by branch CI
  • VM external-driver E2E requires the KVM runner; covered by branch CI

Checklist

@copy-pr-bot

copy-pr-botBot commented Aug 20, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

This was referenced Aug 20, 2026
Merged
Draft
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from 33ffd57 to cdd5fd4CompareAugust 20, 2026 05:10
@drew

drew commented Aug 20, 2026

Copy link
Copy Markdown
CollaboratorAuthor

/ok to test

@drewdrew added the test:e2e Requires end-to-end coverage label Aug 20, 2026
@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown

Label test:e2e applied for cdd5fd4. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from cdd5fd4 to 8341fa8CompareAugust 20, 2026 14:28
@drew

drew commented Aug 20, 2026

Copy link
Copy Markdown
CollaboratorAuthor

/ok to test

@elezarelezar mentioned this pull request Aug 20, 2026
Open
8 tasks
@drew

drew commented Aug 20, 2026

Copy link
Copy Markdown
CollaboratorAuthor

/ok to test

@drew

drew commented Aug 20, 2026

Copy link
Copy Markdown
CollaboratorAuthor

/ok to test

@elezar
elezarforce-pushed the refactor/2417-external-driver-binaries/drew branch from 9af06cb to 1596913CompareAugust 20, 2026 15:07
@elezar

elezar commented Aug 20, 2026

Copy link
Copy Markdown
Member

@drew A quick summary of the review changes I made and force-pushed to this PR branch:

  • Restored the PR head so its local ancestry starts from 25f9a63, the reviewed state of the parent stack.
  • Made the external Kubernetes E2E gateway build use the bundled-z3 feature, so the gateway binary does not depend on libz3 being present in the distroless runtime image. This retains the intent of the superseded 9af06cb change without adding a build dependency to the workflow runner.
  • Added a fail-closed configuration check for Kubernetes workspace_mode = operator when Kubernetes is selected through a remote socket. Shared and managed modes remain supported externally. Issue feat(kubernetes): support operator workspace mode with external compute drivers #2839 tracks proper external operator-mode support.
  • Kept the namespace validator fail-closed with a generic invariant error rather than inferring that a missing allowlist necessarily means an external driver.
  • Added focused tests for the external-driver Unix socket contract: directory and socket modes, cleanup, symlink-parent rejection, non-socket conflicts, stale owned socket replacement, and same-UID acceptance.
  • Updated the debug-openshell-cluster skill with the first-party socket ownership, 0700/0600 permission, and same-UID peer requirements.

Validation completed with mise run pre-commit and the five focused openshell-core socket tests. The current PR head is 1596913.

@elezar

elezar commented Aug 20, 2026

Copy link
Copy Markdown
Member

/ok-to-test 1596913

@elezar
elezarforce-pushed the refactor/2417-external-driver-binaries/drew branch from 1596913 to d445c81CompareAugust 20, 2026 15:44
elezar
elezar previously approved these changes Aug 20, 2026
View reviewed changes

@elezarelezar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviwed and made some changes as called out in #2822 (comment)

LGTM, but feel free to get a second opinion.

Also note the follow-ups, but it may be that these are already addressed in #2823.

@elezar

elezar commented Aug 20, 2026

Copy link
Copy Markdown
Member

/ok-to-test d445c81

@elezar
elezar marked this pull request as ready for review August 20, 2026 15:52
@elezar
elezar requested review from a team and derekwaynecarr as code ownersAugust 20, 2026 15:52
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from d445c81 to 6153c36CompareAugust 20, 2026 16:10
@copy-pr-bot

copy-pr-botBot commented Aug 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@politerealismpoliterealism mentioned this pull request Aug 20, 2026
Open
3 tasks
@drew
drew dismissed elezar’s stale reviewAugust 20, 2026 19:18

The merge-base changed after approval.

@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from 6153c36 to 02f11e2CompareAugust 20, 2026 19:18
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from 02f11e2 to db8de1bCompareAugust 20, 2026 20:50
Base automatically changed from refactor/2417-compute-driver-registry/drew to mainAugust 20, 2026 21:56
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from db8de1b to 187ba3eCompareAugust 20, 2026 21:56
@pimlockpimlock mentioned this pull request Aug 20, 2026
Draft
10 tasks
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from 187ba3e to f06b131CompareAugust 20, 2026 22:51
pimlock
pimlock previously approved these changes Aug 20, 2026
View reviewed changes
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from f06b131 to 7ea169bCompareAugust 20, 2026 23:49
drewand others added 4 commits August 20, 2026 17:46
@drew
feat(compute): add standalone first-party drivers
75ad91b
Build Docker, Podman, Kubernetes, and VM drivers as external binaries and
exercise each through the public compute-driver API. Keep the external E2E
setup complete at introduction, including VM image selection, Kubernetes
post-renderer isolation, supervisor reuse, and scoped Podman coverage.
External Kubernetes endpoints support shared and managed workspace modes.
Operator mode remains restricted to the in-process driver because gateway
authentication and the driver must share a dynamic namespace allowlist.
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew
ci(e2e): run managed and external drivers independently
e812dde
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@elezar@drew
test(compute): cover external driver socket contract
37a647f
Signed-off-by: Evan Lezar <elezar@nvidia.com>
@drew
ci(e2e): install bundled Z3 build dependency
e8c4bc8
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew
drewforce-pushed the refactor/2417-external-driver-binaries/drew branch from 7ea169b to e8c4bc8CompareAugust 21, 2026 00:46
@drew
fix(server): gate in-tree driver tracing
3f573b6
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew
drew added this pull request to the merge queueAug 21, 2026
Merged via the queue into main with commit 40f8229Aug 21, 2026
187 of 193 checks passed
@drew
drew deleted the refactor/2417-external-driver-binaries/drew branch August 21, 2026 03:03
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pimlockpimlockpimlock approved these changes
@elezarelezarelezar left review comments
@mrunalpmrunalpAwaiting requested review from mrunalpmrunalp is a code owner
@sjenningsjenningAwaiting requested review from sjenningsjenning is a code owner
@derekwaynecarrderekwaynecarrAwaiting requested review from derekwaynecarrderekwaynecarr is a code owner

Assignees

No one assigned

Labels

test:e2eRequires end-to-end coverage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@drew@elezar@pimlock