feat(policy): establish direct TCP egress foundation

[johntmyers]

Summary

Establish the policy and proxy foundations for RFC 0005's direct DNS and transparent TCP work without enabling either runtime path yet. Explicit protocol: tcp is accepted as the current L4 host/port behavior, while proxy authorization now returns one generation-consistent decision snapshot that later adapters can consume safely.

Related Issue

Implements the first delivery slice of the accepted RFC in #2155. The RFC was merged without a separate tracking issue.

Changes

• Accept explicit protocol: tcp in policy YAML, profile validation, ambiguity checks, Rego evaluation, and incremental policy update parsing
• Keep explicit TCP equivalent to omitted protocol today and reject L7-only fields on TCP endpoints
• Replace split proxy policy lookups with one authoritative, generation-consistent authorization snapshot containing the action, matched policy, full matched endpoint metadata (including credential_binding), exact-host signal, and policy generation
• Fail closed when that combined authorization evaluation errors; missing optional metadata within a successful authorization snapshot still uses conservative defaults instead of re-querying policy
• Hydrate TLS, L7, destination constraints, and relay planning from that single snapshot
• Add dormant transparent-TCP transport and pinned-address destination-validation primitives for later stack PRs
• Document current behavior and update policy-generation/CLI agent guidance

Testing

• mise run pre-commit passes
• mise run docs passes
• mise run test passes
• Unit tests added/updated
• Regression proves an authoritative OPA evaluation error denies egress and exposes no fallback endpoint metadata
• Replaced poisoned-lock tests for the retired multi-query behavior with conservative-default tests for successful snapshots that omit optional metadata
• E2E tests added/updated (not applicable: DNS interception and transparent TCP routing remain disabled)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[github-actions]

🌿 Preview your docs:https://nvidia-preview-pr-2711.docs.buildwithfern.com/openshell

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as the explicitly authorized first delivery slice of accepted RFC #2155 and the authoritative policy-DNS/transparent-TCP series plan. This review evaluates only stage-1 responsibilities and does not require DNS, mapping-store, capture, or native transparent-TCP functionality deferred to later PRs.
Head SHA: c9ee490d4c2ea7ae7e93a1eee723425022414fed
Base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Merge base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Patch ID: 64c0cd43a0b601c291aab47f2c53c7bd1a8217f6
Gator payload: 4
Review mode: initial
Previous reviewed SHA: none
Review budget exhausted: no
Maintainer decision required: no

Blocking findings:

• GATOR-c9ee490d-01: The authoritative authorization result and ambiguity model must preserve explicit TCP eligibility and a generation-stable matched endpoint identity, even though explicit TCP and omitted protocol remain equivalent for CONNECT.
• GATOR-c9ee490d-02: A pinned resolved-address destination mode must construct an unopened connector from policy-DNS-pinned addresses without performing connect-time DNS resolution.
• GATOR-c9ee490d-03: Explicit protocol: tcp must reject fields that promise L7-only enforcement or transformation rather than accepting and silently ignoring them.
• GATOR-c9ee490d-04: Published policy and CLI reference documentation must list protocol: tcp and accurately describe its current L4-only constraints when the public parser accepts it.

Carried findings:

• None

Non-blocking suggestions:

• None

Docs: Missing canonical Fern updates for the newly accepted public protocol: tcp policy and CLI syntax.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @johntmyers. I reviewed the new implementation commit and your note that the Fern documentation is being deferred to a separate branch. I checked the stage-1 code obligations against the authoritative policy-DNS/transparent-TCP series plan and checked the canonical published policy and CLI pages.

Validation: This remains project-valid as PR 1 of the authorized series. DNS, mapping-store, capture, and native transparent-TCP runtime functionality remain correctly deferred to later PRs.
Head SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Merge base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Patch ID: 544beca42375b85776ff2da0101389b02c7d4998
Gator payload: 4
Review mode: follow_up
Previous reviewed SHA: c9ee490d4c2ea7ae7e93a1eee723425022414fed
Review budget exhausted: no
Maintainer decision required: no

Resolved findings:

• GATOR-c9ee490d-01: the authoritative result now preserves matched endpoint identity and explicit TCP eligibility, with ambiguity coverage.
• GATOR-c9ee490d-02: the pinned-address validation plan now constructs an unopened connector without DNS resolution and is covered by deterministic tests.
• GATOR-c9ee490d-03: TCP activation now rejects the represented L7-only field family across policy and provider-profile paths.

Blocking findings:

• GATOR-c9ee490d-04 remains open. docs/reference/policy-schema.mdx still instructs users to omit the protocol for TCP passthrough, while docs/sandboxes/policies.mdx still omits tcp from the values accepted by policy update. Because this PR directly ships the public policy and CLI syntax and the stage-1 plan requires documenting its current L4-only semantics, a separate in-flight docs branch does not make this PR's published contract accurate and is not a maintainer waiver. Please update those two existing Fern pages here, or obtain an explicit maintainer waiver for deferring them. No navigation change is needed.

Carried findings:

• GATOR-c9ee490d-04: canonical Fern policy and CLI documentation must describe explicit protocol: tcp and its current L4-only constraints.

Docs: Missing for the direct public policy/CLI syntax change.

Next state: gator:in-review

[johntmyers]

@ gator, as mentioned in the reply to your comment we will defer the docs. #2711 (comment)

[github-actions]

Label test:e2e applied for 2dedd9c. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

Maintainer Review Nudge

This PR has been in gator:approval-needed for more than 48 business hours with no maintainer approval.

@NVIDIA/openshell-maintainers @NVIDIA/openshell-codeowners @mrunalp@maxamillion@derekwaynecarr, can someone review and either approve, request changes, or close this out?

[johntmyers]

gator-agent

Author Follow-Up Nudge

This PR has been in gator:in-review for more than 48 business hours with unresolved review feedback.

@johntmyers, please either update docs/reference/policy-schema.mdx and docs/sandboxes/policies.mdx for the new protocol: tcp contract, or obtain an explicit maintainer waiver for deferring those canonical docs. If the standalone docs PR is the intended resolution, please link it so maintainers can make that decision.

[johntmyers]

gator-agent

Maintainer Convergence Decision

Thanks @johntmyers. I checked the current head after your response to @pimlock's request to require a non-empty host and a valid concrete port for explicit protocol: tcp endpoints. The independent critical-only review found no newly introduced Critical defect, and the four earlier stage-1 findings remain resolved.

Head SHA: 5157062bebefdc9271432b10829964b093b950f0
Base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Merge base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Patch ID: 31b4660a50039e2c01ad49c0137f06200b82e4b8
Gator payload: 4

The autonomous Warning budget is exhausted, and a specific maintainer decision is required before review can proceed.

Root-cause findings:

• The requested explicit-TCP destination validation was broadened to every network endpoint. The new MissingEndpointHost check rejects the existing hostless allowed_ips mode, even though proto/sandbox.proto documents that an empty host with allowed_ips matches any hostname resolving into the allowed range and sandbox-policy.rego implements that behavior. This is a concrete public policy compatibility change outside the plan's stage-1 requirement that only explicit protocol: tcp eligibility have a non-empty host and concrete port.

Scope growth:

• The remediation changes validation semantics for all endpoint protocols and removes a currently supported endpoint shape, rather than enforcing the TCP-specific eligibility boundary requested by the series plan and @pimlock.

Reviewer-quality signals:

• The normalized critical-only review proposed no new findings; this convergence request comes from the separate mandatory scope-growth check.

Maintainer action: @pimlock or another OpenShell maintainer, please decide whether this PR should (a) limit the new host/port requirement to explicit protocol: tcp, preserving hostless allowed_ips, or (b) intentionally retire the hostless mode here and require the corresponding public-contract, compatibility, and migration treatment. A maintainer confirmation of the intended boundary will unblock the next step.

Next state: gator:blocked
Blocked reason: review_convergence_decision_required

[johntmyers]

gator-agent

Blocked

Thanks @pimlock. I checked your maintainer clarification that the existing hostless non-TCP allowed_ips mode remains supported while explicit protocol: tcp requires a DNS hostname. Thanks @johntmyers. I checked current head 26d10b1de8a1915e72aae2b87ad93e8986cac16c; the shared validation now preserves that legacy mode and applies the DNS-host boundary to explicit TCP with dedicated validation coverage. That resolves the prior maintainer convergence decision.

Head SHA: 26d10b1de8a1915e72aae2b87ad93e8986cac16c
Gator payload: 4

Gator is now blocked because GitHub reports this head as unmergeable with merge conflicts (mergeable: false, mergeable_state: dirty). The required critical-only author-delta review cannot proceed against a conflicted patch.

Next action: @johntmyers, resolve the conflicts with main and push the conflict-resolved head. Gator will then rebuild the feedback ledger and review only the new author delta under the stage-1 boundary.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @johntmyers. I reviewed conflict-resolved head 808f5fbad349fadcd38ddd3772e67d0308546194 after your update. Thanks @pimlock. I also preserved your maintainer clarification that hostless non-TCP allowed_ips remains supported while explicit protocol: tcp requires a DNS hostname and at least one valid concrete port.

Validation: This remains project-valid as stage 1 of the accepted policy-DNS/transparent-TCP series. It establishes the authoritative generation-consistent authorization boundary, stable endpoint identity, explicit protocol: tcp policy contract, future transport/correlation interfaces, and the unused pinned-address connector foundation. Policy DNS, resolved-endpoint storage, kernel capture, resolver injection, native transparent TCP, and runtime enablement remain correctly deferred to later PRs.

Head SHA: 808f5fbad349fadcd38ddd3772e67d0308546194
Base SHA: 998db0478060976b3f530cdbbc26f336752b90a5
Merge base SHA: 998db0478060976b3f530cdbbc26f336752b90a5
Patch ID: ba14bc8d90b491c93759c15596f8237b08851e06
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: 22a4de473a88e7d8da23a93ed787a30afbf4ce3d
Review budget exhausted: yes
Maintainer decision required: no

What I checked: the complete authoritative series plan, the durable feedback ledger, the conflict-resolution author delta, the TCP-specific host and port validation boundary, preservation of hostless non-TCP allowed_ips, and the current-head workflow dispatch.

Blocking findings:

• No newly introduced Critical findings in the current author-only delta.

Carried findings:

• None. GATOR-c9ee490d-01 through GATOR-c9ee490d-04 remain resolved.

Docs: The canonical Fern policy pages continue to document explicit protocol: tcp, its current L4 equivalence to omitted protocol, and its rejected L7-only fields.

Checks: The merge-conflict blocker is cleared. test:e2e remains applied; DCO is green, Branch Checks is running, and Helm Lint and E2E are queued for this head.

Disposition: Review feedback is resolved and the required current-head workflows are dispatched.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

Blocked

Head SHA: 062ce26e11c2aff44b9f21d23afc7c7ac9b1c47a
Gator payload: 4

Stage-1 review is complete. The feedback ledger identifies this head as rebase-equivalent to reviewed head 808f5fbad349fadcd38ddd3772e67d0308546194; all four prior findings remain resolved. Thanks @pimlock, maintainer approval is present.

Gator is blocked by the required OpenShell / E2E gate. Run 32309575283 failed across the VM, Podman, and Kubernetes jobs during shared tool bootstrap: Go followed golangci-lint deprecation metadata to nonexistent revision v2.13.0. This is not a PR-owned stage-1 regression: mise.toml has the same golangci-lint = "2.12" pin on this head, its base, and current main, and the jobs failed before exercising the changed policy/proxy behavior.

Next action: @NVIDIA/openshell-maintainers, fix or update the shared golangci-lint tool pin on main and rebase this PR, or rerun the current-head Branch E2E workflow after the upstream revision is available. Gator will then re-check the required gate and move to gator:merge-ready if it is green with the existing approval.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @johntmyers. I reviewed current head 6b01545702a7a2da2384bf319fceb489a887eb27 after the rebase that cleared the shared E2E bootstrap blocker. I preserved @pimlock's maintainer clarification that hostless non-TCP allowed_ips remains supported while explicit protocol: tcp requires a DNS hostname and at least one valid concrete port.

Validation: This remains project-valid as stage 1 of the accepted policy-DNS/transparent-TCP series. It establishes the authoritative generation-consistent authorization boundary, stable endpoint identity, explicit protocol: tcp policy contract, future transport/correlation interfaces, and the unused pinned-address connector foundation. Policy DNS, resolved-endpoint storage, kernel capture, resolver injection, native transparent TCP, and runtime activation remain correctly deferred to later PRs.

Head SHA: 6b01545702a7a2da2384bf319fceb489a887eb27
Base SHA: 9505ca5ed1159c462296dbeaa045bfd1fc9ba53b
Merge base SHA: 9505ca5ed1159c462296dbeaa045bfd1fc9ba53b
Patch ID: 0a8e1c40ef62e028b3298e92ddbe2bf1e1e64501
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: 808f5fbad349fadcd38ddd3772e67d0308546194
Review budget exhausted: yes
Maintainer decision required: no

What I checked: the complete authoritative series plan, the durable feedback ledger, the author-only range comparison across the rebase, the Podman stop/restart fence conflict resolution, preservation of the stage-1 TCP host/port validation boundary, and current-head workflow dispatch.

Blocking findings:

• No newly introduced Critical findings in the latest author-only delta.

Carried findings:

• None. GATOR-c9ee490d-01 through GATOR-c9ee490d-04 remain resolved.

Docs: The canonical Fern policy pages continue to document explicit protocol: tcp, its current L4 equivalence to omitted protocol, and its rejected L7-only fields.

Checks: The prior shared E2E bootstrap blocker is cleared by the rebase. test:e2e remains applied; DCO and Helm Lint are green, while current-head Branch Checks and E2E are queued.

Disposition: Review feedback is resolved and the required current-head workflows are dispatched.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Head SHA: 6b01545702a7a2da2384bf319fceb489a887eb27
Gator payload: 4

Final status: Gator reached gator:merge-ready after the stage-1 review completed, required checks passed, and maintainer approval was present.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.