Uh oh!
There was an error while loading. Please reload this page.
feat(ocsf): comprehensive audit telemetry and off-box OTLP export - #3
Open
mattj-monad wants to merge 3 commits into
Open
feat(ocsf): comprehensive audit telemetry and off-box OTLP export#3mattj-monad wants to merge 3 commits into
mattj-monad wants to merge 3 commits into
Conversation
Add the delivery substrate that carries gateway and sandbox logs, and OCSF security events, off the box to any OTLP/gRPC collector, plus the OCSF v1.7.0 event model they travel in. - Export gateway and sandbox logs as OTLP log records on the same endpoint and resource identity as traces, opt-in via `[openshell.gateway.otlp] export_logs`, with TLS for https collectors. - Accountable delivery (Tier 2a): a bounded in-memory queue, the exporter driven directly rather than through the SDK batch processor, bounded retries, and a `telemetry_gap` record for every dropped line so loss is never silent. - OCSF event classes, builders, objects, enums, shorthand and JSONL formatters, and a tracing-layer bridge; `Actor` gains an optional user alongside its now-optional process. - Raw is the default push format: the full OCSF document travels in one `ocsf.raw` attribute (with `ocsf.severity_id`), several-fold cheaper per record than the flattened shape; `OPENSHELL_OCSF_PUSH_FORMAT=flat` opts back. OCSF events export at their own severity. - Gateway-scoped events (no sandbox in play) export on a gateway lane with their structured fields as attributes; export-path targets are excluded so a failing collector cannot amplify its own failure. - Benchmarks for the fan-in and export paths. Signed-off-by: clippy <matt@monad.com>
Every state-changing gateway operation now emits one OCSF audit event carrying the authenticated principal as the actor, the outcome, and a request-id for correlation. Governed by `[openshell.gateway.audit]` (master `enabled` on by default, mirrored by env vars and CLI flags with CLI > env > TOML precedence). Event classes: - Entity Management [3004]: workspace, member, provider, profile, credential, sandbox lifecycle, provider attach/detach, ssh session, exec and tcp-forward session initiation, and sandbox-token minting. - Config State Change [5019]: settings (before/after values, credential keys always redacted), policy replace/merge, draft-chunk decisions, inference routes, and a companion marker when a gateway interceptor rewrites a request in flight. - Authentication [3002]: failures always (mechanism, low-cardinality reason, peer address; never the credential); per-request successes behind a toggle. - Detection Finding [2004]: cross-sandbox access attempts (up-front and mid-stream) and sandbox-principal admin attempts, High and alert-grade, dual-emitted beside the domain denial. Guarantees: outcomes judged from the store (no-op deletes record as Failure at Low severity so alerting sees them); events emit at the store commit so a landed change is on record even if a later step fails; secrets never enter a record; actors come from the session and render as name(uid); the `ocsf` tracing target is pinned at INFO so a coarse RUST_LOG cannot silently sever the trail. Background mutations (credential auto-rotation, compute reconciliation, startup) audit under `system:<component>` actors. Signed-off-by: clippy <matt@monad.com>
Add a three-page gateway-audit guide under docs/observability/ (overview and how-to, the complete toggle reference, and the full event catalog with an example and use-cases per class), wired into the docs navigation and cross-linked from the sandbox logging page, the gateway config reference, and the README. Extend the config reference and Helm chart with the OTLP export and `[openshell.gateway.audit]` surfaces, update the architecture logging overview, and record the emit-an-audit-event rule for new state-changing RPCs in AGENTS.md. Signed-off-by: clippy <matt@monad.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
OpenShell can run agents safely, but understanding the actions the agent took (or were denied by policy) is limited to a subset of actions and the logs cannot be exported for analysis.. This PR makes every security-relevant action a structured, standard-schema (OCSF) event and streams the logs to any OpenTelemetry collector. It turns OpenShell into an enterprise ready solution with full visibility, which can audited, monitored, and learned from.
Why it matters
What we built
Example
Every event is human-greppable shorthand and full OCSF JSON. A sandbox deletion, for example, renders as:
...and exports as a complete OCSF Entity Management record carrying the actor, outcome, correlation id, and severity for downstream filtering and alerting.
Scope
Four OCSF event classes (Entity Management, Config State Change, Authentication, Detection Finding), ~34 instrumented handlers, spanning the shared OCSF and OTLP crates, the sandbox supervisor, and the gateway.
Alignment with roadmap
This is the built, tested implementation of OpenShell's own Enterprise Observability direction (NVIDIA#1055) and the issues the maintainers have already broken it into: control-plane audit events (NVIDIA#2911), gateway authentication and boundary findings (NVIDIA#2912), OCSF-over-OTLP export (NVIDIA#2892), and accountable relay delivery (NVIDIA#2642). Sandbox and control plane, one correlated stream, off the box — ready now.
Next steps.
We are working with the OpenShell maintainer team to get the fixes and issues aligned with requests and requirements. Once merged, these contributions should allow enterprise teams to adopt OpenShell with full visibility into the entire projects actions.