Skip to content

chore(devel): enable CLI telemetry in the traced sandbox kit - #3382

Open
migmartri wants to merge 1 commit into
mainfrom
chore/sandbox-kit-cli-telemetry
Open

chore(devel): enable CLI telemetry in the traced sandbox kit#3382
migmartri wants to merge 1 commit into
mainfrom
chore/sandbox-kit-cli-telemetry

Conversation

@migmartri

@migmartrimigmartri commented Aug 28, 2026

Copy link
Copy Markdown
Member

The traced-Claude sandbox kit (devel/sandbox-kit) set DO_NOT_TRACK=1 and did not allow egress to the telemetry endpoint, so CLI usage from sessions run inside the sandbox was never reported.

This removes the DO_NOT_TRACK environment variable from the kit and adds crb.chainloop.dev:443 to the kit's network allow rules, so those sessions report usage like any other CLI invocation. The host is deliberately kept out of NO_PROXY, since telemetry is plain HTTPS and does not need the proxy bypass that the gRPC endpoints require.

AI disclosure: this change was produced with the assistance of Claude Code.

Review in cubic

@chainloop-platform

chainloop-platformBot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

AI Session Checks — 🟡 60% · ✅ 0 failing

Avg scoreSessionsFailing policiesAttributionFilesLinesTotal Duration
🟡 60%1✅ 0100% AI / 0% Human1+1 / -11h54m28s

🟡 60% — 100% AI — ✅ All policies passing

Aug 28, 2026 13:52 UTC · 1h54m28s · $3.26 · 126 in / 25.0k out · claude-code 2.1.246 (claude-opus-5)

View session details ↗

Change Summary

  • Removes DO_NOT_TRACK from the traced sandbox kit environment.
  • Adds crb.chainloop.dev:443 to the sandbox-kit network allowlist.
  • Prepares a signed commit and branch for the change, but no PR is shown as opened.

AI Session Overall Score

🟡 60% — Focused config change, but verification never exercised the sandbox behavior end-to-end.

AI Session Analysis Breakdown

🟢 92% · scope-discipline

🟢 Both requested config changes stayed in spec.yaml. · High Impact

🟢 88% · solution-quality

🟢 AI used direct allowlist and env changes instead of bypasses. · High Impact

🟢 86% · user-trust-signal

🟢 The user moved between asks without any correction cycle. · Medium Impact

🟡 70% · context-and-planning

🟠 The PR-creation phase started without an auth or remote precheck, so setup was partly improvised. · Medium Severity

💡 For PR-creation asks, front-load branch, signing, and push prerequisites before editing repo config or attempting network actions.

🟡 68% · alignment

🟠 The user asked for a PR, but the session only shows a local commit plus failed or missing push results. · Medium Severity

💡 Confirm remote auth and PR prerequisites before treating a local commit as progress toward a requested PR.

🔴 32% · verification

🔴 The sandbox-kit change was syntax-checked, but nobody exercised the new telemetry path end-to-end. · High Severity

💡 For config-shaped changes, run the new path end-to-end once; parsing the file is not enough verification.


File Attribution

████████████████████ 100% AI / 0% Human

StatusAttributionFileLines
modifiedaidevel/sandbox-kit/spec.yaml+1 / -1

Policies (4)

StatusPolicyMaterialMessages
✅ Passedai-config-ai-agents-allowedai-coding-session-6a4633-
✅ Passedai-config-no-dangerous-commandsai-coding-session-6a4633-
✅ Passedai-config-no-secretsai-coding-session-6a4633-
✅ Passedai-config-mcp-servers-allowedai-coding-session-6a4633-

Security Checks — ⚠️ 1 failing

iac-scan

StatusPolicyMessages
✅ Passediac-misconfiguration-

PR info

StatusPolicyMessages
✅ Passedpr-description-required-
⚠️ Failedpr-user-story-linkedPR/MR #3382 does not reference a user story or issue in title, description, or branch 'chore(devel): enable CLI telemetry in the traced sandbox kit'. Expected patterns: ["(?i)[A-Z]+-[0-9]+", "#[0-9]+", "(?i)[A-Z]{2", "}-[0-9]+", "(?i)gh-[0-9]+", "(?i)\[[A-Z]+-[0-9]+\]"]

⏭️ 2 scans not applied

ScanReason
vulnerability-scanno manifest/lockfile changed
github-actions-scanno workflow files changed

View attestation ↗


Powered by Chainloop and Chainloop Trace

Drop DO_NOT_TRACK=1 from the sandbox kit environment and allow egress to
crb.chainloop.dev, the endpoint the CLI reports usage telemetry to, so
sessions run through the kit are counted like any other CLI usage.
Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
Chainloop-Trace-Sessions: 6a463356-6105-4697-a537-fd08c713afa2
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@migmartri
migmartriforce-pushed the chore/sandbox-kit-cli-telemetry branch from 2644f50 to f80771bCompareAugust 28, 2026 15:47

@cubic-dev-aicubic-dev-aiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@migmartri