feat: shadow-mode SQL pre-validation telemetry

[anandgupta42]

What does this PR do?

Ships telemetry-only instrumentation to measure whether pre-execution SQL validation is worth implementing as a blocking check.

The original scope (blocking validation, apply_patch retry, file-not-found cache, prompt rules, warehouse guidance, validate auto-pull) was deferred after analysis showed the telemetry sample (8 machines, 1 day, 2-3 outlier users) didn't justify 297 LOC of production defenses.

This PR adds:

• sql_pre_validation telemetry event (outcome, reason, schema_columns, duration_ms, error_message)
• preValidateSql() runs fire-and-forget before every sql_execute — validates the query against the cached schema via altimate_core.validate, emits telemetry, and never blocks execution
• Zero user-facing latency impact (async, detached)

After 2 weeks of shadow telemetry we can answer:

• What % of sql_execute calls would have been blocked?
• What's the validation latency distribution?
• How often is the cache stale / empty / missing?
• Would blocking have false-positive risk?

Then decide whether to flip to blocking mode.

Type of change

• Bug fix
• New feature (shadow telemetry)
• Test coverage
• Documentation
• Refactoring
• Infrastructure

Issue for this PR

Follow-up to telemetry analysis of 2026-03-30 (Azure AppInsights, altimate-code-os).

How did you verify your code works?

• Typecheck passes
• Marker guard passes
• Pre-validation is fire-and-forget with .catch(() => {}) — cannot fail the sql_execute call
• trackPreValidation() emits telemetry with one of 4 outcomes (skipped, passed, blocked, error)

Checklist

• Typecheck passes
• Marker guard passes
• No user-facing behavior change (shadow mode)
• Telemetry event typed and documented

Summary by CodeRabbit

• New Features
  • Added non-blocking SQL pre-execution validation that checks query structure against cached schema to enhance reliability.
  • Implemented comprehensive telemetry event tracking for SQL validation activities, capturing outcomes, performance metrics, and diagnostic details.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

The PR adds SQL pre-validation functionality that executes before SQL processing using cached schema information, and introduces corresponding telemetry events to track validation outcomes (skipped, passed, blocked, or error).

Changes

Cohort / File(s)	Summary
Telemetry Event Definition packages/opencode/src/altimate/telemetry/index.ts	Added new sql_pre_validation event variant to Telemetry.Event union with outcome classification, timing, schema metadata, and optional error details.
SQL Pre-Validation Logic packages/opencode/src/altimate/tools/sql-execute.ts	Implemented pre-execution SQL validation using cached schema; includes warehouse resolution with fallback, TTL-based cache freshness checks, schema context building with truncation tracking, validation invocation, and telemetry emission for all outcomes.

Sequence Diagram

sequenceDiagram
participant SQLExec as SQL Execute Tool
participant WH as Warehouse Registry
participant Cache as Schema Cache
participant Validator as altimate_core.validate
participant Telemetry as Telemetry Tracker
SQLExec->>WH: Resolve target warehouse<br/>(with fallback)
WH-->>SQLExec: Warehouse instance
SQLExec->>Cache: Check cache availability<br/>and freshness (TTL)
Cache-->>SQLExec: Cached schema columns
SQLExec->>SQLExec: Build schema context<br/>(apply scan limit)
SQLExec->>Validator: Invoke validation<br/>with cached schema
Validator-->>SQLExec: Validation errors
alt Validation Errors Present
SQLExec->>SQLExec: Categorize as 'blocked'
else No Errors
SQLExec->>SQLExec: Categorize as 'passed'
end
SQLExec->>Telemetry: Emit pre-validation event<br/>(outcome + metadata)
Telemetry-->>SQLExec: Tracked

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• feat: add telemetry intelligence signals for debugging and improvements #564: Modifies both telemetry/index.ts and sql-execute.ts to extend Telemetry.Event and add SQL-related telemetry emission, directly overlapping with this PR's scope.
• feat: add core_failure telemetry with PII-safe input signatures #245: Adds SQL-related telemetry event types and error masking utilities used by SQL validation and error handling workflows.
• feat: [AI-5975] add sql_quality telemetry for issue prevention metrics #446: Extends Telemetry.Event union with additional event variants, following the same pattern as the new sql_pre_validation event.

Suggested reviewers

• mdesmet

Poem

🐰 A shadow runs before the SQL starts to play,
Checking schemas through the cache's hidden way,
Validation whispers what the query cannot see,
Telemetry hops along in perfect harmony! ✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description is incomplete. It lacks the required 'PINEAPPLE' marker at the top (required for AI-generated contributions), and the Test Plan section is missing.	Add 'PINEAPPLE' at the very top of the description as required, and include a Test Plan section explaining how the changes were validated.
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding shadow-mode SQL pre-validation telemetry. It is concise, specific, and clearly conveys the primary objective.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/pre-execution-validation-and-error-resilience

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

3 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/altimate/tools/sql-execute.ts">
<violation number="1" location="packages/opencode/src/altimate/tools/sql-execute.ts:120">
P2: When `warehouse` is omitted, this validates against the cache’s first warehouse instead of the warehouse `sql.execute` will actually use.</violation>
<violation number="2" location="packages/opencode/src/altimate/tools/sql-execute.ts:140">
P2: Limiting validation to 10k columns can create false structural errors on large warehouses and bias the telemetry sample.</violation>
<violation number="3" location="packages/opencode/src/altimate/tools/sql-execute.ts:229">
P1: Mask the validator error before emitting telemetry; this currently uploads raw schema identifiers into App Insights.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}