feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(runtime): validate provider request traces - #1464

Merged
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace
Jul 28, 2026
Merged

feat(runtime): validate provider request traces#1464
Astro-Han merged 9 commits into
apache:mainfrom
Nyvo-io:feat/1269-request-trace

Conversation

@Nyvo-io

@Nyvo-ioNyvo-io commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version provider-request captures as schema v2 while preserving legacy v1 reads
  • hash protocol-independent request semantics while preserving normalized reasoning effort
  • parse capture and physical-attempt rows and fail closed on malformed, torn, missing, or mismatched evidence
  • validate exact expected identity sets across continuation runs and autonomous retries, including retries that use different sessions
  • preserve the first trace-write failure in memory, attempt header and sentinel persistence independently, and carry the latch into the terminal header

The exported artifact is intentionally scoped to top-level Harbor invocations. Semantic-compaction and child-agent provider dispatches are outside this contract; the Runtime normalization and attempt-codec ownership cleanups are deferred to the broader Runtime refactor rather than duplicated here.

This is the second flat split requested in #1451 and addresses the request-trace evidence findings on that PR.

Refs #1269

Verification

  • provider-request-trace: 23 passed
  • SessionManager: 235 passed
  • terminal ledger: 26 passed
  • Runtime full suite: 2,558 passed, 7 skipped
  • Headless full suite: 1,365 passed, 2 skipped
  • runtime and headless typecheck passed
  • lint: 2,160 files passed
  • format check: 1,116 files passed
  • git diff check passed
  • local review-agent and independent Codex diff review: no findings

Not run: the paid Kimi/Harbor benchmark. That belongs to the separate stacked benchmark PR.

@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 45d3b5c to 0a2a07fCompareJuly 25, 2026 03:25

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the P1 trace-completeness issue noted inline.

Non-blocking suggestion: the attempt parser requires integer timestamps, but AiSdkBackendInput.now, ProviderRequestAttemptRecord, and the AgentRun codec allow any finite number. A fractional monotonic clock produces a valid attempt that the reader rejects as invalid_attempt. Either accept finite non-negative values or normalize the timestamps at the Runtime producer. Keeping the decoder next to the Runtime record type would also prevent the two contracts from drifting again.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All requested changes are addressed and the inline thread is resolved. d62b136 fails closed on trace write/corruption sentinels and accepts the Runtime finite timestamp contract; 3e2c222 normalizes equivalent cross-protocol request evidence; 370c437 validates same-session continuation executions. The focused regressions, full workspace suite, and GitHub typecheck/test/e2e checks all pass. @Astro-Han, please re-review when convenient.

Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io
Nyvo-ioforce-pushed the feat/1269-request-trace branch from 370c437 to ab70f7fCompareJuly 26, 2026 08:34
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Latest P1 is addressed in af6079f and ab70f7f, with the requested real FileAgentRunStore export/read/assert regression chain plus torn-tail fail-closed coverage. The branch is rebased onto current main; npm test, typecheck, lint, format:check, diff --check, and the completed independent review all pass. @Astro-Han, please re-review when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier trace-latch and torn-tail failures are fixed. I found two other ways the trace can still pass when the experiment is not comparable or complete; details inline.

Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/harbor-cell.ts Outdated
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

All four inline findings are addressed and resolved at c1917e8f:

  • trace-write header latch and torn-tail evidence are preserved through the real FileAgentRunStore export path;
  • protocol-independent hashes retain normalized reasoning effort and reject max/low/none mismatches;
  • missing evidence for an earlier continuation invocation now emits an identity-bearing diagnostic and fails closed.

The branch is clean, rebased onto current main, and the focused/full tests plus typecheck, lint, format, diff checks, and independent review pass. Please re-review the current head when convenient.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0 or P1 issue, so I am approving this PR. The earlier trace-integrity findings are fixed. I found these follow-ups:

  • [P2] A failed attempt append can still be missed if the first header-latch write also fails and later terminal writes recover.
  • [P2] Autonomous attempts from different sessions are exported into one file, but the reader treats the second session as mixed_identity.
  • [P2] Semantic-compaction provider calls bypass the request tracker.
  • [P2] Child-agent runs are outside the exported run set, so their provider calls are absent without a missing-evidence diagnostic.
  • [P3] Request-shape normalization repeats provider-option and output-limit rules owned by Runtime.
  • [P3] Headless still maintains a separate decoder for Runtime-owned attempt records.

These affect compound I/O failures, optional execution modes, or benchmark evidence rather than production state.

Please push back if provider-request traces intentionally exclude compaction or child-agent dispatches. If that is the intended boundary, narrowing the public contract may be enough. There is no need to accept a code change solely to clear the comment.

Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/provider-request-trace.ts
Comment threadpackages/headless/src/harbor-cell.ts
Comment threadpackages/runtime/src/request-shape.ts
Comment threadpackages/headless/src/provider-request-trace.ts
@Nyvo-io

Copy link
Copy Markdown
ContributorAuthor

Addressed the latest review in d73dc7c2:

  • the first trace-write failure is now latched in memory, header and sentinel writes are attempted independently, and the terminal header carries the latch;
  • autonomous retries across different sessions are accepted while run/turn identity conflicts still fail closed, with exact expectedIdentities validation available;
  • the public contract now explicitly covers exported top-level Harbor invocations only, so semantic compaction and child-agent dispatches are intentionally outside this artifact;
  • the two P3 ownership cleanups are left for the broader Runtime refactor instead of adding more duplication in this PR.

Focused tests, full Runtime/Headless suites, typecheck, lint, format, diff checks, and independent reviews pass. GitHub typecheck, test, and e2e are also green.

@Astro-Han
Astro-Han merged commit b308282 into apache:mainJul 28, 2026
3 checks passed
@Astro-Han

Copy link
Copy Markdown
Contributor

Reviewed d73dc7c2; merged as b308282d1.

The last commit fixes trace-write error latching and multi-session identity handling. The scope for compaction and child-agent calls is now explicit instead of being counted as complete provider accounting. I tested the merge with the then-current main; 544 focused tests passed. No new P0–P3.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@Nyvo-io@Astro-Han