Skip to content

test(dynamic-workflow): mechanical claims table and reflection formation example - #45

Open
modacker wants to merge 2 commits into
MiniMax-AI:mainfrom
modacker:community/claims-and-examples
Open

modacker wants to merge 2 commits into
MiniMax-AI:mainfrom
modacker:community/claims-and-examples

Conversation

@modacker

@modacker modacker commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

What changes

  1. Mechanical claims table (VERIFICATION.md + scripts/verify-claims.mjs): the claims a machine can re-check are tabulated in a ```verify fenced block. The runner executes each row via /bin/sh -c, prints one PASS/FAIL line per claim, and exits 0 only when every row matches its expected exit status (1 on the first mismatch, 2 when the tool itself cannot run). Rows needing development dependencies say so (npm ci first); non-mechanical claims (dashboard acceptance, real-agent calls, platform coverage) intentionally stay prose.
  2. Reflection formation example (examples/reflection.js): draft → independent critique → revision, three phases with separate agents, a checkpoint snapshot of the draft, and explicit open-issue handling — styled after examples/audit.js.

Verification

  • node scripts/verify-claims.mjs: 4/4 PASS on a committed tree. V-04 failing on a dirty tree is the feature: it detects drifted committed assets.
  • examples/reflection.js passes validateScript
  • Repository npm run check: green

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@modacker

Copy link
Copy Markdown
Contributor Author

Related: #47 (zh/en term-consistency audit) — same spirit as the claims table here, extending mechanical checks from behavior to documentation. No action needed on this PR.

@modacker
modacker force-pushed the community/claims-and-examples branch from a950f42 to 8d4a56e Compare September 18, 2026 01:57

@hetaoBackend hetaoBackend left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes for exact current head 8d4a56e.

Blocking issues:

  1. The PR changes the maintained plugins/hetaoBackend/mcode-dynamic-workflows/ tree, while the PR author/commits are modacker / moc <moc@sgmov.com> and the manifests still identify hetaoBackend as owner (plugin.json:6, .claude-plugin/plugin.json:5). Under CONTRIBUTING.md:28,80, provide explicit owner authorization or move the change to the contributor's own plugin scope before merge.
  2. scripts/verify-claims.mjs:10-17 extracts Markdown cells and passes each cell directly to /bin/sh -c. This turns a modified verification document into arbitrary local command execution, with no allowlist for commands/argv or shell metacharacters. Replace it with structured, fixed commands/argv (or an explicit documented trusted-only policy plus strict validation); do not use an unrestricted shell for a claims verifier.
  3. The parser only collects rows matching one regex (verify-claims.mjs:12) and silently ignores malformed/unexpected/duplicate rows. It can exit 0 after verifying a subset while claiming “every row”. Validate the header, exact expected IDs/order, uniqueness, columns, and that every table row is consumed; add focused parser tests.
  4. The new verifier and examples/reflection.js are not executed by the current CI workflow. The green checks cover the existing 73 source checks and package test, but not node scripts/verify-claims.mjs or validateScript on the reflection example. Add both checks to CI and provide exact-head output.
  5. examples/reflection.js:12-15 gives the revision agent the draft and critique but not the original task/material passed to the critic. The runtime contract says agents are self-contained and do not inherit the parent conversation; the revision cannot independently verify the critique evidence. Pass the original material (or a bounded equivalent) to revision and add a regression assertion.

The current [code]smith check is SKIPPED and is not evidence. Reconcile the ownership issue and add the missing executable CI evidence before approval.

- VERIFICATION.md gains a verify-fenced claims table; the new
  scripts/verify-claims.mjs executes each row and exits 0 only when all
  match (1 first mismatch, 2 tool failure)
- examples/reflection.js: draft -> independent critique -> revision
  formation with a checkpoint snapshot, styled after audit.js
@modacker
modacker force-pushed the community/claims-and-examples branch from 8d4a56e to b3b8acc Compare September 18, 2026 03:26
@modacker

Copy link
Copy Markdown
Contributor Author

Note on the failing validate (ubuntu-latest): the two failures (new-branch attribution considers baselines beyond the first 256 tips, a workspace fetch makes commit attribution explicitly unavailable) are both in cli-agent-bridge's git-attribution tests — the same intermittent family documented with three runs of evidence in #43 (comment). This PR touches only dynamic-workflow docs/examples and a verification script; zero overlap with the failing plugin. Evidence thread on #43 has the details.

@modacker

Copy link
Copy Markdown
Contributor Author

Thank you for the exact-head review — all five blockers are actionable. Point-by-point:

  1. Ownership (CONTRIBUTING.md:28,80): fix(dynamic-workflow): deterministic wait polling and checkpoint lineage in repair reuse #44 — same maintained plugin tree, same commit authorship, reviewed against exact head and approved+merged — was our basis for treating contributor fixes to mcode-dynamic-workflows as authorized. If that reading is wrong: this is the explicit request for the owner's authorization for this PR's changes to your plugin scope (and for feat(dynamic-workflow): tamper-evident integrity ledger for repair_cache and events #48/feat(dynamic-workflow): opt-in content-addressed cross-run reuse #49, already under your review). If you prefer a different authorization form, say the word.
  2. verify-claims shell execution: agreed — /bin/sh -c over document cells is an execution hazard. Reworking to your preferred first shape: claims as fixed argv data in the script (no shell, no markdown on the execution path; node resolved to process.execPath), with the markdown table demoted to a validated mirror (a consistency test asserts the mirror equals the executable claims — malformed or smuggled rows fail loudly instead of being silently skipped).
  3. Parser strictness: subsumed by (2); the mirror-consistency test covers exact ID/column/uniqueness/consumption parity.
  4. CI coverage: we cannot push .github/workflows/** (OAuth token lacks workflow scope). Two options — (a) maintainer adds two steps (snippet will be in the rework: node scripts/verify-claims.mjs and an examples demo-smoke); or (b) we complete a one-time workflow-scope authorization and add them ourselves. Exact-head outputs of both commands will be posted on this PR with the rework.
  5. reflection.js: agreed — agents are self-contained; the revision node will receive the original task/material alongside draft+critique, the example will be made demo-executor-safe, and a demo-mode end-to-end regression check is being added.

Rework incoming on this branch; will re-request review against the exact new head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants