Skip to content

feat(B): 验收 DSL 编译器 + 骨架熵仪器(IR-0004 AC-8/AC-9 rev6,卡 .github#329/#330) - #100

Merged
randypanding merged 3 commits into
mainfrom
quality-instruments-b
Aug 25, 2026
Merged

feat(B): 验收 DSL 编译器 + 骨架熵仪器(IR-0004 AC-8/AC-9 rev6,卡 .github#329/#330)#100
randypanding merged 3 commits into
mainfrom
quality-instruments-b

Conversation

@randypanding

@randypanding randypanding commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

变更

  1. pipeline/dsl/(AC-8):compile.py(spec frontmatter AC→确定性 pytest 骨架,头部 spec-hash/逐 AC hash/regenerate 行)+ verify.py(三重校验:hash 重算+逐 AC 比对+应然内容逐字节重渲染——手改正文即使保留 hash 头也红)。与 g060/T-13 边界:hash 溯源管编译生成物、T-13 管一般测试文件、双命中从严者生效(ADR-0085 承接声明)。
  2. pipeline/entropy/divergence.py(AC-9 rev6 仪器化):四节骨架解析→3-gram Jaccard 相似度(参数版本化 v1/阈值 0.85,超阈=串通标记)→契约/路线分歧归类(关键词启发式+证据片段)→交集/并集/趋同度→fanout-products.jsonl 燃料产物(AC-13 契约)。与既有 ADR-0066 语义熵管线(cluster/examine/judge)同域并存、分工不重叠(DECISION-05 承接)。
  3. spec-dsl-gate.yml:specs/** PR 的可复用关卡(frontmatter 可解析+generated/ 存在则 verify)。

运行时证据

子代理自测 28 用例全绿(含对 IR-0004 真实 spec 的编译→verify 全链对拍、21 测试函数 pytest 实收、手改 hash 头不动也红的负向);集成后本地复跑 py_compile+YAML 通过。

依据

IR-0004 rev6(PR .github#359);ADR-0085 决策 7(门禁由 spec 自然携带、系统强制)。

Summary by CodeRabbit

  • 新功能

    • 新增可复用的 PR 规范校验流程,自动检查规范文件、验收条件及生成结果。
    • 支持对生成的测试骨架进行完整性与篡改检测,校验失败时阻止合并。
    • 新增骨架差异分析与报告输出,帮助识别内容趋同、契约分歧及测试覆盖差异。
  • 文档

    • 补充 DSL 编译、校验、分析工具的使用说明、验收标准及自测命令。

Cards: Cloudbird-Software/.github#329 Cloudbird-Software/.github#330

Copilot AI lite review requested due to automatic review settings August 25, 2026 00:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@randypanding
randypanding force-pushed the quality-instruments-b branch from bd7a8e2 to 7794a20 Compare August 25, 2026 00:51
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

Next included review available in 2 minutes.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 71107f2b-3b4a-4c2a-8146-4ec3718f58b2

📥 Commits

Reviewing files that changed from the base of the PR and between b480224 and ca59d3b.

📒 Files selected for processing (6)
  • .github/workflows/spec-dsl-gate.yml
  • pipeline/INSTRUMENTS-B.md
  • pipeline/dsl/__init__.py
  • pipeline/dsl/compile.py
  • pipeline/dsl/verify.py
  • pipeline/entropy/divergence.py
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2399d1d7-c470-4b17-987f-0797a2f822a8

📥 Commits

Reviewing files that changed from the base of the PR and between b480224 and ca59d3b.

📒 Files selected for processing (6)
  • .github/workflows/spec-dsl-gate.yml
  • pipeline/INSTRUMENTS-B.md
  • pipeline/dsl/__init__.py
  • pipeline/dsl/compile.py
  • pipeline/dsl/verify.py
  • pipeline/entropy/divergence.py

📝 Walkthrough

Walkthrough

Changes

该 PR 新增验收 DSL 编译与 hash 溯源校验工具、可复用 PR 门禁工作流,以及骨架方差分析工具。新增文档说明接口、职责边界、输出格式和自测命令。

DSL 编译与生成物校验

Layer / File(s) Summary
DSL 编译与生成物校验
pipeline/dsl/*, pipeline/INSTRUMENTS-B.md
解析 spec.mdacceptanceCriteria,生成确定性的 pytest 骨架,并校验 spec hash、AC hash 和完整渲染内容。

PR 工作流门禁

Layer / File(s) Summary
PR 工作流门禁
.github/workflows/spec-dsl-gate.yml, pipeline/INSTRUMENTS-B.md
新增可复用 workflow_call 工作流。工作流固定检测器来源,扫描变更的 spec 目录,并累计编译和生成物校验失败状态。

骨架方差分析与输出

Layer / File(s) Summary
骨架方差分析与输出
pipeline/entropy/divergence.py, pipeline/INSTRUMENTS-B.md
解析四节 Markdown 骨架,计算 3-gram Jaccard、章节分歧、测试交并集和假设并集,并输出 JSON 与 JSONL 产物。

Suggested labels: security, feature

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch quality-instruments-b

Comment @coderabbitai help to get the list of available commands.

Comment on lines +60 to +69
- name: Checkout 检测器(CI-Workflows 同源副本,只读)
if: github.event_name == 'pull_request' && inputs.detector-ref != ''
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: Cloudbird-Software/CI-Workflows
ref: ${{ inputs.detector-ref }}
path: detector
persist-credentials: false

- name: Setup Python 3.11(stdlib only——零 pip 依赖,无网络需求)
Comment thread .github/workflows/spec-dsl-gate.yml Fixed
Comment thread .github/workflows/spec-dsl-gate.yml Fixed
Comment thread .github/workflows/spec-dsl-gate.yml Fixed
Comment thread .github/workflows/spec-dsl-gate.yml Fixed
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add spec DSL compiler/verify gate and skeleton divergence instrument (IR-0004 AC-8/9)

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Compile spec.md frontmatter acceptanceCriteria into deterministic pytest skeletons with hash
 provenance.
• Add verify gate to detect any manual edits via spec-hash, per-AC hash, and byte-for-byte rerender.
• Introduce skeleton divergence instrument (3-gram Jaccard + divergence/fuel outputs) and reusable
 GH workflow.
Diagram

graph TD
  A["specs/<IR>/spec.md"] --> B["pipeline/dsl/compile.py"] --> C["generated pytest skeleton"] --> D["pipeline/dsl/verify.py"]
  E[".github/workflows/spec-dsl-gate.yml"] --> F["PR specs diff detector"] --> B
  F --> D
  G["skeletons/*.md"] --> H["pipeline/entropy/divergence.py"] --> I["divergence-report.json + fanout-products.jsonl"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Use full YAML parser (PyYAML/ruamel) for frontmatter
  • ➕ More robust YAML support (multiline, nested structures, edge cases)
  • ➕ Less custom parsing logic to maintain
  • ➖ Introduces third-party dependency/network/pip concerns that this repo explicitly avoids
  • ➖ Increases supply-chain surface area for a security gate tool
2. Only verify hashes (skip full rerender comparison)
  • ➕ Simpler verifier; less coupled to renderer
  • ➕ Fewer false negatives if header format evolves
  • ➖ Leaves a blind spot: manual edits that preserve header hashes could slip through
  • ➖ Weaker fail-closed integrity posture for generated artifacts
3. Fold skeleton divergence into existing semantic-entropy pipeline
  • ➕ Single entry point/tooling surface
  • ➕ Shared reporting conventions and output directories
  • ➖ Conflates two distinct purposes (multi-skeleton independence vs single-spec ambiguity)
  • ➖ Raises coupling/risk in an established pipeline; harder to keep optional/PM-driven

Recommendation: Current approach is the best fit for the stated constraints: stdlib-only, deterministic, and fail-closed. Keeping a strict deterministic renderer plus byte-for-byte rerender verification closes integrity gaps beyond hash headers, and separating divergence (multi-skeleton variance) from semantic entropy (single-spec ambiguity) preserves clear ownership and avoids coupling a new AC-9 instrument into ADR-0066’s pipeline.

Files changed (20) +2442 / -1557

Enhancement (5) +682 / -0
__init__.pyIntroduce pipeline.dsl package marker +2/-0

Introduce pipeline.dsl package marker

• Adds module docstring to define the acceptance DSL compiler and hash verifier package.

pipeline/dsl/init.py

compile.pyAdd deterministic spec→pytest skeleton compiler with provenance header +251/-0

Add deterministic spec→pytest skeleton compiler with provenance header

• Implements stdlib-only frontmatter subset parsing for acceptanceCriteria and renders deterministic pytest skeletons (one test per AC) with spec-hash, per-AC hash, and regenerate command. Adds --check mode delegating to verify for fail-closed CI validation.

pipeline/dsl/compile.py

verify.pyAdd triple-check verifier for generated skeleton integrity +126/-0

Add triple-check verifier for generated skeleton integrity

• Validates generated files by recomputing spec-hash, per-AC hashes, and performing a full byte-for-byte rerender comparison against compile.render. Emits actionable remediation guidance and aligns boundary semantics with existing test-integrity tooling.

pipeline/dsl/verify.py

__init__.pyIntroduce pipeline.entropy package marker +2/-0

Introduce pipeline.entropy package marker

• Adds module docstring indicating coexistence of the new divergence instrument with existing semantic entropy tooling.

pipeline/entropy/init.py

divergence.pyAdd skeleton divergence analyzer with similarity, classification, and fuel outputs +301/-0

Add skeleton divergence analyzer with similarity, classification, and fuel outputs

• Adds a deterministic analyzer that parses four required skeleton sections, computes pairwise 3-gram Jaccard similarity with versioned parameters and collusion thresholding, classifies contract/route divergences with evidence, and emits divergence-report.json plus fanout-products.jsonl records.

pipeline/entropy/divergence.py

Refactor (7) +962 / -962
cluster.pyNormalize semantic-entropy clustering module (no functional addition implied) +221/-221

Normalize semantic-entropy clustering module (no functional addition implied)

• Large churn with equal add/remove suggests reserialization/formatting or import/style normalization while preserving clustering semantics and zero-LLM constraints. Review should confirm no behavioral drift in clustering outputs.

pipeline/entropy/cluster.py

derive.pyNormalize semantic-entropy derivation module (no functional addition implied) +240/-240

Normalize semantic-entropy derivation module (no functional addition implied)

• Large churn with equal add/remove suggests formatting/docstring or mechanical normalization of the derivation orchestration and wrapper invocation logic. Review should confirm prompts, replay handling, and determinism constraints are unchanged.

pipeline/entropy/derive.py

examine.pyNormalize semantic-entropy examination module (no functional addition implied) +182/-182

Normalize semantic-entropy examination module (no functional addition implied)

• Large churn with equal add/remove suggests mechanical normalization of the cross-examination orchestration while keeping existing wrapper-based calling patterns. Review should confirm replay/live parity and output shapes remain consistent.

pipeline/entropy/examine.py

judge.pyNormalize semantic-entropy verdict module (no functional addition implied) +68/-68

Normalize semantic-entropy verdict module (no functional addition implied)

• Large churn with equal add/remove suggests formatting or small refactor while keeping verdict thresholds sourced from policy.py. Review should confirm attribution/noise-margin behavior remains intact.

pipeline/entropy/judge.py

nli_deberta.pyNormalize DeBERTa NLI adapter module +55/-55

Normalize DeBERTa NLI adapter module

• Large churn with equal add/remove suggests mechanical normalization; adapter remains the external entailment engine option and should continue to avoid default import surface expansion.

pipeline/entropy/nli_deberta.py

policy.pyNormalize semantic-entropy policy constants module +62/-62

Normalize semantic-entropy policy constants module

• Large churn with equal add/remove suggests docstring/style normalization; continues to be the single source of thresholds/config for zero-LLM judge/cluster path.

pipeline/entropy/policy.py

run.pyNormalize semantic-entropy end-to-end runner module +134/-134

Normalize semantic-entropy end-to-end runner module

• Large churn with equal add/remove suggests mechanical normalization while preserving the derive→cluster→judge→examine orchestration and report assertions.

pipeline/entropy/run.py

Documentation (1) +79 / -0
INSTRUMENTS-B.mdDocument DSL compiler/verify and divergence instrument usage +79/-0

Document DSL compiler/verify and divergence instrument usage

• Adds a dedicated B-instruments doc describing AC-8 DSL compilation + verification, AC-9 divergence instrument outputs, and how the reusable gate is wired. Captures boundaries with existing spec-check/T-13 and includes self-test invocation.

pipeline/INSTRUMENTS-B.md

Other (7) +719 / -595
spec-dsl-gate.ymlAdd reusable spec DSL gate workflow (compile + verify) +124/-0

Add reusable spec DSL gate workflow (compile + verify)

• Introduces a workflow_call gate that detects changed specs/<IR>/ directories in PRs, dry-runs DSL compilation, and conditionally verifies suite/generated/*.py with hash provenance checks. Enforces fail-closed detector pinning via detector-ref and uses stdlib-only Python 3.11 setup.

.github/workflows/spec-dsl-gate.yml

spec-ambiguous.mdReserialize ambiguous spec fixture +31/-31

Reserialize ambiguous spec fixture

• Fixture shows full-line replacement, consistent with newline/encoding normalization; content should remain equivalent for semantic-entropy tests.

pipeline/entropy/tests/fixtures/spec-ambiguous.md

test_cluster.pyReserialize clustering tests +119/-119

Reserialize clustering tests

• Full-file churn suggests formatting/normalization; verify asserts and expected cluster behavior remain the same.

pipeline/entropy/tests/test_cluster.py

test_e2e.pyReserialize end-to-end entropy pipeline tests +172/-172

Reserialize end-to-end entropy pipeline tests

• Full-file churn suggests mechanical normalization; verify that e2e report shapes and gating assertions are unchanged.

pipeline/entropy/tests/test_e2e.py

test_examine.pyReserialize cross-examination tests +88/-88

Reserialize cross-examination tests

• Full-file churn suggests formatting/normalization; confirm replay/live caller behavior assertions remain intact.

pipeline/entropy/tests/test_examine.py

test_judge.pyReserialize verdict tests +84/-84

Reserialize verdict tests

• Full-file churn suggests formatting/normalization; confirm noise margin and attribution logic expectations are unchanged.

pipeline/entropy/tests/test_judge.py

test_static_zero_llm.pyReserialize static zero-LLM import-surface test +101/-101

Reserialize static zero-LLM import-surface test

• Full-file churn suggests formatting/normalization; test continues to enforce import blacklist invariants for zero-LLM modules.

pipeline/entropy/tests/test_static_zero_llm.py

@qodo-code-review

qodo-code-review Bot commented Aug 25, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (6) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Indented AC lists rejected 🐞 Bug ≡ Correctness
Description
extract_acceptance_criteria() treats a new acceptance-criteria item as valid only when -  starts
in column zero, but the repository’s canonical YAML spec format nests list entries under
acceptanceCriteria: with two-space indentation. This causes acs to be parsed as empty and valid
specs to be rejected, making the new compiler/PR gate fail on existing specs that the YAML-based
validator accepts.
Code

pipeline/dsl/compile.py[R99-106]

+        if raw.startswith("- "):  # 新列表项(首字段随行)
+            cur = {}
+            acs.append(cur)
+            _fill(cur, raw[2:].strip(), source)
+        elif raw.startswith("  ") and cur is not None and ":" in raw:
+            _fill(cur, raw.strip(), source)
+        else:  # 下一个零缩进键 → 节终止
+            break
Relevance

●●● Strong

Deterministic indentation mismatch rejects the repo's canonical spec format, contradicting PR's own
compiler-gate intent.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The parser logic only recognizes list items when the raw line starts with the unindented pattern `-
 (i.e., raw.startswith("- ")`), so it fails to detect the repository’s normal YAML nested list
form where acceptance-criteria entries appear as   - id: ... under acceptanceCriteria: as shown
by the authoring template and a checked-in fixture. Because the parser then checks for an empty
acceptance-criteria list, this indentation mismatch becomes a compilation/gate failure, whereas the
canonical validator loads the YAML with yaml.safe_load and validates the resulting list without
requiring column-zero list syntax.

pipeline/dsl/compile.py[83-115]
pipeline/spec-template.md[8-15]
pipeline/adversary/fixtures/intent-backstop/clean-spec.md[1-15]
.github/workflows/spec-dsl-gate.yml[95-101]
pipeline/dsl/compile.py[83-116]
pipeline/entropy/tests/fixtures/spec-ambiguous.md[1-16]
scripts/spec-check.py[94-126]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The DSL parser rejects valid `acceptanceCriteria` lists because `extract_acceptance_criteria()` only recognizes list items when `- ` begins at column zero, while repository specs (including fixtures/templates) use standard YAML nesting with two-space-indented list entries under `acceptanceCriteria:`.

## Issue Context
Repository specs place acceptance-criteria items beneath the YAML key and indent them (e.g., `  - id: ...`). The new parser currently branches only on unindented list markers, which results in `acs` being empty and triggers the parser’s empty-list failure, causing the new compiler/PR gate to reject specs that the canonical YAML-based spec validator (using `yaml.safe_load`) accepts.

## Fix Focus Areas
- pipeline/dsl/compile.py[83-116]
- pipeline/adversary/fixtures/intent-backstop/clean-spec.md[6-10]
- scripts/spec-check.py[94-126]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Nested changes use wrong spec 🐞 Bug ≡ Correctness
Description
The workflow maps each changed file to its immediate parent and appends /spec.md, so a change to
specs/<IR>/suite/generated/test_*.py looks for specs/<IR>/suite/generated/spec.md instead of
specs/<IR>/spec.md. This causes generated-test changes to be rejected as “missing spec” before
compilation and the advertised provenance/hash verification can run.
Code

.github/workflows/spec-dsl-gate.yml[R89-93]

+          for d in $(printf '%s\n' "${CHANGED[@]}" | xargs -n1 dirname | sort -u); do
+            SPEC="$d/spec.md"
+            if [ ! -f "$SPEC" ]; then
+              echo "::error::$SPEC 不存在($d 有变更但缺 spec.md)"
+              FAIL=1; continue
Relevance

●●● Strong

Clear nested-path correctness bug in a new gate; similar path-completeness fixes to gate workflows
were accepted.

PR-#8

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The gate collects changed files under the specs root, then in its loop derives a directory d using
a single dirname and immediately checks for the existence of $d/spec.md, only afterward
considering generated-output paths like $d/$GEN_SUB. The compiler/verifier documentation indicates
that generated artifacts live nested under suite/generated while the authoritative spec.md
remains at the higher spec root (e.g., specs/<IR>/spec.md), so for changes inside
suite/generated the constructed lookup path (.../suite/generated/spec.md) is incorrect and
triggers the missing-spec failure before the build/provenance verification steps.

.github/workflows/spec-dsl-gate.yml[83-110]
pipeline/dsl/compile.py[21-25]
pipeline/dsl/verify.py[24-27]
specs/IR-E2E/README.md[1-2]
specs/IR-E2E/test/spec.md[1-20]
.github/workflows/spec-dsl-gate.yml[81-92]
.github/workflows/spec-dsl-gate.yml[103-110]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Changed files nested below a spec root are treated as though their immediate parent were the spec directory (by doing `dirname` + `/spec.md`), which fails for nested generated files and causes false missing-spec failures that prevent compilation and generated artifact provenance/hash verification.

## Issue Context
A valid generated output is located below `specs/<IR>/suite/generated/`, while its source of truth is `specs/<IR>/spec.md`. For each changed path, the workflow should locate the nearest owning ancestor containing `spec.md` under the `specs` directory (rather than blindly appending `spec.md` to `dirname`), and handle any grouping/metadata concerns separately from determining the owning spec directory.

## Fix Focus Areas
- .github/workflows/spec-dsl-gate.yml[83-110]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. AC ID breaks generated Python 🐞 Bug ☼ Reliability
Description
The raw AC ID is inserted into a triple-quoted docstring without escaping, although the other AC
text is escaped. A valid YAML string such as id: 'AC-1"""' passes the DSL field checks but
generates syntactically invalid Python; verify.py re-renders the same invalid text and reports it
green.
Code

pipeline/dsl/compile.py[R159-164]

+        body.append(f"def {fname}():")
+        body.append(f'    """{ac["id"]}')
+        body.append("")
+        body.append(f"    given: {_esc_doc(ac['given'])}")
+        body.append(f"    when: {_esc_doc(ac['when'])}")
+        body.append(f"    then: {_esc_doc(ac['then'])}")
Relevance

●●● Strong

Unescaped trusted field can break generated Python while verify.py still reports green; a real
reliability bug.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The compiler stores IDs without a character restriction, function_name() sanitizes only the
function identifier, and render() emits the original ID without _esc_doc(). The repository’s
structural checker accepts every non-empty string ID, and verification compares only against the
same renderer rather than compiling the generated module.

pipeline/dsl/compile.py[119-125]
pipeline/dsl/compile.py[134-147]
pipeline/dsl/compile.py[159-166]
scripts/spec-check.py[111-126]
pipeline/dsl/verify.py[57-98]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Raw AC IDs are emitted inside Python triple-quoted docstrings and can terminate the literal, producing invalid generated test files.

## Issue Context
The existing spec validator only requires an ID to be a non-empty string, and the compiler already has an escaping helper for the other docstring fields.

## Fix Focus Areas
- pipeline/dsl/compile.py[119-125]
- pipeline/dsl/compile.py[145-166]
- scripts/spec-check.py[111-126]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


4. Append-only records are truncated 🐞 Bug ☼ Reliability
Description
write_outputs() opens fanout-products.jsonl in "w" mode, deleting all existing fuel records
whenever the output directory is reused. This contradicts the stated append-only contract and loses
prior divergence and assumption records.
Code

pipeline/entropy/divergence.py[R260-262]

+    with open(out / "fanout-products.jsonl", "w", encoding="utf-8", newline="\n") as f:
+        for rec in fanout_records(report, card_id, spec_hash, base_sha):
+            f.write(json.dumps(rec, ensure_ascii=False) + "\n")
Relevance

●●● Strong

Opening an explicitly append-only JSONL log in write mode deterministically destroys prior records;
clear reliability bug.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Both the module and instrument documentation identify the JSONL fuel product as append-only, but the
implementation opens it with the truncating write mode before emitting current records.

pipeline/entropy/divergence.py[21-24]
pipeline/entropy/divergence.py[255-262]
pipeline/INSTRUMENTS-B.md[46-48]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Every run truncates the append-only fanout product file and destroys records produced by earlier runs.

## Issue Context
Write new JSONL records using append-safe behavior consistent with the contract, while retaining deterministic overwrite behavior for the standalone report if desired.

## Fix Focus Areas
- pipeline/entropy/divergence.py[255-262]
- pipeline/INSTRUMENTS-B.md[46-48]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Single skeleton crashes CLI 🐞 Bug ☼ Reliability
Description
A directory containing exactly one valid skeleton passes the non-empty directory validation but
produces no pair combinations, causing the mean pair similarity (mean_sim/mean_pair_similarity)
calculation to divide by zero. Because the CLI only catches EntropyError, this otherwise valid
input triggers an uncaught ZeroDivisionError traceback instead of a controlled report or input
error consistent with documented exit behavior.
Code

pipeline/entropy/divergence.py[214]

+    mean_sim = round(sum(p["similarity"] for p in pairs) / len(pairs), 4)
Relevance

●●● Strong

Valid single-skeleton input deterministically raises ZeroDivisionError, escaping the documented CLI
error handling.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The cited logic rejects only the case where the directory has zero .md files, so a directory with
one valid skeleton is accepted; however, pair generation uses itertools.combinations(..., 2),
which yields an empty set of pairs when there is only one name. The mean similarity computation then
divides by the length of the empty list (noted at line 214 as len(pairs)), producing a
ZeroDivisionError. Since the CLI exception handling is limited to EntropyError, this arithmetic
exception is not intercepted and escapes as a traceback rather than a controlled outcome.

pipeline/entropy/divergence.py[161-175]
pipeline/entropy/divergence.py[213-214]
pipeline/entropy/divergence.py[282-287]
pipeline/entropy/divergence.py[156-173]
pipeline/INSTRUMENTS-B.md[35-48]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A directory with exactly one valid skeleton currently passes input validation but generates zero pairwise combinations, leading the mean pair similarity calculation to divide by zero and crash with an uncaught `ZeroDivisionError`.

## Issue Context
The tool’s directory validation only rejects directories with zero `.md` files, but pairwise metrics inherently require at least two skeleton inputs. The CLI currently catches only `EntropyError`, so the division-by-zero escapes and violates the expected/documented controlled CLI exit behavior; the fix should either reject fewer than two skeletons via `EntropyError` or define an explicit, intentional behavior for the no-pairs case (and ensure it is covered by a CLI test).

## Fix Focus Areas
- pipeline/entropy/divergence.py[156-173]
- pipeline/entropy/divergence.py[213-214]
- pipeline/entropy/divergence.py[282-287]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

6. Malformed headings accepted 🐞 Bug ≡ Correctness
Description
The documented skeleton grammar requires a space after the heading hashes, but _HEADING makes that
whitespace optional. Consequently malformed lines such as ##路线陈述 satisfy the required-section
validation instead of being rejected as invalid input.
Code

pipeline/entropy/divergence.py[90]

+_HEADING = re.compile(r"^\s*#{1,6}\s*(\S+)\s*$")
Relevance

●●● Strong

Regex permissiveness contradicts documented heading grammar, deterministically accepting malformed
headings.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The module documentation defines headings as hashes followed by a space and section name, while the
parser regex uses \s*; parsed headings are then recorded as present for the mandatory-section
check.

pipeline/entropy/divergence.py[5-7]
pipeline/entropy/divergence.py[90-123]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The skeleton heading regex accepts headings that violate the documented required-space grammar.

## Issue Context
Parser recognition controls the required-section validation, so accepting malformed headings prevents fail-closed input validation.

## Fix Focus Areas
- pipeline/entropy/divergence.py[90-123]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: 🧠 Deep: This is a dense, cross-cutting change with new DSL compilation/verification, a new entropy instrument, workflow gating, and broad edits across multiple independent code paths where redundant review could catch subtle defects.

Grey Divider

Tip of the day
💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment on lines +89 to +93
for d in $(printf '%s\n' "${CHANGED[@]}" | xargs -n1 dirname | sort -u); do
SPEC="$d/spec.md"
if [ ! -f "$SPEC" ]; then
echo "::error::$SPEC 不存在($d 有变更但缺 spec.md)"
FAIL=1; continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Nested changes use wrong spec 🐞 Bug ≡ Correctness

The workflow maps each changed file to its immediate parent and appends /spec.md, so a change to
specs/<IR>/suite/generated/test_*.py looks for specs/<IR>/suite/generated/spec.md instead of
specs/<IR>/spec.md. This causes generated-test changes to be rejected as “missing spec” before
compilation and the advertised provenance/hash verification can run.
Agent Prompt
## Issue description
Changed files nested below a spec root are treated as though their immediate parent were the spec directory (by doing `dirname` + `/spec.md`), which fails for nested generated files and causes false missing-spec failures that prevent compilation and generated artifact provenance/hash verification.

## Issue Context
A valid generated output is located below `specs/<IR>/suite/generated/`, while its source of truth is `specs/<IR>/spec.md`. For each changed path, the workflow should locate the nearest owning ancestor containing `spec.md` under the `specs` directory (rather than blindly appending `spec.md` to `dirname`), and handle any grouping/metadata concerns separately from determining the owning spec directory.

## Fix Focus Areas
- .github/workflows/spec-dsl-gate.yml[83-110]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +260 to +262
with open(out / "fanout-products.jsonl", "w", encoding="utf-8", newline="\n") as f:
for rec in fanout_records(report, card_id, spec_hash, base_sha):
f.write(json.dumps(rec, ensure_ascii=False) + "\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Append-only records are truncated 🐞 Bug ☼ Reliability

write_outputs() opens fanout-products.jsonl in "w" mode, deleting all existing fuel records
whenever the output directory is reused. This contradicts the stated append-only contract and loses
prior divergence and assumption records.
Agent Prompt
## Issue description
Every run truncates the append-only fanout product file and destroys records produced by earlier runs.

## Issue Context
Write new JSONL records using append-safe behavior consistent with the contract, while retaining deterministic overwrite behavior for the standalone report if desired.

## Fix Focus Areas
- pipeline/entropy/divergence.py[255-262]
- pipeline/INSTRUMENTS-B.md[46-48]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread pipeline/dsl/compile.py
Comment on lines +99 to +106
if raw.startswith("- "): # 新列表项(首字段随行)
cur = {}
acs.append(cur)
_fill(cur, raw[2:].strip(), source)
elif raw.startswith(" ") and cur is not None and ":" in raw:
_fill(cur, raw.strip(), source)
else: # 下一个零缩进键 → 节终止
break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Indented ac lists rejected 🐞 Bug ≡ Correctness

extract_acceptance_criteria() treats a new acceptance-criteria item as valid only when -  starts
in column zero, but the repository’s canonical YAML spec format nests list entries under
acceptanceCriteria: with two-space indentation. This causes acs to be parsed as empty and valid
specs to be rejected, making the new compiler/PR gate fail on existing specs that the YAML-based
validator accepts.
Agent Prompt
## Issue description
The DSL parser rejects valid `acceptanceCriteria` lists because `extract_acceptance_criteria()` only recognizes list items when `- ` begins at column zero, while repository specs (including fixtures/templates) use standard YAML nesting with two-space-indented list entries under `acceptanceCriteria:`.

## Issue Context
Repository specs place acceptance-criteria items beneath the YAML key and indent them (e.g., `  - id: ...`). The new parser currently branches only on unindented list markers, which results in `acs` being empty and triggers the parser’s empty-list failure, causing the new compiler/PR gate to reject specs that the canonical YAML-based spec validator (using `yaml.safe_load`) accepts.

## Fix Focus Areas
- pipeline/dsl/compile.py[83-116]
- pipeline/adversary/fixtures/intent-backstop/clean-spec.md[6-10]
- scripts/spec-check.py[94-126]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

as_union = sorted({it for m in as_maps for it in m.values()})

convergence = round(100.0 * len(inter_keys) / len(union_keys), 2) if union_keys else 0.0
mean_sim = round(sum(p["similarity"] for p in pairs) / len(pairs), 4)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

4. Single skeleton crashes cli 🐞 Bug ☼ Reliability

A directory containing exactly one valid skeleton passes the non-empty directory validation but
produces no pair combinations, causing the mean pair similarity (mean_sim/mean_pair_similarity)
calculation to divide by zero. Because the CLI only catches EntropyError, this otherwise valid
input triggers an uncaught ZeroDivisionError traceback instead of a controlled report or input
error consistent with documented exit behavior.
Agent Prompt
## Issue description
A directory with exactly one valid skeleton currently passes input validation but generates zero pairwise combinations, leading the mean pair similarity calculation to divide by zero and crash with an uncaught `ZeroDivisionError`.

## Issue Context
The tool’s directory validation only rejects directories with zero `.md` files, but pairwise metrics inherently require at least two skeleton inputs. The CLI currently catches only `EntropyError`, so the division-by-zero escapes and violates the expected/documented controlled CLI exit behavior; the fix should either reject fewer than two skeletons via `EntropyError` or define an explicit, intentional behavior for the no-pairs case (and ensure it is covered by a CLI test).

## Fix Focus Areas
- pipeline/entropy/divergence.py[156-173]
- pipeline/entropy/divergence.py[213-214]
- pipeline/entropy/divergence.py[282-287]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread pipeline/dsl/compile.py
Comment on lines +159 to +164
body.append(f"def {fname}():")
body.append(f' """{ac["id"]}')
body.append("")
body.append(f" given: {_esc_doc(ac['given'])}")
body.append(f" when: {_esc_doc(ac['when'])}")
body.append(f" then: {_esc_doc(ac['then'])}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

5. Ac id breaks generated python 🐞 Bug ☼ Reliability

The raw AC ID is inserted into a triple-quoted docstring without escaping, although the other AC
text is escaped. A valid YAML string such as id: 'AC-1"""' passes the DSL field checks but
generates syntactically invalid Python; verify.py re-renders the same invalid text and reports it
green.
Agent Prompt
## Issue description
Raw AC IDs are emitted inside Python triple-quoted docstrings and can terminate the literal, producing invalid generated test files.

## Issue Context
The existing spec validator only requires an ID to be a non-empty string, and the compiler already has an escaping helper for the other docstring fields.

## Fix Focus Areas
- pipeline/dsl/compile.py[119-125]
- pipeline/dsl/compile.py[145-166]
- scripts/spec-check.py[111-126]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@randypanding
randypanding merged commit d443731 into main Aug 25, 2026
23 of 24 checks passed
@randypanding
randypanding deleted the quality-instruments-b branch August 25, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants