feat(ISSUE-263): W3-C3 llm-verifier 接入 (#279) - #72
Merged
Conversation
接入 llm-verifier（pip 钉点+lock）： - pipeline/adversary/llm_verifier.py：endpoint 三探测（logprobs/top_logprobs/ structured_outputs，fail-closed）+ K 次重复评估 + criteria 逐文件溯源 AC + token 账与 metering 交叉核对（偏差超阈值作废转人工） - pipeline/adversary/verifier-config.yaml：verifier 角色档与采样参数锁定 - pipeline/adversary/criteria/ISSUE-263/ac-criteria.yaml：示例 criteria 文件 - scripts/llm-verify.sh：CLI 入口（封装 llm_verifier.py） - .github/requirements-llm-verifier.txt：pip 钉点 pyyaml==6.0.3 与 W3-C1/W3-C5 接口兼容（输入输出 JSON schema 一致）。 Closes Cloudbird-Software/.github#279
randypanding
added a commit
that referenced
this pull request
Aug 26, 2026
对近一周（#21..#124）全部 PR 复盘后的机械债清理：仅删除 AST 级验证 「全仓零引用」的未用导入/未用名，不改任何判定逻辑、阈值、白名单或 policy 数据。逐文件出处： - pipeline/adversary/cnb_bridge.py：删未用 `from typing import Any`（#73/#74） - pipeline/adversary/golden_set.py：删未用 `from typing import Any`（#80/#82/#83） - pipeline/adversary/holdout_registry.py：删未用 `from typing import Any`（#81/#82） - pipeline/adversary/e2e/e2e-runner.py：删未用 `from typing import Any`（#89） - pipeline/adversary/llm_verifier.py：删未用 `import math`；可选库导入行去掉 未用名 extract_score（call_verifier/create_openai_client 均在用，保留）（#72/#76） - pipeline/entropy/tests/test_e2e.py：删未用 `import sys`（#56） - pipeline/selftest-c/tests/test_registry.py：删未用 `import copy`（#103） - pipeline/trust-gate/tests/test_adjudicate.py：删未用 `import copy`（#63） - pipeline/trust-gate/tests/test_cli.py：from-import 去掉未用名 PREDICATES/UNLOCK_STATE（保留 trust_gate 可导入性冒烟导入与 noqa 惯例）（#63） - scripts/dep-supply-chain-check.py：删未用 `import copy`（#36/#43） 刻意不动（已核验非死代码）：各模块 `from __future__ import annotations`； fuzz/sast/symbolic 的 `_yamlmini` 双模式导入守卫（noqa F401，保证包路径）； golden_set 等 try-import yaml 的环境 fail-closed 守卫；org-gate / suppression-gate / adversary-gate 等关卡 workflow 与 policy/suppressions.yaml 基线数据——门语义一概不变。 验证： - py_compile 全部 scripts/pipeline *.py 通过；bash -n 全部 *.sh 通过 - workflows/policy/pipeline 共 62 个 YAML 解析通过 - scripts/test-integrity-fixtures/run.sh、scripts/suppression-budget-selftest.sh 通过 - python -m unittest：trust-gate test_adjudicate+test_cli 17 例、 selftest-c tests.test_registry 14 例、entropy tests.test_e2e 10 例——全绿 Co-authored-by: randypanding <randypanding@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要
接入 llm-verifier（pip 钉点+lock）：criteria 一卡一文件且机器可追溯到卡 AC；endpoint 三探测每次 run 前执行（精度折损声明）；token 账挂接 ADR-0062 metering + usage 交叉核对（偏差超阈值 run 作废转人工）。
变更
pipeline/adversary/llm_verifier.py：endpoint 三探测（logprobs/top_logprobs/structured_outputs，不满足 fail-closed）+ K 次重复评估 + 阈值 gate + token 交叉核对pipeline/adversary/verifier-config.yaml：verifier 角色档与采样参数锁定（与 adversary-config.yaml 同构）pipeline/adversary/criteria/ISSUE-263/ac-criteria.yaml：示例 criteria 文件（AC-1/AC-2/AC-10/AC-11）scripts/llm-verify.sh：CLI 入口.github/requirements-llm-verifier.txt：pip 钉点 pyyaml==6.0.3接口兼容
与 W3-C1/W3-C5 接口兼容（输入输出 JSON schema 一致，verifier-report/v1）。
Closes Cloudbird-Software/.github#279