Critical-review fixes, AGENTS.md/Agent Skills portability, and the cross-harness securability layer (v2.2.0) by Xcaciv · Pull Request #8 · Securability-Engineering/securability-engineering-capability

Xcaciv · 2026-08-25T14:51:13Z

Summary

Six commits taking the plugin from 2.0.0 to 2.2.0: correctness fixes found in a full critical review, conversion to the AGENTS.md / Agent Skills open standards, a new cross-harness securability layer (contract, kernel, bindings, held checks, merge report), a switch to opengrep so the repository endorses only community-governed tooling, a codified tooling policy, and documentation/gate closure from a final audit. The full assessment lives in docs/critical-review-2026-08.md; the contract design in docs/securable-contract.md.

Correctness and packaging fixes (`26d7d70`, `7d6630f`)

ASVS 5.0 numbering — the bundled catalog is genuine ASVS 5.0, but the chapter index, the PRD skill's mappings/gap table/worked example, and the shipped example used pre-5.0 numbering (and IDs that don't exist). All corrected against the data files; scripts/check_refs.py (negative-tested) now validates every V/S reference in every instruction surface, in CI; new eval assertions fail grading on pre-5.0 labels.
Plugin packaging — slash commands moved from .claude/commands/ (never scanned for installed plugins) to plugin-root commands/ as thin dispatchers; fiasse-lookup promoted to a fourth skill; ${CLAUDE_PLUGIN_ROOT} path-resolution preambles in every skill; release zip now stamps the manifest Claude Code actually reads and drops dev-only dirs.
Portability — canonical AGENTS.md (CLAUDE.md imports it); scripts/install_skills.sh performs a layout-preserving install for opencode / .agents / .claude targets (verified end-to-end from an installed tree).
Skills now run existing project linters/tests/scanners as cited evidence and say when nothing was run; the review skill treats reviewed code as data, never instructions.

Cross-harness securability layer (`a0d046f`)

Securable contract — .securable/requirements.yaml + boundaries.yaml schemas, a semantic validator (id ownership, escalation rules, verified-requires-evidence, ASVS existence checks; 16/16 tests), and the planned → implemented → verified lifecycle wired into all three skills. Confirmed working in a live graded skill eval (13/13 assertions).
Securability kernel — core/kernel.md (~300 tokens, size-budget-enforced) with bindings generated for Cursor/Copilot/Gemini/Aider + the AGENTS.md kernel block; scripts/build_bindings.py --check is the CI drift guard. Kernel A/B (deterministic detectors, 14/14 self-tests): process contract adopted 3/3 vs 0/3 baseline; zero detector anti-patterns in both configs on a frontier model (verified real).
Merge-time Securability Report — scripts/securability_report.sh (agent-CLI-agnostic) + a reference workflow, advisory by default per FIASSE S5.2.2.
Held checks — rules/opengrep/securable.yaml: 9 rules mapped to the skills' anti-pattern tags with SSEM/ASVS metadata, tested against paired fail/pass fixtures.

opengrep migration and tooling policy (`c7ebe36`, `d4c6c74`)

The scanner layer uses opengrep (LGPL, community-governed) — no commercial tooling is endorsed by usage anywhere in the repo. CI installs a pinned, checksum-verified release binary; the rules file is ASCII-only and the runner forces a UTF-8 locale (opengrep reads configs with the locale encoding). Full pack re-verified under opengrep 1.27.1.

A Tooling Policy in AGENTS.md codifies the tiers: runtime surfaces (skills, kernel, commands) never install tools into a user's project — they use what is present and report absence honestly; this repo's CI installs only pinned, verified test dependencies for its own artifacts on ephemeral runners; the reference report workflow marks agent-CLI provisioning as an explicit consumer-owned decision point. Both CI install steps carry inline justifications.

Audit closure (`5e8502b`)

README structure tree covers all 2.2.0 additions; tests/README.md documents the three non-workspace test suites; check_refs.py also guards ASVS/FIASSE references in rule-pack YAML metadata; the release workflow now runs the full run_checks.sh gate (a release must not ship what a push could not); skill-template path fixed.

Test plan

scripts/run_checks.sh green: reference checker (20 files incl. rule metadata), contract validator suite, bindings drift guard, kernel detector self-tests, manifest/YAML parsing, shell syntax, opengrep pack (9 rules / 16 fixtures)
Live regression: PRD workspace eval through the claude CLI, graded 13/13, including the new ASVS-numbering and contract-emission behavior
CI runs all of the above via the validate and opengrep-pack jobs

🤖 Generated with Claude Code

https://claude.ai/code/session_017LupEJCBDKT1xRyzNHiW4q

Copilot

Pull request overview

This PR advances the repository/plugin from v2.0.0 → v2.2.0 by incorporating critical-review correctness fixes, adopting AGENTS.md + Agent Skills conventions for portability, and adding a cross-harness “securability layer” (contract + kernel + held checks + merge-time report) with CI enforcement.

Changes:

Introduces the .securable contract (schemas + semantic validator + tests) and wires it into skills/workflows.
Adds the ~300-token securability kernel plus generated bindings and a drift guard.
Migrates held checks to an opengrep rule pack with fixtures and CI jobs; updates packaging/docs/commands accordingly.

Reviewed changes

Copilot reviewed 62 out of 62 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/securable-contract/test_validate.py	Adds validator regression tests (valid example + negative fixture mutations).
tests/prd-securability-enhancement-workspace/evals/evals.json	Tightens eval assertions and formatting; adds ASVS 5.0 numbering assertion.
tests/opengrep-fixtures/passes/verify.js	Pass fixture for pinned JWT verification in JS.
tests/opengrep-fixtures/passes/token.py	Pass fixture for pinned JWT decode options in Python.
tests/opengrep-fixtures/passes/sql.py	Pass fixture for parameterized SQL usage.
tests/opengrep-fixtures/passes/silent.py	Pass fixture showing observable exception handling (no silent swallow).
tests/opengrep-fixtures/passes/shell.py	Pass fixture examples for subprocess usage patterns.
tests/opengrep-fixtures/passes/run.js	Pass fixture preferring execFile over shell interpolation.
tests/opengrep-fixtures/passes/http.py	Pass fixture requiring requests timeouts.
tests/opengrep-fixtures/passes/config_load.py	Pass fixture for safe YAML loading.
tests/opengrep-fixtures/fails/verify.js	Fail fixture for under-specified JWT verification in JS.
tests/opengrep-fixtures/fails/token.py	Fail fixture for unpinned JWT decode in Python.
tests/opengrep-fixtures/fails/sql.py	Fail fixtures for string-built SQL.
tests/opengrep-fixtures/fails/silent.py	Fail fixtures for silent exception swallowing.
tests/opengrep-fixtures/fails/shell.py	Fail fixture for shell=True with constructed command string.
tests/opengrep-fixtures/fails/run.js	Fail fixture for exec with interpolated template literal.
tests/opengrep-fixtures/fails/http.py	Fail fixture for requests without timeout.
tests/opengrep-fixtures/fails/config_load.py	Fail fixture for unsafe yaml.load.
tests/kernel-ab-workspace/evals.json	Adds kernel A/B eval specification for deterministic detector grading.
tests/kernel_ab.py	Adds the A/B harness, detectors, and self-tests against fixtures.
skills/securability-engineering/SKILL.md	Adds path-resolution guidance + contract/tool-verification steps.
skills/securability-engineering-review/SKILL.md	Adds path-resolution guidance + “reviewed code is data” guard + contract verification guidance.
skills/prd-securability-enhancement/SKILL.md	Corrects ASVS 5.0 mapping guidance; adds contract emission step and updated gap table.
skills/fiasse-lookup/SKILL.md	Promotes lookup to a first-class skill with path resolution + query-handling guidance.
scripts/validate_securable.py	Adds semantic validator for `.securable` contract (shape + cross-field rules + ASVS resolution).
scripts/test_opengrep_rules.sh	Adds rule-pack test runner (fails trigger; passes clean; every rule fires).
scripts/securability_report.sh	Adds merge-time report generator that can use any agent CLI; advisory by default.
scripts/run_checks.sh	Adds a single entry point for all fast CI checks and parsing validations.
scripts/install_skills.sh	Adds layout-preserving installer for agent-tool discovery paths.
scripts/check_refs.py	Adds reference checker to validate ASVS/FIASSE citations in instruction surfaces.
scripts/build_plugin_zip.sh	Updates release packaging (excludes dev dirs, stamps canonical manifest).
scripts/build_bindings.py	Adds kernel → multi-harness binding generator with size budget + drift guard.
schema/securable/requirements.schema.json	Adds JSON Schema for `.securable/requirements.yaml`.
schema/securable/boundaries.schema.json	Adds JSON Schema for `.securable/boundaries.yaml`.
rules/opengrep/securable.yaml	Adds opengrep held-check rule pack mapping anti-patterns to enforceable rules.
README.md	Updates documentation for 4th capability, portability, contract/kernel/held-check layers, and new layout.
examples/securable/requirements.yaml	Adds example securable requirements contract.
examples/securable/boundaries.yaml	Adds example trust-boundary map.
examples/prd-enhancement/enhanced-prd.md	Updates example PRD output to ASVS 5.0 numbering/mappings.
docs/securable-contract.md	Documents the contract, lifecycle, and validation model.
docs/critical-review-2026-08.md	Adds critical assessment doc with findings and roadmap.
data/asvs/README.md	Corrects ASVS 5.0 chapter index and adds renumbering warning.
core/kernel.md	Adds the canonical securability kernel (source of truth for bindings).
commands/secure-generate.md	Adds plugin-root slash command dispatcher for generation skill.
commands/securability-review.md	Adds plugin-root slash command dispatcher for review skill.
commands/prd-securability-enhance.md	Adds plugin-root slash command dispatcher for PRD enhancement skill.
commands/fiasse-lookup.md	Adds plugin-root slash command dispatcher for lookup skill.
CLAUDE.md	Converts to thin stub importing AGENTS.md (avoids duplicated guidance).
bindings/gemini/GEMINI.md	Adds generated kernel binding for Gemini.
bindings/cursor/securable.mdc	Adds generated kernel binding for Cursor (alwaysApply).
bindings/copilot/copilot-instructions.md	Adds generated kernel binding for GitHub Copilot instructions.
bindings/aider/CONVENTIONS.md	Adds generated kernel binding for Aider.
AGENTS.md	Adds canonical agent entry point and embeds generated kernel block.
.github/workflows/securability-report.yml	Adds reference workflow for generating/posting securability reports.
.github/workflows/release.yml	Adds ASVS/FIASSE reference validation step to release workflow.
.github/workflows/ci.yml	Adds CI jobs for fast checks and pinned opengrep pack testing.
.claude/settings.json	Fixes dev permissions syntax.
.claude/commands/secure-generate.md	Removes project-only command file (migrated to plugin-root commands/).
.claude/commands/securability-review.md	Removes project-only command file (migrated to plugin-root commands/).
.claude/commands/prd-securability-enhance.md	Removes project-only command file (migrated to plugin-root commands/).
.claude-plugin/plugin.json	Bumps plugin version to 2.2.0.
.claude-plugin/marketplace.json	Bumps marketplace version to 2.2.0.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot · 2026-08-25T18:03:39Z

+const jwt = require("jsonwebtoken");
+
+function auth(token, secret) {
+  return jwt.verify(token, secret, { algorithms: ["HS256"], audience: "api" });
+}
+
+module.exports = { auth };


Fixed in 70c4ef9 by adding an explicit issuer to the JWT verify pass fixture.

Copilot · 2026-08-25T18:03:39Z

+def detect(name: str, text: str) -> bool:
+    if name == "requests_no_timeout":
+        return bool(re.search(r"\brequests\.(get|post|put|delete|head|request)\s*\(", text)) and "timeout" not in text
+    if name == "jwt_verify_unpinned":
+        return bool(re.search(r"\bjwt\.verify\s*\(", text)) and "algorithms" not in text
+    desc, pattern = DETECTORS[name]
+    assert pattern is not None, name
+    return bool(re.search(pattern, text))


Fixed in 70c4ef9 by scoping both special-case detectors to the matched call argument list (timeout= in requests.*(...) and algorithms: in jwt.verify(...)).

+# docs/ is intentionally not scanned: the critical-review document quotes
+# invalid references as findings and must be able to name them verbatim.
+SCAN_TARGETS = [
+    "AGENTS.md",
+    "README.md",
+    "core",
+    "skills",
+    "commands",
+    "plays",
+    "templates",
+    "examples",
+]
+
+ASVS_DIR = REPO / "data" / "asvs"
+FIASSE_DIR = REPO / "data" / "fiasse"
+
+# V6 | V6.3 | V6.3.8 — require a word boundary and not part of e.g. "V6.3.x"
+ASVS_REF = re.compile(r"\bV(\d{1,2})(?:\.(\d{1,2}))?(?:\.(\d{1,2}))?\b(?!\.[x\d])")
+# S4.4.1.2 / SA.4 / S8 — avoid matching things like "SP 800" or "SHA256"
+FIASSE_REF = re.compile(r"\bS(A)?\.?((?:\d{1,2})(?:\.\d{1,2}){0,3})\b")
+
+
+def iter_files() -> list[Path]:
+    files: list[Path] = []
+    for target in SCAN_TARGETS:
+        path = REPO / target
+        if path.is_file():
+            files.append(path)
+        elif path.is_dir():
+            files.extend(sorted(path.rglob("*.md")))
+    return files
+


…d checks Implements M1-M4 of the cross-harness strategy so the pack elevates any code-generation harness, carrying its impact in three layers: ambient (kernel), on-demand (skills), and repo-resident (contract + merge report). M1 — Securable contract (FIASSE S4.1 made durable): - schema/securable/: JSON Schemas for .securable/requirements.yaml and boundaries.yaml (per-feature requirements with ASVS 5.0 refs, testable acceptance criteria, planned->implemented->verified lifecycle; trust boundary map with authority sources) - scripts/validate_securable.py: semantic validator — id ownership, escalation rules for above-baseline levels, verified-requires-evidence, boundary cross-refs, and every ASVS reference resolved against the bundled catalog (degrades to warning without it) - tests/securable-contract/: valid example + 12 invalid mutations, each asserted to fail for its specific reason - Skills wired: PRD skill emits the contract (Step 7), generation honors it and flips planned->implemented only, review verifies implemented claims against acceptance criteria (a refuted claim is a finding) M2 — Securability kernel: - core/kernel.md: ~300-token always-on distillation with size budget - scripts/build_bindings.py: generates Cursor/Copilot/Gemini/Aider bindings and the AGENTS.md kernel block from the single source; --check mode is the CI drift guard; bindings are generated, never edited - tests/kernel_ab.py + workspace: A/B harness with deterministic anti-pattern detectors (self-tested against the semgrep fixtures). Measured on the claude CLI: baseline already avoided all detector anti-patterns on a frontier model (verified real); kernel's process contract adopted 3/3 (Securability Notes) vs 0/3 baseline M3 — Merge-time Securability Report (FIASSE S5.2.1): - scripts/securability_report.sh: agent-CLI-agnostic (claude/codex/ opencode), advisory by default per S5.2.2, dry-run testable - .github/workflows/securability-report.yml: reference workflow for consuming repos (workflow_dispatch-only here) M4 — Held checks under the instructions: - rules/semgrep/securable.yaml: 9 rules mapped to the anti-pattern tags with SSEM/ASVS metadata; paired fail/pass fixtures; test runner asserts every rule fires on its fixture and no clean fixture triggers Supporting: scripts/run_checks.sh one-shot suite; CI extended with the contract, bindings-drift, detector, and semgrep-pack checks; installer now ships schema/core/rules and the validator so installed trees are fully functional (verified end-to-end); docs/securable-contract.md; critical-review addendum; version 2.2.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017LupEJCBDKT1xRyzNHiW4q

The rule pack, fixtures, test runner, CI job, and all documentation now use opengrep (LGPL, community-governed) instead of a commercially licensed scanner — this repository should not endorse commercial tools, and usage is endorsement. The rule syntax is unchanged. - Rename rules/semgrep -> rules/opengrep, tests/semgrep-fixtures -> tests/opengrep-fixtures, scripts/test_semgrep_rules.sh -> scripts/test_opengrep_rules.sh (env var OPENGREP_BIN) - CI installs a pinned opengrep release binary with sha256 verification - Make the rules file ASCII-only and force a UTF-8 locale in the runner: opengrep reads configs with the locale encoding and fails on non-ASCII under C/POSIX locales - Full pack re-verified under opengrep 1.27.1: 9 rules, 8 fail fixtures all firing, 8 pass fixtures clean; detector self-tests and run_checks.sh green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017LupEJCBDKT1xRyzNHiW4q

Sanity check on tool installation, prompted by review: enumerate every install site and justify or reframe it. - Runtime surfaces (skills, kernel, commands): confirmed clean — they use tools already present, never install, and report absence honestly. Now stated as policy in AGENTS.md (Tooling Policy), not just in skill prose. - This repo's CI: the only installs are pinned test dependencies for this repo's own shipped artifacts (PyYAML; checksum-verified opengrep to prove the rule pack executes), on ephemeral runners — justification now inline in the workflow. - Reference securability-report workflow: the agent-CLI install is reframed as an explicit consumer-owned provisioning decision point, not a default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017LupEJCBDKT1xRyzNHiW4q

…gate - README project-structure tree now lists core/, bindings/, schema/, rules/, the new scripts, docs, and test surfaces added in 2.2.0 - tests/README.md documents the three non-workspace test suites (contract validator, opengrep fixtures, kernel A/B) and how to run them - check_refs.py also scans rules/**/*.yaml, so ASVS/FIASSE references in rule-pack metadata are guarded against drift like every other surface - Release workflow runs the full run_checks.sh gate instead of only the reference checker - a release must not ship what a push could not - Skill template's stale plays/tierN-category path replaced with the real category names Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017LupEJCBDKT1xRyzNHiW4q

Co-authored-by: Xcaciv <23732819+Xcaciv@users.noreply.github.com>

Copilot AI lite review requested due to automatic review settings August 25, 2026 14:51

Copilot started reviewing on behalf of Xcaciv August 25, 2026 14:51 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

Copilot started work on behalf of Xcaciv August 25, 2026 18:02 View session

claude and others added 6 commits August 25, 2026 18:08

Fix review thread issues in JWT fixture and detector scoping

df99db0

Co-authored-by: Xcaciv <23732819+Xcaciv@users.noreply.github.com>

Harden kernel AB detector argument matching

1e8fae3

Co-authored-by: Xcaciv <23732819+Xcaciv@users.noreply.github.com>

Copilot finished work on behalf of Xcaciv August 25, 2026 18:08

Xcaciv force-pushed the claude/securable-everywhere branch from 4438276 to 1e8fae3 Compare August 25, 2026 18:08

Xcaciv merged commit ad6ad11 into main Aug 25, 2026
4 checks passed

Xcaciv deleted the claude/securable-everywhere branch August 25, 2026 18:35

Navigation Menu

Uh oh!

Critical-review fixes, AGENTS.md/Agent Skills portability, and the cross-harness securability layer (v2.2.0) - #8

Critical-review fixes, AGENTS.md/Agent Skills portability, and the cross-harness securability layer (v2.2.0)#8
Xcaciv merged 6 commits into
mainfrom
claude/securable-everywhere

Xcaciv commented Aug 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 25, 2026

Uh oh!

Copilot AI Aug 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Xcaciv commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Correctness and packaging fixes (26d7d70, 7d6630f)

Cross-harness securability layer (a0d046f)

opengrep migration and tooling policy (c7ebe36, d4c6c74)

Audit closure (5e8502b)

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Xcaciv commented Aug 25, 2026 •

edited

Loading

Correctness and packaging fixes (`26d7d70`, `7d6630f`)

Cross-harness securability layer (`a0d046f`)

opengrep migration and tooling policy (`c7ebe36`, `d4c6c74`)

Audit closure (`5e8502b`)