fix: owasp_asi mapping audit (48 corrected, 13 confirmed, 11 flagged ambiguous) - #196
Merged
Merged
Conversation
…ASI Top 10 Audit triggered by two wrong mappings caught during D2 crosswalk verification on wg-security-and-privacy#8 (AVE-2026-00019 and AVE-2026-00038, both tagged against an outdated or incorrect reading of the canonical list). Full corpus checked against the primary-source OWASP Top 10 for Agentic Applications 2026 PDF (genai.owasp.org), including its own explicit differentiators between adjacent categories (e.g. ASI01 vs ASI06 vs ASI10; ASI02 vs ASI03 vs ASI05; ASI04 = compromised at the source vs ASI02 = compromised at runtime; ASI08 = propagation, not origin), not just category names. 70 of 80 records carry an owasp_asi tag. Of those: 13 confirmed correct as-is, 48 corrected (some by removing a wrong tag, some by also adding a confirmed-correct one), 11 left untouched as genuinely ambiguous rather than decided unilaterally. Two systemic patterns account for most of the corrections: - ASI08 (Cascading Failures) was applied to single-instance, non-propagating records; the category requires a fault to spread across agents/sessions/workflows with measurable fan-out, not just "this kind of thing could enable other attacks." - ASI07 (Insecure Inter-Agent Communication) was applied broadly to single-agent tool-misuse and supply-chain records with no other agent or inter-agent message involved at all. - ASI06 (Memory & Context Poisoning) was applied to one-time exfiltration/injection events; the category explicitly excludes one-time input prompts and requires persistent stored/retrievable context. AVE-2026-00019 and AVE-2026-00016, conversely, were missing ASI06 despite being clean matches. AVE-2026-00038 (excessive agency / unbounded tool use) is left unchanged and flagged ambiguous: ASI02, ASI09, and ASI10 are all defensible readings of the primary source depending on framing, and a prior comment on wg-security-and-privacy#8 speculated ASI09 was the "expected" value based on that WG's own gap-analysis framing rather than a direct primary-source check -- that speculation does not hold up as a confident correction on its own and is not applied here. AVE-2026-00063 loses its owasp_asi tag entirely rather than being reassigned: the record explicitly describes a static configuration weakness independent of any instruction text, which does not cleanly match any of the ten ASI categories.
chaksaray
added a commit
that referenced
this pull request
Aug 23, 2026
chaksaray
added a commit
that referenced
this pull request
Aug 23, 2026
Resolves the conflict on PR #199 (develop -> main). Only real conflict was dist/ave-records-latest.manifest.json's generated_at timestamp; records/ itself merged cleanly (main never touched the 48 files #196 corrected). dist/ave-records-latest.json auto-merged without conflict markers and was verified byte-identical to a fresh regeneration via scripts/build-records.js, so no silent divergence there. Manifest regenerated fresh the same way rather than hand-picking a timestamp.
This was referenced Aug 30, 2026
Merged
chaksaray
added a commit
that referenced
this pull request
Sep 4, 2026
…ework_sources Implements #257, after a deep re-check that found and fixed a real arithmetic error in that issue's own summary counts (23+22+21+8=74, not 80) and 11 additional corrections a fuller read of each record's full description (not just the truncated behavioral_fingerprint excerpt used for the first pass) resolved more precisely. Real final disposition, all 80 records: 39 corrected (applied here), 26 confirmed correct as-is (no change), 15 left unchanged with no confidently-better alternative (8 genuinely ambiguous between two defensible options, 7 where none of the 10 current MCP Top 10 categories covers the record's mechanism at all -- forcing a tag onto either group risked a worse error than leaving the existing one). 39 + 26 + 15 = 80. Two systemic patterns corrected, matching the shape #196's owasp_asi audit found: MCP01 (Token Mismanagement and Secret Exposure) was applied to 7+ records with no credential/token mechanism at all (shell injection, goal hijack, jailbreak, hidden instruction, dynamic tool-call injection, STDIO shell injection, zero-click auto-run); MCP10 (Context Injection and Over-Sharing) was applied to 10+ single-session records with no cross-session, cross-tenant, or persistent-sharing component, several of which are near-exact fits for MCP06 (Intent Flow Subversion) instead -- confirmed against AVE's own corpus, which already cross-references AVE-2026-00002/00016/00020/00028/00041/00043/00044/00065 as sharing one underlying mechanism ("the same missing content-versus-instruction boundary"), just realized at different delivery vectors; that grouping maps cleanly onto MCP03 (the tool's own declared surface, read at discovery) versus MCP06 (retrieved or received content, read in-flow) once checked against each category's real, current text. Corrections found and applied during the deep re-check that #257's first pass had marked ambiguous or missed: 00003 (drop MCP05, no command construction described), 00030/00061/00067 (a false-claim/trust-bypass mechanism is MCP07's own text almost verbatim in each case, not MCP02 or MCP05), 00038/00063 (MCP02 alone, dropping a weak secondary), 00041/00059 (MCP03 alone -- both explicitly happen at tool-discovery time per their own record text, not in retrieved context), 00044/00048/00052 (drop a weak secondary once the dominant real mechanism -- MCP06, MCP07, MCP05 respectively -- is identified), 00050 (MCP02's own "not declared in the manifest" language is a near-verbatim match, replacing a weak MCP04), 00076 (natural-language content steering a decision-making classifier is MCP03's territory, not MCP02's). One record (AVE-2026-00073) was upgraded from ambiguous to confirmed correct after the fuller text surfaced a concrete, cited case (CVE-2026-21852) where the described mechanism directly caused a real API-key leak, grounding the MCP01 tag rather than leaving it a stretch. framework_sources.owasp_mcp backfilled for all 80 records, pinned to OWASP/www-project-mcp-top-10 commit 165fe0f78ef104459237b4a8e0f6e78db9b02391 (2026-07-29, confirmed live via the API before use) and today's date (2026-09-05) -- this audit is exactly the dedicated, per-record verification issue #255 deferred this field pending. Found, not fixed here (separate, pre-existing, out of scope for this PR): crosswalks/ave-to-owasp-mcp.md claims in its own header to be 'generated directly from each record's owasp_mcp field' but has no generator script anywhere in the repo and was already stale before this change -- it covers only 56 of the corpus's 80 records. Flagging in the PR body rather than folding a second, unrelated repair into this one. Validated: python scripts/validate_records.py (80/80 valid), python scripts/check_fixtures.py (all pass), pytest tests/ -x -q (463 passed, unchanged from before this PR since no test asserts specific owasp_mcp values), python scripts/check_framework_sources.py (owasp_mcp no longer appears in any finding; only nist_ai_rmf and the 10 mitre_atlas records already deferred by #255 remain), node scripts/build-records.js (dist/ regenerated, diff is only the intended additions). Every touched record diffed by hand for both the tag corrections and the framework_sources insertions -- minimal, byte-exact changes, no unrelated reformatting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-corpus audit of
owasp_asitags against the current canonical OWASP Top 10 for Agentic Applications (2026), triggered by two wrong mappings found during D2 crosswalk work on wg-security-and-privacy#8.Report: 70 of 80 records carry an
owasp_asitag. Of those — 13 confirmed correct as-is, 48 corrected, 11 flagged ambiguous and left unchanged (not decided unilaterally).Canonical list re-confirmed directly from the primary-source PDF (genai.owasp.org/download/52117), not from memory of the prior D2-crosswalk session: ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation · ASI03 Identity and Privilege Abuse · ASI04 Agentic Supply Chain Vulnerabilities · ASI05 Unexpected Code Execution (RCE) · ASI06 Memory & Context Poisoning · ASI07 Insecure Inter-Agent Communication · ASI08 Cascading Failures · ASI09 Human-Agent Trust Exploitation · ASI10 Rogue Agents. Every correction below was checked against that document's own full descriptions and its explicit differentiators between adjacent categories (e.g. ASI01 vs ASI06 vs ASI10; ASI02 vs ASI03 vs ASI05; ASI04 = compromised at the source vs ASI02 = compromised at runtime; ASI08 = propagation of an existing fault, not its origin), not just the category names.
Two systemic patterns behind most corrections
Honesty checks
owasp_asitag entirely rather than being reassigned — the record explicitly describes a static configuration weakness "independent of any instruction text," which doesn't cleanly match any of the ten ASI categories. Per CLAUDE.md: omit rather than force a poor fit.All 80 records still validate against
schema/ave-record-1.1.0.schema.json, all have positive/negative fixtures, and the full pytest suite (339 tests) passes unchanged.