Skip to content

fix: owasp_asi mapping audit (48 corrected, 13 confirmed, 11 flagged ambiguous) - #196

Merged
chaksaray merged 1 commit into
developfrom
fix/owasp-asi-mapping-audit
Aug 23, 2026
Merged

fix: owasp_asi mapping audit (48 corrected, 13 confirmed, 11 flagged ambiguous)#196
chaksaray merged 1 commit into
developfrom
fix/owasp-asi-mapping-audit

Conversation

@chaksaray

Copy link
Copy Markdown
Contributor

Full-corpus audit of owasp_asi tags against the current canonical OWASP Top 10 for Agentic Applications (2026), triggered by two wrong mappings found during D2 crosswalk work on wg-security-and-privacy#8.

Report: 70 of 80 records carry an owasp_asi tag. Of those — 13 confirmed correct as-is, 48 corrected, 11 flagged ambiguous and left unchanged (not decided unilaterally).

Canonical list re-confirmed directly from the primary-source PDF (genai.owasp.org/download/52117), not from memory of the prior D2-crosswalk session: ASI01 Agent Goal Hijack · ASI02 Tool Misuse and Exploitation · ASI03 Identity and Privilege Abuse · ASI04 Agentic Supply Chain Vulnerabilities · ASI05 Unexpected Code Execution (RCE) · ASI06 Memory & Context Poisoning · ASI07 Insecure Inter-Agent Communication · ASI08 Cascading Failures · ASI09 Human-Agent Trust Exploitation · ASI10 Rogue Agents. Every correction below was checked against that document's own full descriptions and its explicit differentiators between adjacent categories (e.g. ASI01 vs ASI06 vs ASI10; ASI02 vs ASI03 vs ASI05; ASI04 = compromised at the source vs ASI02 = compromised at runtime; ASI08 = propagation of an existing fault, not its origin), not just the category names.

Two systemic patterns behind most corrections

  • ASI08 (Cascading Failures) over-applied to single-instance records. The category requires a fault to spread across agents, sessions, or workflows with measurable fan-out — not merely "this could enable other attacks." Most current ASI08 tags sat on records describing a single, contained event with no propagation mechanism described (e.g. a goal hijack, a jailbreak, a false permission claim) — the record was the origin, not the cascade.
  • ASI07 (Insecure Inter-Agent Communication) over-applied to single-agent tool misuse. The category is specifically about compromised real-time messages between agents. Most current ASI07 tags sat on single-agent tool-abuse or supply-chain records with no other agent or inter-agent message involved at all.
  • ASI06 (Memory & Context Poisoning) applied to one-time events. The category explicitly excludes one-time input prompts and requires persistent stored/retrievable context (memory, RAG, embeddings across sessions). Several exfiltration records were mistagged ASI06 for a single-request action; conversely AVE-2026-00019 (the record that started this audit) and AVE-2026-00016 were missing ASI06 despite being clean, even textbook, matches.

Honesty checks

  • AVE-2026-00038 (excessive agency / unbounded tool use) is left unchanged and flagged ambiguous. ASI02, ASI09, and ASI10 are all defensible readings of the primary source depending on framing. A comment I posted on wg-security-and-privacy#8 speculated ASI09 was the "expected" value based on that WG's own gap-analysis framing (D2) rather than a direct primary-source check — having now read the full primary-source descriptions, that speculation doesn't hold up as a confident correction, so it isn't applied here.
  • AVE-2026-00063 loses its owasp_asi tag entirely rather than being reassigned — the record explicitly describes a static configuration weakness "independent of any instruction text," which doesn't cleanly match any of the ten ASI categories. Per CLAUDE.md: omit rather than force a poor fit.
  • 11 records are left with a tag I couldn't confidently confirm or reject (e.g. AVE-2026-00047's hardcoded-credential mechanism doesn't cleanly fit ASI02 or ASI06; AVE-2026-00071's daemon-redirect config issue loosely fits both ASI04 and ASI05 without a clean primary). Flagged for a second read rather than decided here.

All 80 records still validate against schema/ave-record-1.1.0.schema.json, all have positive/negative fixtures, and the full pytest suite (339 tests) passes unchanged.

…ASI Top 10

Audit triggered by two wrong mappings caught during D2 crosswalk
verification on wg-security-and-privacy#8 (AVE-2026-00019 and
AVE-2026-00038, both tagged against an outdated or incorrect reading
of the canonical list). Full corpus checked against the primary-source
OWASP Top 10 for Agentic Applications 2026 PDF (genai.owasp.org),
including its own explicit differentiators between adjacent categories
(e.g. ASI01 vs ASI06 vs ASI10; ASI02 vs ASI03 vs ASI05; ASI04 =
compromised at the source vs ASI02 = compromised at runtime; ASI08 =
propagation, not origin), not just category names.

70 of 80 records carry an owasp_asi tag. Of those: 13 confirmed
correct as-is, 48 corrected (some by removing a wrong tag, some by
also adding a confirmed-correct one), 11 left untouched as genuinely
ambiguous rather than decided unilaterally.

Two systemic patterns account for most of the corrections:
- ASI08 (Cascading Failures) was applied to single-instance,
  non-propagating records; the category requires a fault to spread
  across agents/sessions/workflows with measurable fan-out, not just
  "this kind of thing could enable other attacks."
- ASI07 (Insecure Inter-Agent Communication) was applied broadly to
  single-agent tool-misuse and supply-chain records with no other
  agent or inter-agent message involved at all.
- ASI06 (Memory & Context Poisoning) was applied to one-time
  exfiltration/injection events; the category explicitly excludes
  one-time input prompts and requires persistent stored/retrievable
  context. AVE-2026-00019 and AVE-2026-00016, conversely, were missing
  ASI06 despite being clean matches.

AVE-2026-00038 (excessive agency / unbounded tool use) is left
unchanged and flagged ambiguous: ASI02, ASI09, and ASI10 are all
defensible readings of the primary source depending on framing, and a
prior comment on wg-security-and-privacy#8 speculated ASI09 was the
"expected" value based on that WG's own gap-analysis framing rather
than a direct primary-source check -- that speculation does not hold
up as a confident correction on its own and is not applied here.

AVE-2026-00063 loses its owasp_asi tag entirely rather than being
reassigned: the record explicitly describes a static configuration
weakness independent of any instruction text, which does not cleanly
match any of the ten ASI categories.
@chaksaray
chaksaray merged commit b5e33bd into develop Aug 23, 2026
6 checks passed
@chaksaray
chaksaray deleted the fix/owasp-asi-mapping-audit branch August 23, 2026 08:43
chaksaray added a commit that referenced this pull request Aug 23, 2026
Resolves the conflict on PR #199 (develop -> main). Only real conflict
was dist/ave-records-latest.manifest.json's generated_at timestamp;
records/ itself merged cleanly (main never touched the 48 files #196
corrected). dist/ave-records-latest.json auto-merged without conflict
markers and was verified byte-identical to a fresh regeneration via
scripts/build-records.js, so no silent divergence there. Manifest
regenerated fresh the same way rather than hand-picking a timestamp.
chaksaray added a commit that referenced this pull request Sep 4, 2026
…ework_sources

Implements #257, after a deep re-check that found and fixed a real
arithmetic error in that issue's own summary counts (23+22+21+8=74, not
80) and 11 additional corrections a fuller read of each record's full
description (not just the truncated behavioral_fingerprint excerpt used
for the first pass) resolved more precisely.

Real final disposition, all 80 records: 39 corrected (applied here), 26
confirmed correct as-is (no change), 15 left unchanged with no
confidently-better alternative (8 genuinely ambiguous between two
defensible options, 7 where none of the 10 current MCP Top 10 categories
covers the record's mechanism at all -- forcing a tag onto either group
risked a worse error than leaving the existing one). 39 + 26 + 15 = 80.

Two systemic patterns corrected, matching the shape #196's owasp_asi
audit found: MCP01 (Token Mismanagement and Secret Exposure) was applied
to 7+ records with no credential/token mechanism at all (shell injection,
goal hijack, jailbreak, hidden instruction, dynamic tool-call injection,
STDIO shell injection, zero-click auto-run); MCP10 (Context Injection and
Over-Sharing) was applied to 10+ single-session records with no
cross-session, cross-tenant, or persistent-sharing component, several of
which are near-exact fits for MCP06 (Intent Flow Subversion) instead --
confirmed against AVE's own corpus, which already cross-references
AVE-2026-00002/00016/00020/00028/00041/00043/00044/00065 as sharing one
underlying mechanism ("the same missing content-versus-instruction
boundary"), just realized at different delivery vectors; that grouping
maps cleanly onto MCP03 (the tool's own declared surface, read at
discovery) versus MCP06 (retrieved or received content, read in-flow)
once checked against each category's real, current text.

Corrections found and applied during the deep re-check that #257's first
pass had marked ambiguous or missed: 00003 (drop MCP05, no command
construction described), 00030/00061/00067 (a false-claim/trust-bypass
mechanism is MCP07's own text almost verbatim in each case, not MCP02 or
MCP05), 00038/00063 (MCP02 alone, dropping a weak secondary), 00041/00059
(MCP03 alone -- both explicitly happen at tool-discovery time per their
own record text, not in retrieved context), 00044/00048/00052 (drop a
weak secondary once the dominant real mechanism -- MCP06, MCP07, MCP05
respectively -- is identified), 00050 (MCP02's own "not declared in the
manifest" language is a near-verbatim match, replacing a weak MCP04),
00076 (natural-language content steering a decision-making classifier is
MCP03's territory, not MCP02's).

One record (AVE-2026-00073) was upgraded from ambiguous to confirmed
correct after the fuller text surfaced a concrete, cited case
(CVE-2026-21852) where the described mechanism directly caused a real
API-key leak, grounding the MCP01 tag rather than leaving it a stretch.

framework_sources.owasp_mcp backfilled for all 80 records, pinned to
OWASP/www-project-mcp-top-10 commit
165fe0f78ef104459237b4a8e0f6e78db9b02391 (2026-07-29, confirmed live via
the API before use) and today's date (2026-09-05) -- this audit is
exactly the dedicated, per-record verification issue #255 deferred this
field pending.

Found, not fixed here (separate, pre-existing, out of scope for this
PR): crosswalks/ave-to-owasp-mcp.md claims in its own header to be
'generated directly from each record's owasp_mcp field' but has no
generator script anywhere in the repo and was already stale before this
change -- it covers only 56 of the corpus's 80 records. Flagging in the
PR body rather than folding a second, unrelated repair into this one.

Validated: python scripts/validate_records.py (80/80 valid), python
scripts/check_fixtures.py (all pass), pytest tests/ -x -q (463 passed,
unchanged from before this PR since no test asserts specific owasp_mcp
values), python scripts/check_framework_sources.py (owasp_mcp no longer
appears in any finding; only nist_ai_rmf and the 10 mitre_atlas records
already deferred by #255 remain), node scripts/build-records.js (dist/
regenerated, diff is only the intended additions). Every touched record
diffed by hand for both the tag corrections and the framework_sources
insertions -- minimal, byte-exact changes, no unrelated reformatting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant