docs: capability-behavior-vulnerability taxonomy audit (all 80 records) - #224
Merged
Merged
Conversation
Full audit per the adopted A/B/C/D/E design: for every record, what security property is violated, and what makes it a vulnerability rather than merely a capability or attacker behavior. Step 0 (inserted verification step): confirmed AVE-2026-00074, 00078, and 00080 against the live corpus before treating them as positive controls -- all three exist and match what the audit design assumed about them. Full seven-question treatment for Priority 1 (5 capability-heavy records), Priority 2 (11 technique-vs-vulnerability records), Priority 3 (12 conventional-vs-agentic records), and the seven positive controls -- 35 records total. Sprint 4's remaining 45 records get a real but proportionately lighter pass (classification, boundary, decision, Q7 three-line artifact), per the task's own instruction not to review all 80 with equal effort. Result: 38 VALID-AVE, 23 INSUFFICIENT-BOUNDARY (real mechanism, needs clearer description text, not a structural change), 2 TECHNIQUE-CONFLATION (00009, 00032), 1 CAPABILITY-CONFLATION (00038), 16 GENERIC-VULNERABILITY (conventional CWE-territory mechanisms with varying strength of agentic-specific justification -- three flagged as a genuinely open maintainer question: 00052/00053/00060 are pure MCP-implementation bugs with no behavioral component at all). No schema change proposed. security_condition/security_boundary were the two candidates evaluated; the audit's own evidence is that every record already has an identifiable boundary and violated property -- the real gap was almost always articulation in existing prose fields, not a missing structured field. Deliberately does not open the community-invitation umbrella issue. astrogilda and narko4u both have active, current threads (#218/#219, the #214 review offer, the GenAI Crosswalk manual-PR recommendation) that predate this audit; opening a third simultaneous ask without checking that queue first is what this task's own sequencing note asked not to do by default.
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full audit against the adopted A/B/C/D/E design: for every record, what security property is violated, and what makes it a vulnerability rather than merely a capability or attacker behavior.
Step 0 (the inserted verification step): confirmed
AVE-2026-00074,00078, and00080against the live corpus before treating them as positive controls. All three exist and match what the audit design assumed about them — no correction needed.Method: full seven-question treatment for Priority 1 (5 capability-heavy records), Priority 2 (11 technique-vs-vulnerability), Priority 3 (12 conventional-vs-agentic), and the 7 positive controls — 35 records total, all classified individually against the real record content, not from memory or the label alone. Sprint 4's remaining 45 records get a real but proportionately lighter pass (classification, boundary, decision, and the Q7 three-line artifact), per the task's own instruction not to review all 80 with equal effort.
Result: 38 VALID-AVE, 23 INSUFFICIENT-BOUNDARY (real mechanism, just needs the violated property stated more explicitly — a prose fix, not structural), 2 TECHNIQUE-CONFLATION (
00009,00032), 1 CAPABILITY-CONFLATION (00038), 16 GENERIC-VULNERABILITY (conventional CWE-territory mechanisms with a real spread of agentic-justification strength — three flagged as a genuinely open question:00052/00053/00060are pure MCP-implementation bugs with no behavioral component at all, and AVE's value there is a coverage argument, not a mechanism-level distinction).Positive controls held up: all seven classified cleanly as A under the same standard applied to everything else — real, positive evidence the method discriminates rather than defaulting either direction, since Priority 1–3 produced a genuine spread across all five classifications on that same standard.
No schema change proposed.
security_condition/security_boundarywere the two candidates this audit was asked to evaluate. The evidence doesn't support either: every record already has an identifiable boundary and violated property — the real gap was almost always articulation in the existingdescription/behavioral_fingerprintprose, not a missing structured field. Recommendations (R-001 through R-005) are documentation and process fixes, not data-model changes.Deliberately does not open the community-invitation umbrella issue. astrogilda and narko4u both have active, current threads (
#218/#219, the#214review offer, the GenAI Crosswalk manual-PR recommendation) that predate this audit. Opening a third simultaneous ask without checking that queue first is what this task's own sequencing note asked not to do by default — noted explicitly in the audit's governance section for whoever picks that up next.All 80 records still validate, fixtures intact, 342 tests pass (docs-only change, no records touched).
Umbrella tracking/discussion issue for this audit: #225. Not a "closes" relationship deliberately — #225 stays open for challenge and disagreement after this merges, not resolved by the PR landing.