Surfaced by the new export tests in T-ENG01, which read the OSCAL catalogs back for the first time.
What is wrong
In several mapping files the control identifier and the requirement prose are in the wrong columns, so the generated control_id holds a sentence:
{
"framework": "SOC 2",
"control_id": "Prompt injection documented as a threat in LLM application risk assessment — vectors, likelihood, impact assessed",
"control_name": "CC3.2 — Risk assessment identifies threats and vulnerabilities"
}The identifier is in control_name; control_id is the requirement text. A reader following an id back to the framework cannot, and the control-level join in src/ resolves by luck rather than by design.
Where it comes from
It is a parser behaviour, not hand-entered data. Files like llm-top10/LLM_SOC2.md use a two-column table:
| Criteria | How it applies to prompt injection |
|---|---|
| CC3.2 — Risk assessment identifies threats and vulnerabilities | Prompt injection documented as a threat … |
parseControlTable() in scripts/generate.js tries looksLikeId(col1) — the prose fails — then a Name (CODE) pattern — also fails — and falls through to controlId = col1, which is the description. The id and name end up swapped.
The fix looks mechanical: when column 0 matches ^(<token>)\s*[—–-]\s*(.+)$, split it into control_id and control_name and take column 1 as notes. That would repair every affected row deterministically, with no mapping judgment involved.
Scale
576 prose-shaped ids, by framework:
| Framework |
Prose ids |
| SOC 2 |
168 |
| EU AI Act |
118 |
| OWASP NHI Top 10 |
103 |
| NIST SP 800-218A |
40 |
| PCI DSS v4.0 |
40 |
| CIS Controls v8.1 |
31 |
| NIST SP 800-82 Rev 3 |
29 |
| CWE/CVE |
25 |
| OWASP AI Testing Guide |
16 |
| AIUC-1 |
3 |
| ISO/IEC 42001:2023 |
3 |
What is already done
- The OSCAL exports no longer break on it.
oscalToken() in compliance-report.js coerces ids into valid OSCAL tokens and preserves the original verbatim in a source-control-id prop and in the control title. Before that change, 13 of 25 frameworks emitted catalogs that no OSCAL tool would load — 750 invalid ids — and nothing had ever noticed.
- A regression fence in
scripts/exports.test.mjs pins the table above. Any framework that gains prose ids, or any framework not on the list that starts producing them, fails the test. A second test fails if a baseline entry names a framework that no longer exists, so the fence cannot rot into dead permission.
What is not done
The data is still wrong. This needs the parser fix plus a regeneration, and a check of the 153 dangling references in reports/coverage-of-registry.md, most of which are expected to be the same root cause.
Not fixed inside T-ENG01 because it changes 576 rows across 25 frameworks and deserves its own review.
Surfaced by the new export tests in T-ENG01, which read the OSCAL catalogs back for the first time.
What is wrong
In several mapping files the control identifier and the requirement prose are in the wrong columns, so the generated
control_idholds a sentence:{ "framework": "SOC 2", "control_id": "Prompt injection documented as a threat in LLM application risk assessment — vectors, likelihood, impact assessed", "control_name": "CC3.2 — Risk assessment identifies threats and vulnerabilities" }The identifier is in
control_name;control_idis the requirement text. A reader following an id back to the framework cannot, and the control-level join insrc/resolves by luck rather than by design.Where it comes from
It is a parser behaviour, not hand-entered data. Files like
llm-top10/LLM_SOC2.mduse a two-column table:parseControlTable()inscripts/generate.jstrieslooksLikeId(col1)— the prose fails — then aName (CODE)pattern — also fails — and falls through tocontrolId = col1, which is the description. The id and name end up swapped.The fix looks mechanical: when column 0 matches
^(<token>)\s*[—–-]\s*(.+)$, split it intocontrol_idandcontrol_nameand take column 1 asnotes. That would repair every affected row deterministically, with no mapping judgment involved.Scale
576 prose-shaped ids, by framework:
What is already done
oscalToken()incompliance-report.jscoerces ids into valid OSCAL tokens and preserves the original verbatim in asource-control-idprop and in the controltitle. Before that change, 13 of 25 frameworks emitted catalogs that no OSCAL tool would load — 750 invalid ids — and nothing had ever noticed.scripts/exports.test.mjspins the table above. Any framework that gains prose ids, or any framework not on the list that starts producing them, fails the test. A second test fails if a baseline entry names a framework that no longer exists, so the fence cannot rot into dead permission.What is not done
The data is still wrong. This needs the parser fix plus a regeneration, and a check of the 153 dangling references in
reports/coverage-of-registry.md, most of which are expected to be the same root cause.Not fixed inside T-ENG01 because it changes 576 rows across 25 frameworks and deserves its own review.