Uh oh!
There was an error while loading. Please reload this page.
Report renderer drops pandoc and converts in process (LLP 0208) - #705
Conversation
Supersedes LLP 0196 open question 1's keep-pandoc resolution through the escape hatch that resolution named for itself: the only pandoc property the component vocabulary relies on is gfm passing raw HTML through untouched, "so in-process rendering stays available if the dependency ever becomes a problem." Server-side generation (hypaware-server LLP 0112) made it one twice over: pandoc would be that daemon's first non-npm binary, and execFileSync blocks the single thread every customer org shares, measured at ~40ms per page. The substitution was measured before it was made: all 94 files of a real reports tree converted under both engines; 66 structurally identical, 28 differing only in pandoc's syntax-highlighting markup, 0 genuine differences. After the swap the same tree renders to 94 pages with zero structural diffs against the pandoc reference, and the style-before-theme link order holds on every page. - pandocPage becomes htmlPage: marked (pinned, zero transitive deps) plus an explicit standalone template doing what -s did, with assets/head.html inlined after the base stylesheet link so theme.css keeps loading last (LLP 0196#theme-layer). - A renderer override reproduces pandoc's heading ids, -1 suffixes included, so existing in-page anchors keep resolving. - hasPandoc, the CLI preflight, the help text, the CI apt-get step, and the skill's prerequisite all go. The render tests lose their skip guard and run everywhere: a renderer with no external dependency has no excuse for untested paths. 3892 tests pass, none skipped for pandoc. - Syntax highlighting is the one visible change, light mode only: the stylesheet's token rules sit in its dark-mode block and flattened pandoc's spans to one colour anyway. language-* classes survive for later colour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fd963fd to
cc489bbCompare… file-size fix Both changes touch `maintainGeneration` and `compactGeneration`, and they are complementary: master (#701, LLP 0207) decides *when* a partition is rewritten at all, this branch decides how the rewrite sizes its output files. Each hunk is composed rather than taken from one side. - `maintainCache`: keep this branch's `async (span) =>` callback and its `compacted` / `data_files_before` / `data_files_after` / `rows` / `bytes_written` attributes, and pass master's `rebaselinesCounter` through to `maintainGeneration`. - `maintainGeneration`: keep master's three-way branch (foreign sorted replace re-baselines, dry run reports, otherwise compact) with its single `loadCompactionTableInfo` call, and re-add this branch's `r.compactedBytesWritten = result.bytesWritten` inside the compact arm. - `compactGeneration`: keep master's `tableInfo` parameter and the absence of the in-function metadata load (one metadata load per compaction), and keep this branch's streaming sink, its try/finally, and its `abort()` path. The JSDoc keeps master's `@param tableInfo` and this branch's `bytesWritten` return. - `cache-retention-maintenance.test.js`: union of both import lists. A generation written by the streaming sink commits through `stageSnapshotForAppend`, so its current snapshot is `append`, never `replace`: `foreignSortedReplace` cannot fire on our own rewrite even though we now carry the declared sort order forward. Also renumbers this branch's LLP 0208 to 0209: PR #705 independently took 0208 from master's high-water mark. Filename, header, every `@ref LLP 0208#...` in src and test, and LLP 0199's `Extended-by:` line (which now names both 0207 and 0209) move with it.
…by tests Round-1 review of #705. The pandoc-to-marked swap was measured structurally, which is exactly the comparison that cannot see either of these: one changes an attribute's spelling, the other changes text the browser never shows. - headingId was fed marked's already-escaped inline HTML, so `&` reached the slug rule as `&` and left its letters behind: `Cost & Usage` minted cost-amp-usage where pandoc mints cost--usage, and every authored [link](#whats-next) pointed at what39s-next. It now strips tags, unescapes, collapses whitespace runs before dropping punctuation (a dropped `&` leaves two spaces and pandoc emits both hyphens), and hyphenates per space rather than per run. Unicode letter and number classes keep `Café résumé` and `日本語` intact, as pandoc does. Re-verified 24/24 against pandoc 3.1.11 `-f gfm -t html5`, `-1` repeat suffixes included. - A tablecell override restates column alignment as pandoc's inline `style="text-align: ..."`. marked's built-in emits `align="right"`, a presentational hint the cascade ranks below assets/style.css's `th, td { text-align: left }`, so every right-aligned numeric column and every centred column silently rendered left, against tabular-nums. - test/core/report-render.test.js gains a golden page over the authoring vocabulary (component block byte-for-byte, aligned table, fenced block with its language class, headings carrying `&`, `'` and `/`) plus an anchor integrity check: every href="#..." must match an id on the page. Both fixes above fail this test before they land. - src/core/reports/README.md drops the stale "pandoc is still a hard dependency" rule and pandoc's `-H` for head.html. It ships in the package. - assets/style.css suppresses the list marker on task lists, which pandoc's template did through a task-list class marked does not emit. Footnotes are a real loss (marked's gfm has none) and are accepted and recorded rather than extension-patched. - LLP 0208 narrows its measurement claim to what is true of tables, and records the footnote and task-list consequences. LLP 0196 gains a header-level Superseded-in-part-by forward-ref. Co-Authored-By: Claude <noreply@anthropic.com>
philcunliffe
commented
Aug 10, 2026
Review round 1 - head |
…k lists Four residual round-2 findings on PR #705's pandoc-to-marked swap, each verified against a real pandoc 3.1.11 binary. Finding A: heading ids dropped combining marks (`[^\p{L}\p{N}_\s-]` stripped `\p{M}`), mangling Indic, Thai, Arabic/Hebrew, Vietnamese and decomposed Latin text unconditionally. headingId now NFC-normalizes before lowercasing and keeps `\p{M}` in the retained class, matching pandoc on decomposed "Café résumé", Turkish dotted-I, and Devanagari. Finding B: unescapeHtml only covered the five entities marked itself emits, so an author-written entity (`’`, `'`, ...) leaked its raw letters/digits into the slug and could dangle an in-page anchor. Replaced the five-entity table with a general decoder: numeric (`&#NNN;`, `&#xHH;`) plus a bounded, case-sensitive "HTML4" named-entity table (Latin-1, Greek, typography), sourced from the WHATWG entity list and stored as codepoints rather than literal characters. Case-sensitive matching was chosen over the case-insensitive suggestion after measuring pandoc itself: it decodes `&` (a real legacy dual-case alias) but not `&MDASH;` or `&RSQUO;`, and a case-fold would wrongly collide distinct entries like `Α`/`α`. Fixing this also surfaced a second bug: whitespace-run collapsing had to move to before entity decoding, or a decoded ` ` merged with real spaces around it into one hyphen instead of pandoc's one-hyphen-per-token count (verified: `A B` -> `a---b`). Finding C: the `ul:has(> li > input[type="checkbox"])` rule only matched "tight" task lists; a blank-line-separated ("loose") list wraps the checkbox in `li > p > input`, so it kept its bullet. The selector now matches both shapes. Also matched pandoc's cheap-to-copy indent behavior: pandoc's template kept the list's normal indent and pulled the checkbox left with a negative margin, rather than zeroing padding-left. LLP 0208's task-list bullet is corrected to name both shapes instead of just one. Finding D: a heading that reduces to nothing (`## <emoji>`) minted `id=""`; pandoc emits no id attribute. The heading renderer now omits the attribute when the slug is empty, while still running pandoc's own de-dup counter against the empty base for a repeat (`id="-1"`). Each fix is pinned in test/core/report-render.test.js, verified to fail against the pre-fix renderer before the change landed. Co-Authored-By: Claude <noreply@anthropic.com>
philcunliffe
commented
Aug 11, 2026
Review round 2 - head |
philcunliffe
commented
Aug 11, 2026
Stuck at triage: a blocker neutral introduced, past its review-round budgetWhat neutral was doing. PR #705 (adopted) replaces pandoc with in-process Why it cannot proceed. Triage found a true blocker at the current head The blocker: a heading with an out-of-range numeric character reference crashes the entire renderA heading like
That is a crash plus destruction of build output on author-controllable input, which is a production defect under LLP 0017 rather than a preference. Reports here are model-authored, so the input is not fully under a human's control. The fix is small and known: guard A second, non-blocking divergence that should ride the same fix
What neutral needs from youThe PR has used both of its review rounds (LLP 0059), so neutral will not keep iterating on its own - that cap exists to stop unbounded self-directed churn. Your call, and either is one comment:
What is verified good at this headEverything else checks out, independently re-derived against a real pandoc 3.1.11 binary rather than from the earlier rounds' tables: 35 of 36 heading-id cases at parity (the one miss is the trim case above), including combining marks, Devanagari, Turkish dotted capital I in both spellings, How to unstick: reply with a comment on this PR (or push to the branch). Neutral monitors this thread and will re-engage with your guidance on its next tick. |
philcunliffe
commented
Aug 11, 2026
neutral: rounds +2 — granted by U099BSGPZU4 via Slack |
philcunliffe
commented
Aug 11, 2026
add 2 more rounds — relayed from Slack |
philcunliffe
commented
Aug 11, 2026
Re-engaging with your guidance - two extra review rounds granted, so neutral will guard the |
Two heading-id findings from the round-2 entity decoder, both measured against a real pandoc 3.1.11 binary. An out-of-range numeric character reference in a heading crashed the whole render. `unescapeHtml` called `String.fromCodePoint` behind only a `Number.isFinite` guard, and that throws above U+10FFFF. The window is reachable rather than theoretical: marked's escaper passes numeric references of up to 7 decimal or 6 hex digits through untouched, so every value in `�`-`�` and `�`-`�` arrived verbatim. The blast radius is what made it worth fixing now: `renderReports` wipes `html/` before building and builds in sorted slug order, so one bad heading in one report destroyed the already-built pages of every report sorting at or after it and left the landing page stale. Reports are model-authored, so the input is not fully under a human's control. pandoc substitutes U+FFFD, which its slug then strips, minting `a--b` for `## A � B`, so the guard substitutes U+FFFD too rather than returning the entity intact, which would leak its digits into the id. Lone surrogates and U+10FFFF stay on the `fromCodePoint` path, where they already matched pandoc. `headingId` also trimmed after the punctuation strip, which pandoc does not do. Any heading starting or ending with stripped punctuation next to a space minted a different id: `## 🚀 Rollout plan` gave `rollout-plan` where pandoc gives `-rollout-plan`, and `## end &` gave `end` where pandoc gives `end-`. Emoji-led headings are ordinary in model-authored reports, so every pandoc-era `#-rollout-plan` anchor dangled silently, contradicting LLP 0208's promise that existing in-page anchors keep resolving. The trim is dropped, which restores parity across the whole measured table, including the all-whitespace case (pandoc mints `-` for `## ( )`, not no id) and the de-dup counters that run off those degenerate bases. Headings that reduce to the truly empty string still carry no id, and the reader has already trimmed authored outer whitespace, so no stray leading hyphen appears. Regression tests cover both, plus the amplifier: a malformed entity in one report must not destroy the pages of the reports that sort after it. Co-Authored-By: Claude <noreply@anthropic.com>
philcunliffe
commented
Aug 11, 2026
Review round 3 - head |
| heading | pandoc | pre-fix | post-fix |
|---|---|---|---|
## A � B | a--b | THREW | a--b |
## A � B | a--b | THREW | a--b |
## A�B | ab | THREW | ab |
## � | (no id) | THREW | (no id) |
## A � B | a--b | a--b | a--b |
## 🚀 Rollout plan | -rollout-plan | rollout-plan | -rollout-plan |
## ✅ Done items | -done-items | done-items | -done-items |
## end & | end- | end | end- |
## & start | -start | start | -start |
## Hello | hello-- | hello | hello-- |
## ( ) | - | (no id) | - |
## *emph* & | emph- | emph | emph- |
## Hello | hello | hello | hello |
## Cost & Usage | cost--usage | cost--usage | cost--usage |
Seven of the nineteen pre-fix divergences were crashes; twelve were wrong ids. The last rows are pre-existing contracts, confirmed unregressed. The fix worker found two divergences beyond the briefed table - ## Hello and inline markup followed by stripped punctuation (## *emph* &, ## `code` &) - same root cause, same fix.
Tests
Three new tests, each verified to fail pre-fix for the right reason: the out-of-range test with RangeError: Invalid code point 1114112 at render.js:236:50, the trim test with an emoji-led heading keeps pandoc's leading hyphen, and a blast-radius test that renders clean, then adds both a bad heading to the middle report and a brand-new report after it, asserting all four pages exist and the landing page lists the new one (a stale landing page would not). That last one pins the amplifying factor, not just the crash.
Gate
npm test 3916 pass / 0 fail, typecheck clean, npm pack --dry-run clean, llp-ref-hygiene 11/11.
The head moved to 606e70a, so the next tick reviews it as round 4 on the granted budget.
…pellings Three heading-id divergences from the pandoc that master shelled out to. Each one still renders the heading correctly, so the only visible symptom is a pandoc-era `#anchor` that silently dangles. All measured against a real pandoc 3.1.11 binary with one heading per document, so no de-dup counter drift can cascade between cases. Uppercase `&#X...;` never decoded. The entity regex spelled the hex alternative `#x[0-9a-fA-F]+`, which made the existing `body[1] === 'X'` guard unreachable: marked's escaper passes the uppercase form through (its no-encode pattern spells the prefix `#[Xx]`), so it arrived verbatim and leaked its digits. `## A A B` minted `a-x41-b` against pandoc's `a-a-b`. This also left `�` inside the out-of-range window unsubstituted, leaking digits where the guard's own comment says it must not, so that comment is corrected too. `'` was missing from NAMED_ENTITIES, so `## What's next` minted `whataposs-next` against pandoc's `whats-next` - a single-entry hole in the exact case the decoder's docstring uses to motivate itself, while `'`, `'` and `’` were all already right. The rest of the HTML5 ASCII-punctuation block had the same hole and is added with it: all 27 names were verified individually against pandoc rather than taken on trust. `<br>` was deleted outright by the general tag strip where pandoc's reader yields a space, welding `## Line one<br/>Line two` into `line-oneline-two`. It now substitutes a space before the tag strip and after the whitespace collapse, because pandoc counts that space as its own token: `## A <br /> B` mints `a---b`, not `a-b`. The match is lowercase-only, since pandoc treats `<BR>` as raw inline HTML contributing nothing. Every other tag still vanishes without a trace; `<span>a</span>b` and `A<em>B</em>C` are pinned so a future general tag-to-space rule cannot regress them. Moving the collapse ahead of the tag strip also stops a tag's own inner whitespace being merged away, which pandoc likewise counts as its own token. The heading renderer's comment claiming pandoc emits no id for a `<br>`-only heading was wrong and is corrected: pandoc mints `-`, and now so does this. Two measured divergences are left alone deliberately. U+2028, U+2029 and U+FEFF are in JS's `\s` but pandoc drops them, each minting one extra hyphen; the literal-authored spellings break further upstream in marked's block parser (no heading is emitted at all), so a character-class fix would only close half the case. And pandoc slugs a Markdown image's alt text (`## ` -> `img`) but emits no id for a raw-HTML `<img alt="...">`, which this renderer already matches; marked emits byte-identical HTML for both, so there is no string-level discriminator and a fix belongs at the token level, not here. LLP 0208's footnote consequence is tightened: it described only the multi-token definition, which survives as literal text. A single-token one (`[^1]: notes.md`) is parsed as a link reference definition instead, so the definition line disappears and the reference becomes a live link that rewriteHrefs then rewrites. Co-Authored-By: Claude <noreply@anthropic.com>
philcunliffe
commented
Aug 11, 2026
Review round 4 - head |
| case set | pre-fix | post-fix |
|---|---|---|
| round-4 finding cases (29) | 17/29 | 25/29 (only the two skipped nits remain) |
| ASCII-punct named entities (27) | 0/27 | 27/27 |
<br> and tag-whitespace (17) | 5/17 | 17/17 |
| broad prose regression sweep (46) | 46/46 | 46/46, no regression |
Each new test gives every heading distinct letters so no two share a base - otherwise the de-dup counter masks divergences, the exact aggregate-run trap noted above.
Gate
At 3c6876e: npm test 3919 pass / 0 fail, typecheck clean, npm pack --dry-run clean, llp-ref-hygiene 11/11.
Scoping note for the maintainer: every finding in this round pre-dated the round-3 commit, so none is a regression of the granted rounds. They are regressions relative to master, which shelled out to real pandoc, so each silently dangled the corresponding pandoc-era anchor. The granted +2 budget is now spent and the head moved to 3c6876e, so the next tick triages it.
philcunliffe
commented
Aug 11, 2026
Triage at head |
Uh oh!
There was an error while loading. Please reload this page.
master carries PR #705 as the squash 12cb2f4, whose blobs for llp/0208-report-renderer-drops-pandoc.decision.md, src/core/reports/render.js and test/core/report-render.test.js are byte-identical to 3c6876e, this branch's parent commit. The add/add and content conflicts were therefore purely lineage, not intent: master had nothing to contribute to those three files beyond what 3c6876e already held, so each resolves to this branch's side, which is 3c6876e plus the #709 fix. The merged tree differs from master by exactly the #709 commit's diff (verified byte for byte).
…710) * Report renderer drops pandoc and converts in process (LLP 0205) Supersedes LLP 0196 open question 1's keep-pandoc resolution through the escape hatch that resolution named for itself: the only pandoc property the component vocabulary relies on is gfm passing raw HTML through untouched, "so in-process rendering stays available if the dependency ever becomes a problem." Server-side generation (hypaware-server LLP 0112) made it one twice over: pandoc would be that daemon's first non-npm binary, and execFileSync blocks the single thread every customer org shares, measured at ~40ms per page. The substitution was measured before it was made: all 94 files of a real reports tree converted under both engines; 66 structurally identical, 28 differing only in pandoc's syntax-highlighting markup, 0 genuine differences. After the swap the same tree renders to 94 pages with zero structural diffs against the pandoc reference, and the style-before-theme link order holds on every page. - pandocPage becomes htmlPage: marked (pinned, zero transitive deps) plus an explicit standalone template doing what -s did, with assets/head.html inlined after the base stylesheet link so theme.css keeps loading last (LLP 0196#theme-layer). - A renderer override reproduces pandoc's heading ids, -1 suffixes included, so existing in-page anchors keep resolving. - hasPandoc, the CLI preflight, the help text, the CI apt-get step, and the skill's prerequisite all go. The render tests lose their skip guard and run everywhere: a renderer with no external dependency has no excuse for untested paths. 3892 tests pass, none skipped for pandoc. - Syntax highlighting is the one visible change, light mode only: the stylesheet's token rules sit in its dark-mode block and flattened pandoc's spans to one colour anyway. language-* classes survive for later colour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Report render: pandoc-parity heading ids and table alignment, pinned by tests Round-1 review of #705. The pandoc-to-marked swap was measured structurally, which is exactly the comparison that cannot see either of these: one changes an attribute's spelling, the other changes text the browser never shows. - headingId was fed marked's already-escaped inline HTML, so `&` reached the slug rule as `&` and left its letters behind: `Cost & Usage` minted cost-amp-usage where pandoc mints cost--usage, and every authored [link](#whats-next) pointed at what39s-next. It now strips tags, unescapes, collapses whitespace runs before dropping punctuation (a dropped `&` leaves two spaces and pandoc emits both hyphens), and hyphenates per space rather than per run. Unicode letter and number classes keep `Café résumé` and `日本語` intact, as pandoc does. Re-verified 24/24 against pandoc 3.1.11 `-f gfm -t html5`, `-1` repeat suffixes included. - A tablecell override restates column alignment as pandoc's inline `style="text-align: ..."`. marked's built-in emits `align="right"`, a presentational hint the cascade ranks below assets/style.css's `th, td { text-align: left }`, so every right-aligned numeric column and every centred column silently rendered left, against tabular-nums. - test/core/report-render.test.js gains a golden page over the authoring vocabulary (component block byte-for-byte, aligned table, fenced block with its language class, headings carrying `&`, `'` and `/`) plus an anchor integrity check: every href="#..." must match an id on the page. Both fixes above fail this test before they land. - src/core/reports/README.md drops the stale "pandoc is still a hard dependency" rule and pandoc's `-H` for head.html. It ships in the package. - assets/style.css suppresses the list marker on task lists, which pandoc's template did through a task-list class marked does not emit. Footnotes are a real loss (marked's gfm has none) and are accepted and recorded rather than extension-patched. - LLP 0208 narrows its measurement claim to what is true of tables, and records the footnote and task-list consequences. LLP 0196 gains a header-level Superseded-in-part-by forward-ref. Co-Authored-By: Claude <noreply@anthropic.com> * Report render: fix round-2 findings on heading ids, entities, and task lists Four residual round-2 findings on PR #705's pandoc-to-marked swap, each verified against a real pandoc 3.1.11 binary. Finding A: heading ids dropped combining marks (`[^\p{L}\p{N}_\s-]` stripped `\p{M}`), mangling Indic, Thai, Arabic/Hebrew, Vietnamese and decomposed Latin text unconditionally. headingId now NFC-normalizes before lowercasing and keeps `\p{M}` in the retained class, matching pandoc on decomposed "Café résumé", Turkish dotted-I, and Devanagari. Finding B: unescapeHtml only covered the five entities marked itself emits, so an author-written entity (`’`, `'`, ...) leaked its raw letters/digits into the slug and could dangle an in-page anchor. Replaced the five-entity table with a general decoder: numeric (`&#NNN;`, `&#xHH;`) plus a bounded, case-sensitive "HTML4" named-entity table (Latin-1, Greek, typography), sourced from the WHATWG entity list and stored as codepoints rather than literal characters. Case-sensitive matching was chosen over the case-insensitive suggestion after measuring pandoc itself: it decodes `&` (a real legacy dual-case alias) but not `&MDASH;` or `&RSQUO;`, and a case-fold would wrongly collide distinct entries like `Α`/`α`. Fixing this also surfaced a second bug: whitespace-run collapsing had to move to before entity decoding, or a decoded ` ` merged with real spaces around it into one hyphen instead of pandoc's one-hyphen-per-token count (verified: `A B` -> `a---b`). Finding C: the `ul:has(> li > input[type="checkbox"])` rule only matched "tight" task lists; a blank-line-separated ("loose") list wraps the checkbox in `li > p > input`, so it kept its bullet. The selector now matches both shapes. Also matched pandoc's cheap-to-copy indent behavior: pandoc's template kept the list's normal indent and pulled the checkbox left with a negative margin, rather than zeroing padding-left. LLP 0208's task-list bullet is corrected to name both shapes instead of just one. Finding D: a heading that reduces to nothing (`## <emoji>`) minted `id=""`; pandoc emits no id attribute. The heading renderer now omits the attribute when the slug is empty, while still running pandoc's own de-dup counter against the empty base for a repeat (`id="-1"`). Each fix is pinned in test/core/report-render.test.js, verified to fail against the pre-fix renderer before the change landed. Co-Authored-By: Claude <noreply@anthropic.com> * Report render: an out-of-range numeric entity no longer aborts the build Two heading-id findings from the round-2 entity decoder, both measured against a real pandoc 3.1.11 binary. An out-of-range numeric character reference in a heading crashed the whole render. `unescapeHtml` called `String.fromCodePoint` behind only a `Number.isFinite` guard, and that throws above U+10FFFF. The window is reachable rather than theoretical: marked's escaper passes numeric references of up to 7 decimal or 6 hex digits through untouched, so every value in `�`-`�` and `�`-`�` arrived verbatim. The blast radius is what made it worth fixing now: `renderReports` wipes `html/` before building and builds in sorted slug order, so one bad heading in one report destroyed the already-built pages of every report sorting at or after it and left the landing page stale. Reports are model-authored, so the input is not fully under a human's control. pandoc substitutes U+FFFD, which its slug then strips, minting `a--b` for `## A � B`, so the guard substitutes U+FFFD too rather than returning the entity intact, which would leak its digits into the id. Lone surrogates and U+10FFFF stay on the `fromCodePoint` path, where they already matched pandoc. `headingId` also trimmed after the punctuation strip, which pandoc does not do. Any heading starting or ending with stripped punctuation next to a space minted a different id: `## 🚀 Rollout plan` gave `rollout-plan` where pandoc gives `-rollout-plan`, and `## end &` gave `end` where pandoc gives `end-`. Emoji-led headings are ordinary in model-authored reports, so every pandoc-era `#-rollout-plan` anchor dangled silently, contradicting LLP 0208's promise that existing in-page anchors keep resolving. The trim is dropped, which restores parity across the whole measured table, including the all-whitespace case (pandoc mints `-` for `## ( )`, not no id) and the de-dup counters that run off those degenerate bases. Headings that reduce to the truly empty string still carry no id, and the reader has already trimmed authored outer whitespace, so no stray leading hyphen appears. Regression tests cover both, plus the amplifier: a malformed entity in one report must not destroy the pages of the reports that sort after it. Co-Authored-By: Claude <noreply@anthropic.com> * Report render: heading ids keep pandoc's `&#X`, `'` and `<br>` spellings Three heading-id divergences from the pandoc that master shelled out to. Each one still renders the heading correctly, so the only visible symptom is a pandoc-era `#anchor` that silently dangles. All measured against a real pandoc 3.1.11 binary with one heading per document, so no de-dup counter drift can cascade between cases. Uppercase `&#X...;` never decoded. The entity regex spelled the hex alternative `#x[0-9a-fA-F]+`, which made the existing `body[1] === 'X'` guard unreachable: marked's escaper passes the uppercase form through (its no-encode pattern spells the prefix `#[Xx]`), so it arrived verbatim and leaked its digits. `## A A B` minted `a-x41-b` against pandoc's `a-a-b`. This also left `�` inside the out-of-range window unsubstituted, leaking digits where the guard's own comment says it must not, so that comment is corrected too. `'` was missing from NAMED_ENTITIES, so `## What's next` minted `whataposs-next` against pandoc's `whats-next` - a single-entry hole in the exact case the decoder's docstring uses to motivate itself, while `'`, `'` and `’` were all already right. The rest of the HTML5 ASCII-punctuation block had the same hole and is added with it: all 27 names were verified individually against pandoc rather than taken on trust. `<br>` was deleted outright by the general tag strip where pandoc's reader yields a space, welding `## Line one<br/>Line two` into `line-oneline-two`. It now substitutes a space before the tag strip and after the whitespace collapse, because pandoc counts that space as its own token: `## A <br /> B` mints `a---b`, not `a-b`. The match is lowercase-only, since pandoc treats `<BR>` as raw inline HTML contributing nothing. Every other tag still vanishes without a trace; `<span>a</span>b` and `A<em>B</em>C` are pinned so a future general tag-to-space rule cannot regress them. Moving the collapse ahead of the tag strip also stops a tag's own inner whitespace being merged away, which pandoc likewise counts as its own token. The heading renderer's comment claiming pandoc emits no id for a `<br>`-only heading was wrong and is corrected: pandoc mints `-`, and now so does this. Two measured divergences are left alone deliberately. U+2028, U+2029 and U+FEFF are in JS's `\s` but pandoc drops them, each minting one extra hyphen; the literal-authored spellings break further upstream in marked's block parser (no heading is emitted at all), so a character-class fix would only close half the case. And pandoc slugs a Markdown image's alt text (`## ` -> `img`) but emits no id for a raw-HTML `<img alt="...">`, which this renderer already matches; marked emits byte-identical HTML for both, so there is no string-level discriminator and a fix belongs at the token level, not here. LLP 0208's footnote consequence is tightened: it described only the multi-token definition, which survives as literal text. A single-token one (`[^1]: notes.md`) is parsed as a link reference definition instead, so the definition line disappears and the reference becomes a live link that rewriteHrefs then rewrites. Co-Authored-By: Claude <noreply@anthropic.com> * Heading ids: every raw tag starting with `br` yields a space (#709) Deferred findings from PR #705, which this stacks on. Item 1 (fixed). pandoc 3.1.11's stringify keys on a raw inline's LEADING TEXT, not on a parsed tag name, so ANY raw inline HTML token whose text starts with the case-sensitive prefix `<br` becomes a space. The regex here was spelled `<br(?:\s[^>]*)?\/?>` on the stated rationale that it must not "swallow `<brand>`"; pandoc swallows `<brand>` too, so that rationale was false and the code comment recording it is corrected. Widening to `/<br[^>]*>/g` closes 11 divergences with no regressions, verified case by case against a real pandoc 3.1.11 binary, one heading per document so no de-dup counter drift can read as parity: `## A <brand> B` a--b -> a---b (pandoc a---b) `## A <brand>x</brand> B` a-x-b -> a--x-b (only the open tag counts) `## A <bra> B` a--b -> a---b `## A <br-x> B` a--b -> a---b `## A <br2> B` a--b -> a---b `## A <brand attr="y"> B` a--b -> a---b `## A <brand/> B` a--b -> a---b `## A <breakfast time> B` a--b -> a---b `## A <brand > B` a--b -> a---b `## A <brand>B` a-b -> a--b `## A<brand> B` a-b -> a--b The exclusions stay excluded: `<BR>`, `<Br>` and `<bR>` miss the lowercase prefix, `</brand>` starts `</b`, and `<b>`, `<bold>`, `<span>`, `<custom>` and `<q>` are not `br` at all. Widening is safe because the regex never sees a string marked did not already accept as a tag: marked escapes invalid tag syntax to entities first, and pandoc likewise leaves `<br@>` as text. Items 2 and 3 (recorded, not fixed). Both stay deferred, with the measurement behind them now in the doc instead of only the PR thread. Item 4. LLP 0208 records the footnote, task-list and highlighting consequences but promised heading ids "so existing in-page anchors keep resolving" without naming where that is inexact. A new `#heading-id-gaps` consequence names the two remaining classes, and `headingId` carries a `@ref` to it. Co-Authored-By: Claude <noreply@anthropic.com> * Fix PR #710 review findings: revert LLP 0208 settled text, correct @ref gloss LLP 0208 is Status: Active and merged to master, so its Decision paragraph is a settled record; revert the added qualifier on the heading-id sentence back to the master text and keep only the additive #heading-id-gaps consequence section. The @ref gloss in render.js overstated the constraint: the entity spellings of the Unicode separators/BOM are decoded by unescapeHtml and are reachable inside headingId, only the literal spellings break above this function in marked's block parser. Reword the gloss to match the LLP section it cites. * Correct BOM attribution in heading-id gap notes (#710 review round 2) Round 2 verified against marked 18.0.9 through the real renderReports path that a literal U+FEFF survives inline parsing and reaches headingId intact (`## K <BOM> L` renders `<h2 id="k-l">`), so its divergence comes from the slug rule's own `\s` handling, not from marked's block parser. Only a literal U+2028/U+2029 breaks upstream, splitting the block so no heading is produced at all. Reword the @ref gloss and the LLP 0208 #heading-id-gaps sub-bullet so the BOM is no longer cited as evidence of upstream breakage, and record that a slug-rule-only change would close the BOM case in full. Comment and doc accuracy only; no executable code, regex, or test changes. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Brendan McMullen <bgmcmullen@comcast.net> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: test <test@test.com>
What
Replaces
pandocPageinsrc/core/reports/render.jswith in-process conversion viamarked(pinned exact, zero transitive dependencies), and exports the renderer ashypaware/core/reportsso the server can share it. Everything else in the renderer is untouched: href rewriting, asset install, masthead, discovery, landing page.Supersedes LLP 0196 open question 1's keep-pandoc resolution through the escape hatch that resolution named for itself: the only pandoc property the component vocabulary relies on is gfm passing raw HTML through untouched, "so in-process rendering stays available if the dependency ever becomes a problem." LLP 0208 (new in this PR) records the decision.
Why now
The HypAware server generates reports server-side (hypaware-server LLP 0108/0112) and reuses this renderer rather than growing a second one (hypaware-server LLP 0110). On that surface pandoc's costs bite twice:
execFileSyncblocks the single thread every customer org shares, measured at ~40ms per page (~1.1s frozen for a 27-page report).Evidence the swap is safe
language-*class; tables and links match exactly.-1suffixes on repeats included) so existing in-page anchors keep resolving.apt-get install pandocstep and the previously-skipping suites now always run.Visible change
Syntax highlighting in code blocks, light mode only: pandoc emitted per-token spans, marked emits plain
<pre><code class="language-*">. The stylesheet's only token rules sit in its dark-mode block and flattened pandoc's spans to one colour anyway. Thelanguage-*class survives, so colour can return as a stylesheet choice later.Also in this PR
"./core/reports"added to the package exports map (the server-side consumer imports through it).hyp report renderpandoc preflight and help text go; the report skill's Render stage loses the prerequisite in both trees.Downstream consumer already landed against this branch: hypaware-server
feat/server-side-report-generation(its LLP 0110/0112/0113 document the shared-renderer decision and the live verification, including a real end-to-end generated report rendered by this code).🤖 Generated with Claude Code