fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fix(core): strip the deprecated Cf bidi-adjacent controls - #3846

Merged
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls
Aug 27, 2026
Merged

fix(core): strip the deprecated Cf bidi-adjacent controls#3846
Astro-Han merged 2 commits into
apache:mainfrom
Sma1lboy:fix/sanitize-deprecated-cf-controls

Conversation

@Sma1lboy

Copy link
Copy Markdown
Contributor

Summary

U+206A–206F pass both sanitizer classes in packages/core/src/text-sanitize.ts: BIDI_FORMAT_REGEX stops at U+2069 and ZERO_WIDTH_REGEX stops at U+2064, leaving the six code points between them uncovered. All six are category Cf — invisible, deprecated, bidi-related formatting characters — and none of them are whitespace, so the \s+ collapse did not remove them either.

The bidi class is extended to U+206F, replacing with a space to stay consistent with the other bidi controls.

FOREIGN_UNSAFE_CHARS in foreign-session.ts is extended in the same commit rather than a follow-up. Its own doc comment describes it as "one source of truth" with the sanitizer, so fixing only the sanitizer would recreate exactly the drift that comment exists to prevent — and the id guard has the same gap.

Fixes#3823

Verification

npm run lint, npm run format:check, npm --workspace @maka/core run typecheck, and the full @maka/core suite (657/657) pass.

Both new assertions were confirmed RED before the fix by reverting the two source files and rebuilding:

✖ strips the deprecated Cf bidi-adjacent controls U+206A-206F (#3823)
✖ accepts safe tokens and rejects control/bidi/whitespace/overlong ids
ℹ pass 26 fail 2

Behaviour on current main, per code point:

cp name survives_sanitize id_guard_accepts
U+206A INHIBIT SYMMETRIC SWAPPING true true
U+206B ACTIVATE SYMMETRIC SWAPPING true true
U+206C INHIBIT ARABIC FORM SHAPING true true
U+206D ACTIVATE ARABIC FORM SHAPING true true
U+206E NATIONAL DIGIT SHAPES true true
U+206F NOMINAL DIGIT SHAPES true true
surviving_sanitize=6/6 accepted_by_id_guard=6/6

After the fix, both columns are 0/6. The regression test also pins controls that must not move: U+2069, U+2060 and U+202E stay stripped, and /\s/ is confirmed false for the new range — which is why the whitespace collapse never caught them.

Review focus

What this does and does not demonstrate. The reproduction shows the characters pass both pipelines, and that two session names differing only by one of them render identically while comparing unequal. It does not construct an attack — that would need an assumption about who controls session names, which I did not want to invent. Flagging so the change is judged on what is actually shown.

Deliberately out of scope. The issue raises variation selectors (U+FE00–FE0F) and tag characters (U+E0000–E007F) as a possible follow-up. Both are real invisible-content vectors, but they are a separate class decision with different trade-offs (variation selectors carry meaning in legitimate emoji and CJK sequences), so they are not folded in here.

Overlap with #3692. That PR adds packages/core/src/__tests__/text-sanitize.test.ts covering the existing bidi set, not this range. I put the regression in foreign-session.test.ts next to the existing sanitizeForeignText tests because it needs to assert the id-guard side too. If #3692 lands first and a maintainer prefers the coverage consolidated in the new file, I am happy to rebase and move the sanitizer half there.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Opus 5 (Claude Code) — investigation, the fix, and the regression tests. Reviewed and verified locally by me; the commit carries a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this head and found no blocking code issues.

Correctly handles U+206A–206F via sanitizer; verification passed. Hosted test currently has no checks on this head — needs CI green before merge.

[P3] Test coverage gap: normalizeUserSessionName regression not covered (only sanitizeForeignText loop), and isSafeForeignId only samples endpoints — suggest table-driven full range.

简体中文该头无阻断,仅测试覆盖建议。

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@M4n5ter
M4n5terforce-pushed the fix/sanitize-deprecated-cf-controls branch 3 times, most recently from 3549486 to 6225907CompareAugust 26, 2026 09:52
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
…face
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
@Sma1lboy
Sma1lboyforce-pushed the fix/sanitize-deprecated-cf-controls branch from 6225907 to 5f3ab26CompareAugust 26, 2026 18:14
@Sma1lboy

Copy link
Copy Markdown
ContributorAuthor

Both coverage gaps addressed in 5f3ab2625, and the branch is rebased onto current main.

  • isSafeForeignId now covers all six code points (U+206A–206F) rather than sampling the endpoints.
  • Added packages/core/src/__tests__/session-name.test.ts — no session-name test file existed. It exercises the full range through normalizeUserSessionName, which is the user-visible surface the issue names, and pins that an ordinary name is left untouched.

I put the session-name coverage in its own file rather than extending foreign-session.test.ts, since that file imports only from foreign-session.js and the assertion belongs to a different module.

Verification on the rebased head: @maka/core 670 tests, 670 pass. Reverting only text-sanitize.ts and foreign-session.ts while keeping the tests fails 3 of them, including the new session-name one, so the coverage is load-bearing rather than decorative. Lint, format and typecheck pass.

Agreed that the hosted test check still needs to run on this head before merge.

@github-actionsgithub-actionsBot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both P3s from the last head are closed. session-name.test.ts is new and loops the full 0x206a..0x206f range rather than sampling, and isSafeForeignId's rejection list went from seven sampled endpoints to all six of the new code points.

I checked the range boundary rather than trusting it: ⁦- matches U+2069 through U+206F and stops before U+2070 SUPERSCRIPT ZERO, which is a visible No character — so widening the bidi class strips only the deprecated Cf controls and cannot eat printable text. Extending FOREIGN_UNSAFE_CHARS in the same commit is right for the reason you gave: its own comment claims to be one source of truth with the sanitizer, so fixing one and not the other would create exactly the drift that comment exists to prevent.

(Unrelated to this PR: foreign-session.test.ts renders as a binary diff because it contains a literal NUL byte, and that's already true on main. I'll track it separately.)

AI-assisted review: I verified the regex boundaries at U+2069/206A/206F/2070 by running them, and read both test files at head 5f3ab2625. AI review is not independent human review.

@Astro-Han
Astro-Han merged commit 0e37b00 into apache:mainAug 27, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
* fix(core): strip the deprecated Cf bidi-adjacent controls
U+206A-206F (INHIBIT/ACTIVATE SYMMETRIC SWAPPING, INHIBIT/ACTIVATE ARABIC
FORM SHAPING, NATIONAL/NOMINAL DIGIT SHAPES) passed both sanitizer classes:
the bidi class stopped at U+2069 and the zero-width class at U+2064, leaving
the six between them uncovered. They are category Cf and not whitespace, so
the whitespace collapse did not remove them either.
Extend the bidi class to U+206F, replacing with a space to stay consistent
with the other bidi controls. FOREIGN_UNSAFE_CHARS is extended in the same
commit: its own doc comment names it one source of truth with the sanitizer,
so fixing only one side would recreate the drift that comment exists to
prevent.
Generated-by: Claude Opus 5
* test(core): cover the full U+206A-206F range and the session-name surface
Review feedback on apache#3846: the id guard only sampled the endpoints of the
new range, and normalizeUserSessionName had no regression at all even
though it shares the sanitizer this change fixes.
Extend the isSafeForeignId list to all six code points, and add
session-name.test.ts covering the range through normalizeUserSessionName —
the user-visible surface the issue names.
Generated-by: Claude Opus 5
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/SUnder 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(core): invisible format chars U+206A-U+206F pass both Unicode sanitizer pipelines

2 participants

@Sma1lboy@Astro-Han