Skip to content

Filter invisible Unicode after HTML entity normalization #3101

Description

@SamMorrowDrums

Describe the bug

Invisible-character filtering currently occurs before HTML sanitization can decode character entities. Encoded invisible or bidirectional characters can therefore survive filtering. The removal set also omits relevant format characters and variation-selector ranges. Normalize first, then apply a carefully scoped Unicode filter with regression tests.

Affected version

Current main as of 2026-08-19.

Steps to reproduce the behavior

  1. Sanitize text containing numeric HTML entities for invisible or bidirectional Unicode characters.
  2. Observe that HTML processing decodes the entity after the initial invisible-character pass.
  3. See the resulting invisible character remain in output.

Expected vs actual behavior

Expected: invisible-character policy applies to the final normalized text and covers the intended Unicode ranges.

Actual: entity decoding can occur after filtering, and some relevant characters are not covered.

Logs

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions