perf: ASCII fast path for find_str_nocase, verified against the collator

[ddehilster]

Where the time is now

With the I/O and stack-frame work done in 3.8.7/3.8.9, a fresh profile of released 3.8.9 shows the compiled runtime is dominated by case-insensitive string matching:

run!find_str_nocase 32.3% inclusive
run!Arun::match_list 28.6%
icu::Collator::compare 13.2%
icu::UnicodeString::fromUTF8 10.3%
icu::RuleBasedCollator::doCompare 9.6%

For comparison, ntdll!ZwWriteFile — 25.5% back in 3.8.4 — is now 2.2%, and _chkstk is gone. A third of the run is matching a node's name against a rule element's match/fail/except list through an ICU collator at PRIMARY strength.

The change

When both sides are printable ASCII, PRIMARY-strength equality is exactly byte-wise case-insensitive equality, so ICU can be skipped.

That claim is verified, not assumed. A test program built against the same ICU compared the collator against a byte-wise compare:

corpus	pairs	mismatches
all printable-ASCII character pairs (0x20–0x7E)	9,025	0
word / punctuation / spacing variants	1,521	0
control characters (0x01–0x1F)	124	78

That last row is why the guard is "printable ASCII" and not "ASCII". Control characters are primary-ignorable — the collator considers "a\x01" equal to "a". Bytes ≥ 0x80 are excluded too, since PRIMARY strength also folds accents (so "cafe" matches an accented "café", which may well be deliberate in an NLP engine). Either case sends that one comparison to ICU, so no matching behaviour changes.

The needle's UTF-16 form is also built lazily now, so a lookup whose candidates are all plain ASCII — overwhelmingly the common case — does no transcoding at all.

Results

Interleaved A/B, 8 runs per arm, swapping both nlp.exe and run.dll:

	min	p25	median
ICU collator (3.8.9)	7.78s	8.10s	8.75s
ASCII fast path	7.02s	7.49s	8.09s

1.08x – 1.11x. Smaller than the 32% inclusive figure might suggest, because much of that is match_list's own work rather than ICU. The mechanism did engage: RuleBasedCollator::doCompare, ::compare and CollationFastLatin::compareUTF16 all drop out of the profile entirely, and u_strFromUTF8WithSub halves.

Correctness

All 18 analyzer output files byte-identical. That's the strongest check available here — this function decides what the matcher matches, so any semantic drift would show up as different parse output.

🤖 Generated with Claude Code