Unicode issue

[tvdijen]

Description

The following code:

<?php

$string_regex = '/^
 [
 \x09
 \x0A
 \x0D
 \x{20}-\x{7E}
 \x{85}
 \x{A0}-\x{D7FF}
 \x{E000}-\x{FDCF}
 \x{FDF0}-\x{FFFD}
 \x{10000}-\x{1FFFD}
 \x{20000}-\x{2FFFD}
 \x{30000}-\x{3FFFD}
 \x{40000}-\x{4FFFD}
 \x{50000}-\x{5FFFD}
 \x{60000}-\x{6FFFD}
 \x{70000}-\x{7FFFD}
 \x{80000}-\x{8FFFD}
 \x{90000}-\x{9FFFD}
 \x{A0000}-\x{AFFFD}
 \x{B0000}-\x{BFFFD}
 \x{C0000}-\x{CFFFD}
 \x{D0000}-\x{DFFFD}
 \x{E0000}-\x{EFFFD}
 \x{F0000}-\x{FFFFD}
 \x{100000}-\x{10FFFD}
 ]*$/Dxu';

$str = 'ünïcöde €Φ汉';

preg_match($string_regex, $str);

Resulted in this output:

Used to pass, do not know the exact version where it failed (last weeks though)

But I expected this output instead:

No longer passes in 8.5.7

PHP Version

==> Setup PHP
✓ PHP Installed PHP 8.5.7

Operating System

Fails on Centos 9, passes on Windows builds (GH Actions)

[xuwenzhi]

Looks like a duplicate of #20341 — this is the known PCRE2 character-class regression in 10.45–10.47 (upstream: PCRE2Project/pcre2#833). It's already fixed in PCRE2's dev branch (10.48-dev), not released yet.

It works on Windows because Windows builds always compile PHP's bundled PCRE2 (10.44, unaffected). Your CentOS build links the system PCRE2 (--with-external-pcre), which is 10.45+ and has the bug. php-src already downgraded the bundled copy back to 10.44 (#20380), but that doesn't help a build that links the system library.

Check your version on CentOS

php -i | grep "PCRE Library Version"

If it shows 10.45 – 10.47, that's the cause. Recompile php without flag --with-external-pcre would help

[LamentXU123]

cc @youkidearitai :)

[youkidearitai]

Hmm... My environment is seems works fine.

<?php

$string_regex = '/^
 [
 \x09
 \x0A
 \x0D
 \x{20}-\x{7E}
 \x{85}
 \x{A0}-\x{D7FF}
 \x{E000}-\x{FDCF}
 \x{FDF0}-\x{FFFD}
 \x{10000}-\x{1FFFD}
 \x{20000}-\x{2FFFD}
 \x{30000}-\x{3FFFD}
 \x{40000}-\x{4FFFD}
 \x{50000}-\x{5FFFD}
 \x{60000}-\x{6FFFD}
 \x{70000}-\x{7FFFD}
 \x{80000}-\x{8FFFD}
 \x{90000}-\x{9FFFD}
 \x{A0000}-\x{AFFFD}
 \x{B0000}-\x{BFFFD}
 \x{C0000}-\x{CFFFD}
 \x{D0000}-\x{DFFFD}
 \x{E0000}-\x{EFFFD}
 \x{F0000}-\x{FFFFD}
 \x{100000}-\x{10FFFD}
 ]*$/Dxu';

$str = 'ünïcöde €Φ汉';
$e = mb_str_split($str);

foreach ($e as $x) {
 $u = mb_convert_encoding($x, "UTF-32", "UTF-8");
 $b = bin2hex($u);
 var_dump($b);
}

var_dump(preg_match($string_regex, $str));

[root@thinkpad php-src]# sapi/cli/php ~/hoge.php
string(8) "000000fc"
string(8) "0000006e"
string(8) "000000ef"
string(8) "00000063"
string(8) "000000f6"
string(8) "00000064"
string(8) "00000065"
string(8) "00000020"
string(8) "000020ac"
string(8) "000003a6"
string(8) "00006c49"
int(1)
[root@thinkpad php-src]# cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.8 (Olive Jaguar)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.8"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.8 (Olive Jaguar)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.8"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.8"
SUPPORT_END=2032-06-01
[root@thinkpad php-src]# sapi/cli/php -i | grep 'PCRE '
PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.44 2024-06-07
PCRE Unicode Version => 15.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)

I can't reproduce that use the --with-external-pcre option too.

[root@thinkpad php-src]# sapi/cli/php -i | grep 'PCRE '
PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.40 2022-04-14
PCRE Unicode Version => 14.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
[root@thinkpad php-src]# sapi/cli/php ~/hoge.php
string(8) "000000fc"
string(8) "0000006e"
string(8) "000000ef"
string(8) "00000063"
string(8) "000000f6"
string(8) "00000064"
string(8) "00000065"
string(8) "00000020"
string(8) "000020ac"
string(8) "000003a6"
string(8) "00006c49"
int(1)

[tvdijen]

I don't know what I was thinking.. It's not Centos, it's Ubuntu (Github action ubuntu-latest).
It is indeed using a bugged version of the PCRE-library: 10.47 2025-10-21

Thanks for your help and my apologies for the duplicate!

[tvdijen]

One more question:

I have a web-application that relies heavily on regex with character classes.. I can't control what OS, what version of PHP or PCRE-library people run when they install this web-application, and I can't ask them to manually compile PHP without PCRE.
Is there anything I can do to prevent users from all over the world running into issues with this web-application? i.e. restrict them to a specific version of ext-pcre? I'm asking because it doesn't seem v10.48 of the PCRE-library is going to see daylight any time soon.