Restore HTTP::Message 7.04 compatibility

[fglock]

Summary

Restore complete HTTP::Message/HTTP::Request compatibility. HTTP::Request is provided by the pure-Perl HTTP-Message distribution; version 7.04 currently fails under PerlOnJava even though the same upstream suite passes under system Perl.

The archived CPAN random-tester run is 20260825-135237-11904, target HTTP::Request.

Results

System Perl, using the exact HTTP-Message 7.04 source:

All tests successful.
Files=23, Tests=871
Result: PASS

PerlOnJava 5.44.1:

Files=23, Tests=745
Result: FAIL
Failed 5/23 test programs. 9/745 executed subtests failed.

Two test programs abort early, so the executed-subtest count understates the impact. The failing files are:

• t/common-req.t
• t/message-charset.t
• t/message-decode-bzipbomb.t
• t/message-decode-zipbomb.t
• t/message.t

The report previously recorded 502/678 passing assertions in May; the current 736/745 executed assertions are substantial progress, but the distribution has not regained the complete PASS recorded by the March smoke-test investigation for HTTP-Message 7.01.

Confirmed root defect: utf8::downgrade destroys references

HTTP::Message deliberately calls utf8::downgrade on content values, including references. Standard Perl preserves the value and reference type:

my $value = sub { 42 };
print ref($value), '|', utf8::downgrade($value, 1), '|', ref($value), '|', $value->(), "\n";

Expected/system Perl:

CODE|1|CODE|42

Both PerlOnJava backends currently produce:

CODE|1|

The implementation converts the scalar to a string and writes that string back with byte-string type, losing the reference. This directly explains:

• t/common-req.t: dynamic upload content is expected to remain a CODE reference, then aborts while invoking the corrupted value.
• t/message.t: reference content/content_ref behavior aborts with Setting content_ref to a non-ref.

Additional failure clusters

1. Charset and tied-variable semantics

   • content_charset leaves $_ alone reports two unexpected stores.
   • UTF-8 fallback decoding returns undef where system Perl returns the expected byte/character values.

2. Compressed-body size limits and multilayer decoding

   • bzip2 decoding does not raise the expected size-limit error.
   • triple-gzip decoding returns undef instead of the 16 MiB decoded body.
   • likely areas include the Java Compress::Raw::Zlib and Compress::Raw::Bzip2 implementations, especially LimitOutput, buffer sizing, and status values such as Z_BUF_ERROR/BZ_OUTBUFF_FULL.

These should be reduced independently after the reference-preservation defect is fixed, since the early aborts currently hide later assertions.

Impact

HTTP::Message is foundational for LWP and the HTTP stacks used by Catalyst, Dancer2, Plack-related tooling, CPAN clients, and many other distributions. The cached bulk dependency data contains at least 86 unique direct runtime dependants of HTTP::Request, including 23 recent dependants; this is a lower bound rather than an exhaustive targeted result.

The distribution is pure Perl. The work should consist of reusable runtime/backend fixes, not a CPAN source patch.

Acceptance criteria

• Add a standard-Perl-validated regression test proving utf8::downgrade($reference, 1) preserves CODE, ARRAY, HASH, SCALAR, and object references.
• Make that regression pass on both JVM and interpreter backends.
• Add focused regressions for any confirmed tied-$_, Encode, zlib, and bzip2 runtime defects.
• Run the unchanged HTTP-Message 7.04 suite successfully: 23 files and 871 tests, apart from optional Brotli tests when the recommended modules are unavailable.
• Verify the LWP and Catalyst acceptance paths remain green.
• Update the stale HTTP::Message PASS statements in the smoke-test/support documentation and refresh the CPAN compatibility classification.