Skip to content

Fix multiple localized regex capture variables on both backends #1127

Description

@fglock

Summary

PerlOnJava does not correctly update multiple localized regex capture variables. After local($1, $2, $3), a successful three-capture match updates $1, but $2 and $3 remain undefined on both execution backends.

This breaks MIME-tools 5.517 header, encoded-word, and parameter parsing because the distribution deliberately localizes capture variables in several parsing routines.

Minimal reproducer

use strict;
use warnings;
my$text = '=?US-ASCII?Q?Keith_Moore?=';
local($1, $2, $3);
pos($text) = 0;
$text =~ m{\G=\?([^?]*)\?([bq])\?([^?]+)\?=}xgiordie"no match";
my ($charset, $encoding, $payload) = ($1, lc($2), $3);
print"values=[$charset][$encoding][$payload]\n";

System Perl:

values=[US-ASCII][q][Keith_Moore]

PerlOnJava JVM backend:

Use of uninitialized value in concatenation (.) ...
values=[US-ASCII][][]

PerlOnJava interpreter backend produces the same incorrect values.

Removing local($1, $2, $3) makes the same regex populate all three captures on both PerlOnJava backends, so the defect is specifically tied to localization/dynamic scoping of the numbered capture variables.

CPAN integration evidence

Archived run: 20260825-173145-22349

Distribution: DSKOLL/MIME-tools-5.517.tar.gz

Requested module: MIME::Entity

PerlOnJava result:

Files=40, Tests=463
Result: FAIL
Failed 15/40 test programs. 70/463 subtests failed.

MIME-tools uses localized capture lists in:

  • MIME::Words::decode_mimewords: local($1,$2,$3)
  • MIME::WordDecoder: three- and five-capture variants
  • MIME::Field::ParamVal: three- and five-capture variants

This directly causes encoded MIME words, charsets, filenames, and RFC 2231 parameters to become empty or undefined. A focused MIME::Words example decodes:

=?US-ASCII?Q?Keith_Moore?= <moore@example.test>

as:

 <moore@example.test>

instead of:

Keith Moore <moore@example.test>

System Perl baseline

The exact 15 test programs that fail under PerlOnJava pass under system Perl:

Files=15, Tests=242
All tests successful.
Result: PASS

The complete system-Perl suite also passes every functional MIME test. Its only failure in the restricted test environment is t/Smtpsend.t, which cannot create its local test socket. PerlOnJava skips that test because process forking is unavailable; it is unrelated to this regex defect.

Related observations outside primary scope

The archived run repeatedly warns Unknown layer: scalar, but a focused <:scalar/>:scalar read/write reproducer succeeds on system Perl and both PerlOnJava backends. Those warnings should not be treated as the cause of the localized-capture failure.

Some MIME-tools failures may remain after this fix, notably the UU-decoder size mismatch. Retest the distribution after fixing capture localization and track genuinely independent residual failures separately.

Acceptance criteria

  • Add a permanent project-owned regression test for multiple localized numbered capture variables.
  • Verify the test first on system Perl.
  • Make the reproducer pass on both the JVM and interpreter backends.
  • Ensure nested localization restores the previous capture state correctly after scope exit.
  • Rerun MIME-tools 5.517 and reassess any residual failures independently.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions