Keep closure-captured references alive in weak-reference tracking

[fglock]

Summary

PerlOnJava can destroy an object while it is still strongly referenced by a captured lexical in a live closure. This breaks the weak-reference protocol used by Future, leaves Future::Utils::repeat operations permanently pending, and causes many Net::Async::HTTP tests to time out after otherwise successful assertions.

The failure occurs on both the JVM and interpreter backends, so it appears to be shared runtime lifetime/reference-accounting behavior rather than backend-specific compilation.

Minimal reproducer

This uses only the pure-Perl Future distribution; it requires no networking or Net::Async::HTTP:

use strict;
use warnings;
use Future;

my $source = Future->new;
my $holder = Future->new;
my $completed = 0;

{
 my $sequence = $source->then(sub {
 $completed = 1;
 return Future->done;
 });

 $holder->on_cancel(sub { $sequence->cancel });
}

$source->done;
print "completed=$completed\n";

System Perl with Future 0.52 prints:

completed=1

Both PerlOnJava backends print the equivalent of:

Future=HASH(...) lost a sequence Future
completed=0

The live $holder owns an on_cancel closure, and that closure captures $sequence. The captured lexical is a strong reference under Perl semantics, so $sequence must remain alive until the closure or its owner becomes unreachable.

Direct Future::Utils::repeat reproducer

This is the abstraction used by Net::Async::HTTP:

use strict;
use warnings;
use Future;
use Future::Utils qw(repeat);

my $base = Future->new;
my $eventual = &repeat(
 sub {
 return $base
 ->then(sub { Future->done('middle') })
 ->then(sub { Future->done('final') });
 },
 while => sub { 0 },
);

$base->done('start');
print "ready=", ($eventual->is_ready ? 1 : 0),
 " result=", ($eventual->is_ready ? scalar($eventual->get) : 'pending'), "\n";

System Perl prints:

ready=1 result=final

Both PerlOnJava backends warn that a sequence Future was lost and print:

ready=0 result=pending

A direct two-stage then chain works when the final Future is held in an ordinary external scalar. The distinguishing case is that the strong retention is only through a closure-captured lexical, as it is inside Future::Utils::repeat.

Likely mechanism

Future::PP::_sequence stores the sequence Future in a callback record and weakens that particular slot. When the source Future becomes ready, _mark_ready expects the sequence Future still to exist through another strong owner. If it has disappeared, Future warns lost a sequence Future and cannot invoke the sequence callback.

Future::Utils::repeat keeps its pending trial Future in a lexical and installs a cancellation closure that captures it. That closure is reachable from the returned eventual Future and should therefore keep the trial Future alive.

PerlOnJava's selective reference-counting/lifetime machinery appears not to count or traverse captured lexical values as strong references. It consequently destroys the trial/sequence Future even though a live closure still captures it. Another possibility worth excluding is that weakening one callback slot incorrectly weakens the object globally rather than only that reference slot.

CPAN impact: Net::Async::HTTP

The CPAN random tester run 20260825-173204-22576 tested PEVANS/Net-Async-HTTP-0.50.tar.gz for Net::Async::HTTP. Configuration and build succeeded, but the report ended as Unknown test outcome because pending Futures repeatedly hit the tests' 10-second wait timeout and the complete run did not produce a normal harness summary.

Representative focused result from t/05redir.t, identical on the JVM and interpreter backends:

ok 1 - future defined
ok 2 - first request
IO::Async::Future=HASH(...) lost a sequence Future at .../Net/Async/HTTP/Connection.pm line 118
ok 3 - redirect
ok 4 - future not ready after redirect
Nothing was ready after 10 second wait

The same pattern affects t/04fail.t, t/06close.t, t/11response-streaming.t, t/12conn-persistence.t, t/13conn-pipeline.t, t/14conn-max.t, t/15conn-errors.t, t/16max-in-flight.t, and t/22local-connect-pipeline.t: assertions succeed up to an asynchronous transition, a sequence Future is lost, and the operation never becomes ready.

Net::Async::HTTP::_do_request uses Future::Utils::repeat around _do_one_request; _do_one_request then chains get_connection(...)->then(...)->then(...). This is the same object-retention shape as the small reproducers.

Under system Perl, the focused pure/fake/socketpair suite through t/19*.t passes. Some later tests in a full local run could not execute because the sandbox denied local listener creation, and three tests require a newer Test2::V0 than the system installation. Those environmental limitations do not affect the independent dependency-free reproducer or the focused Future::Utils::repeat comparison.

Separate residual failures

The archived distribution log also exposes two unrelated compatibility problems. They should not be folded into this fix:

• t/09cookies.t receives \$Version="1" where $Version="1" is expected from HTTP::Cookies handling.
• t/18content-coding.t sees bundled Compress::Bzip2 2.28 but fails because bzdeflateInit is undefined.

After correcting closure lifetime accounting, rerun the distribution and classify these residuals independently.

Suggested implementation direction

• Treat references in a reachable closure's captured lexical pad as strong edges in the selective lifetime/reference graph.
• Preserve slot-specific weak semantics: weakening one reference to an object must not weaken other references, including closure captures.
• Cover both JVM and interpreter execution paths, since both currently reproduce the bug.
• Preserve deterministic DESTROY behavior and avoid turning genuinely unreachable closure cycles into permanent retention.

Acceptance criteria

• Add a project-owned regression test, first validated against system Perl.
• The minimal closure-capture reproducer prints completed=1 without a lost-sequence warning on both backends.
• The Future::Utils::repeat reproducer prints ready=1 result=final on both backends.
• Ordinary weak references still become undefined when no strong reference remains.
• Captured objects remain alive while their closure is reachable, then become collectible and receive deterministic DESTROY at the appropriate time.
• Net::Async::HTTP's t/05redir.t completes on both backends without the warning or timeout.
• Rerun Net::Async::HTTP and classify the cookie and Bzip2 residuals separately.

Priority

High. This is a core Perl reference-lifetime semantic affecting both execution backends and a widely used asynchronous framework pattern, rather than a distribution-specific test assumption.