Fix flaky reorg tests by tnull · Pull Request #972 · lightningdevkit/ldk-node

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull · 2026-07-08T07:37:19Z

Previously, the reorg tests could be somewhat flaky. Here we attempt to stabilize them.

ldk-reviews-bot · 2026-07-08T07:37:22Z

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

joostjager · 2026-07-08T09:20:38Z

Here we attempt to stabilize them

Were you able to reliably repro the flakes and see that this PR fixed them?

The test gave funding six confirmations while allowing six-block reorgs. That drops regular channel funding to zero confirmations and changes the scenario from a close reorg into a funding force-close. rust-lightning PR #4231 keeps trusted zero-conf channels open after funding is reorged out, but regular channels still force-close at zero confirmations. Mine one extra block so the deepest reorg leaves one confirmation. Preserve the CI counterexample that exposed this boundary. Co-Authored-By: HAL 9000

tnull · 2026-07-15T11:10:36Z

Alright, excuse the delay here. I now revisited the failure cases and narrowed the changes down to what's absolutely necessary, while improving the commit messages and comments. Also found another root cause for a flake, as we'd force-close if we entirely unconfirm a funding for a non-0conf channel.

joostjager

Looks good, clear improvements. Only minor questions.

joostjager · 2026-07-15T12:18:23Z

- generate_blocks_and_wait(bitcoind, electrs, 6).await;
+ // Keep funding confirmed across the deepest reorg. rust-lightning PR #4231 exempts
+ // only trusted zero-conf channels; regular channels still force-close at zero confirmations.
+ generate_blocks_and_wait(bitcoind, electrs, 7).await;


Curious how this could have caused a flake?

LDK currently force-closes a non-0conf channel if the funding is entirely unconfirmed due to a reorg. In the reorg test we randomize reorg height to be 1..=6, and if it happen to roll 6 in this particular instance the channel would force-close and we'd get the flake. The fix here is to just generate one more block to assert that we never completely reorg out the funding transaction here.

I understand the fix, but in my opinion non-deterministic tests are something to avoid. Obviously it's a cheap way to expand coverage, but the cost of that is felt down the line.

Oh, I disagree on that sentiment. Exactly for things like reorgs you want to introduce some cases that make use of randomization (essentially chaos testing) to test cases that you'd otherwise only ever hit in production.
but the cost of that is felt down the line
I have no idea what 'the cost of that' is meant to be in this case.
Also, didn't you just expand/improve reorg and mempool coverage of LDK's fuzzer? How's that materially different from the proptests here?

Fair, I should have been clearer. I’m not against randomized/chaos testing, quite the opposite, that’s also why I expanded the reorg/mempool fuzzer coverage.
The cost I meant is mostly CI signal: before a counterexample is persisted, a deterministic boundary case can look intermittent because it only fails when proptest generates that input. The repro output helps, but there is still triage overhead.
My unease is mostly about the overlap between fuzzing and proptests, since both serve a discovery role. I’m trying to get clearer on which randomized coverage we want in regular CI versus in fuzz testing.

joostjager · 2026-07-15T12:21:08Z

 	let _block_hashes_res = bitcoind.generate_to_address(num, &address);
-	wait_for_block(electrs, cur_height as usize + num).await;
+	let min_height = cur_height as usize + num;
+	wait_for_block(bitcoind, electrs, min_height).await;


Unnecessary var extraction, not consistent with calls below in the integration test?

Will revert.

joostjager · 2026-07-15T12:24:25Z

+	let is_spent =
+ !electrs.script_list_unspent(&txout_script).unwrap().iter().any(|output| {
+ output.tx_hash == outpoint.txid && output.tx_pos == outpoint.vout as usize
+ });


Initial poll looks like an optimization, but is it needed?

It was an optimization that is now gone as you seem to prefer it.

If we need the optimization, it doesn't need to be removed. But if it doesn't add anything meaningful, it's just duplicated code that can get out of sync.

The block helper only compared heights. Replacing invalidated blocks with the same number of new blocks leaves the height unchanged, so it could return while Electrum still exposed the old chain. Wallet syncs then observed stale state and made reorg assertions timing-dependent. Require Electrum's target-height hash to match bitcoind's replacement block before returning. Co-Authored-By: HAL 9000

The spend helper treated any script history as proof of a spend. The funding transaction itself already creates such a history entry, so the helper normally returned before Electrum indexed the closing transaction. A following reorg could therefore begin while the funding outpoint was still unspent. Wait until the exact transaction output leaves Electrum's unspent set. Co-Authored-By: HAL 9000

The force-close loop mined separately for each node even though every node shares one chain. Mining for the first node advanced later nodes past the exact intermediate state the test required. Sweep publication and Electrum indexing are also asynchronous, so immediate assertions could observe either valid state. Advance the shared chain once and poll every claimable sweep through broadcast and confirmation. Preserve the force-close counterexample that fails when only the funding-depth fix is applied. Co-Authored-By: HAL 9000

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

tnull requested a review from joostjager July 8, 2026 07:37

joostjager reviewed Jul 8, 2026
View reviewed changes

Jolah1 mentioned this pull request Jul 10, 2026
connection: add per-peer exponential reconnection backoff #951
Closed

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f34c3ea to 5174199Compare July 15, 2026 11:09

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from 5174199 to f79d235Compare July 15, 2026 11:26

tnull requested a review from joostjager July 15, 2026 11:27

joostjager reviewed Jul 15, 2026
View reviewed changes

tnull added 3 commits July 15, 2026 16:05

tnull force-pushed the 2026-07-fix-flaky-reorg-tests branch from f79d235 to 87b780eCompare July 15, 2026 14:35

tnull requested a review from joostjager July 15, 2026 14:36

joostjager approved these changes Jul 15, 2026
View reviewed changes

tnull merged commit 03812a0 into lightningdevkit:mainJul 15, 2026
16 of 23 checks passed

Uh oh!

Conversation

tnull commented Jul 8, 2026

Uh oh!

ldk-reviews-bot commented Jul 8, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joostjager commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tnull commented Jul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joostjager left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjagerJul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tnull commented Jul 8, 2026

Uh oh!

ldk-reviews-bot commented Jul 8, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joostjager commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tnull commented Jul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joostjager left a comment

ldk-reviews-bot commented Jul 8, 2026•
edited
Loading

tnull commented Jul 15, 2026•
edited
Loading

joostjagerJul 15, 2026•
edited
Loading

ldk-reviews-bot commented Jul 8, 2026•
edited
Loading

tnull commented Jul 15, 2026•
edited
Loading

joostjagerJul 15, 2026•
edited
Loading

ldk-reviews-bot commented Jul 8, 2026•
edited
Loading