Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); fuzz: add explicit chanmon manager persistence commands by joostjager · Pull Request #4631 · lightningdevkit/rust-lightning · GitHub
Skip to content

fuzz: add explicit chanmon manager persistence commands - #4631

Merged
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence
May 24, 2026
Merged

fuzz: add explicit chanmon manager persistence commands#4631
TheBlueMatt merged 3 commits into
lightningdevkit:mainfrom
joostjager:chanmon-async-manager-persistence

Conversation

@joostjager

Copy link
Copy Markdown
Contributor

Add chanmon_consistency commands to persist each node's ChannelManager state explicitly. This lets the fuzz target exercise delayed manager persistence instead of checkpointing it after every command.

@ldk-reviews-bot

ldk-reviews-bot commented May 21, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Add chanmon_consistency commands to persist each node's
ChannelManager state explicitly. This lets the fuzz target exercise
delayed manager persistence instead of checkpointing it after every
command.
@joostjager
joostjagerforce-pushed the chanmon-async-manager-persistence branch from 9bb46a0 to b33526eCompareMay 21, 2026 14:06
@codecov

codecovBot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.67%. Comparing base (ee456a8) to head (c18fab5).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #4631 +/- ##
===========================================
+ Coverage 28.02% 86.67% +58.65% 
===========================================
Files 126 159 +33 Lines 69960 110568 +40608 Branches 69960 110568 +40608 ===========================================
+ Hits 19606 95836 +76230 + Misses 49020 12213 -36807 - Partials 1334 2519 +1185 
FlagCoverage Δ
fuzzing-fake-hashes7.02% <ø> (+0.40%)⬆️
fuzzing-real-hashes23.28% <ø> (+0.03%)⬆️
tests86.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager
joostjager marked this pull request as ready for review May 21, 2026 16:42
Comment threadfuzz/src/chanmon_consistency.rs Outdated
}

fn restart_node(&mut self, node_idx: usize, v: u8, router: &'a FuzzRouter) {
self.nodes[node_idx].checkpoint_manager_persistence();

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit fake? Perhaps we should just restart from the latest persisted state, but that also means that payment tracking may be off a bit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mean this is kinda emulating the behavior we had before where we persist on every iteration, except now we only make sure we persist every time before we read it...Indeed, we'd have to handle the payment tracking parts, though :/.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was quite the same, because we were potentially stacking up monitor writes a bit more. But I added code to handle the payment tracking to make it more realistic.

I really hope we can improve the current persistence model situation soon. Keeping the separate monitor and manager persistence model coherent is such a time sink.

@ldk-claude-review-bot

ldk-claude-review-bot commented May 21, 2026

Copy link
Copy Markdown
Collaborator

No issues found.

I reviewed every hunk in the diff, tracing through:

  1. Generation tracking correctness: next_manager_persistence_generation() returns current + 1, payments are tagged with this value, and checkpoints increment the counter — the invariant that a payment tagged with generation G is present in all snapshots >= G holds.

  2. restart_node flow: Non-deferred mode checkpoints before reload (ensuring all in-flight payments are captured), while deferred mode intentionally skips it to exercise stale-state reload. The loaded_manager_generation is correctly captured before the post-reload force_checkpoint_manager_persistence() bump.

  3. sync_pending_with_manager_generation: Correctly rolls back only payments from pending (not resolved) whose generation exceeds the loaded snapshot, and cleans up claimed_payment_hashes for those rolled-back payments.

  4. Pre-existing edge case preservation: The refactored NodePayments methods (mark_sent, mark_resolved_without_hash, mark_successful_probe) preserve the exact semantics of the old PaymentTracker methods, just operating on PendingPayment structs instead of bare PaymentIds.

  5. No off-by-one: The comparison first_persisted_manager_generation > loaded_manager_generation correctly keeps payments at the boundary (gen == loaded) and rolls back those strictly above.

  6. Command byte collision: 0x90-0x92 don't overlap with any existing commands.

  7. process_all_events/settle_all still calls checkpoint_manager_persistences(), so settlement paths remain correct.

@joostjagerjoostjager self-assigned this May 21, 2026
@TheBlueMatt
TheBlueMatt removed their request for review May 21, 2026 18:42
Move node-local payment tracking mutations onto NodePayments.
Pending and resolved payment state are updated the same way.
The owner of that state now owns the helper methods.
Stamp pending payments with the first manager generation.
On deferred reload, drop payments born after the loaded snapshot.
This keeps tracker state aligned with explicit persistence.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

@ldk-reviews-bot

Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

@TheBlueMatt
TheBlueMatt merged commit 0c37f08 into lightningdevkit:mainMay 24, 2026
24 checks passed
@joostjager

Copy link
Copy Markdown
ContributorAuthor

One thing I noticed is that we add to claimed_payment_hashes immediately upon calling node.claim_funds and then expect that to result in a payment resolution on the sending side. This should lead to failures (and since it apparently doesn't we should expand coverage so that it does!) in cases where we call claim and then immediately restart without persisting manager or monitors. Instead, we should presumably be adding to claimed_payment_hashes after a PaymentClaimed event.

I tried to repro that failure, but for the cases I tested, the HTLC is then emitted as PaymentClaimable again after reload, the harness claims it again, and makes the tracking check out?

It is cleaner though to mark claimed after PaymentClaimed: #4635

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants

@joostjager@ldk-reviews-bot@ldk-claude-review-bot@TheBlueMatt