Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Drop a pile of Arcs and `OutputSweeper::new_with_kv_store_sync` by TheBlueMatt · Pull Request #4119 · lightningdevkit/rust-lightning · GitHub
Skip to content

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync - #4119

Merged
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs
Sep 25, 2025
Merged

Drop a pile of Arcs and OutputSweeper::new_with_kv_store_sync#4119
TheBlueMatt merged 6 commits into
lightningdevkit:mainfrom
TheBlueMatt:2025-09-fewer-arcs

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Mostly a fix for #4111 but also drops a pile of unnecessary Arcs.

Both `OutputSweeperSync` and `LiquidityManagerSync` added `Arc`s to
their internal state to allow returning a reference to that
internal state in testing. While this is required to get a
forever-lifetime reference to that state, we don't actually need
that in testing.
Instead, we turn off the borrow checker in the
`lightning-background-processor` async tests where required,
avoiding the extra heap indirection.
Given the future returned from async BP methods will be concrete on
whatever types get used, there's not much reason to require that
the arguments be `'static`, we can let Rust figure out if they're
`'static` (and thus the returned future is `'static`).
@ldk-reviews-bot

ldk-reviews-bot commented Sep 24, 2025

Copy link
Copy Markdown

👋 Thanks for assigning @joostjager as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@TheBlueMattTheBlueMatt added this to the 0.2 milestone Sep 24, 2025
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Tagging 0.2 as #4111 is a really awk API currently and I'd rather not ship it, but if it slips so be it.

@codecov

codecovBot commented Sep 24, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.86822% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (4c9c211) to head (5e679b4).
⚠️ Report is 17 commits behind head on main.

Files with missing linesPatch %Lines
lightning-background-processor/src/lib.rs66.30%31 Missing ⚠️
lightning-liquidity/src/utils/time.rs0.00%3 Missing ⚠️
lightning-liquidity/src/manager.rs94.73%1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4119 +/- ##
==========================================
+ Coverage 88.52% 88.55% +0.02% 
==========================================
Files 179 179 Lines 134270 134310 +40 Branches 134270 134310 +40 ==========================================
+ Hits 118865 118939 +74 + Misses 12655 12612 -43 - Partials 2750 2759 +9 
FlagCoverage Δ
fuzzing21.81% <35.29%> (+0.01%)⬆️
tests88.39% <72.86%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

There's no need for these given `DefaultTimeProvider` is a unit
struct.
`OutputSweeper::new_with_kv_store_sync` is a pretty strange API -
it allows building an async `OutputSweeper` where the only `await`s
are on a sync `KVStore`, ie will immediately block until the IO
operation completes.
While this isn't broken (futures are allowed to take their time,
and async runtimes have to handle this, though they often don't
handle it particularly well), its pretty weird. It seems to exist
largely for `process_events_async_with_kv_store_sync`, which does
async `Event` handling but sync `KVStore` operations (like the
existing pre-0.2 "async" background processor).
Instead, we allow passing an `OutputSweeperSync` to
`process_events_async_with_kv_store_sync`, keeping the API
consistent such that a user would use the appropriate
`OutputSweeper` variant, but fetching the inner async
`OutputSweeper` inside the BP.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's not merge this before having confirmed this won't break LDK Node builds. I'll need at least 2, maybe more refactoring PRs on LDK Node's end, before we can actually confirm these change here don't mess something up.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now confirmed this doesn't break anything for LDK Node.

Not quite sure if avoiding some Arcs is worth the effort and last-minute churn (or the unsafe code for that matter), but changes LGTM.

let kv_store = Arc::new(KVStoreSyncWrapper(kv_store_sync));

// Yes, you can unsafe { turn off the borrow checker }
let lm_async: &'static LiquidityManager<_, _, _, _, _, _> = unsafe {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit has a nice end result, but this - I am not sure how many devs understand it...

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the comment is clear enough? Easy to just update the comment later if it makes it more readable.


/// A wrapper around a [`KVStoreSync`] that implements the [`KVStore`] trait. It is not necessary to use this type
/// directly.
#[derive(Clone)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so instead of ref counting, this will now be cloned

O: Deref,
K: Deref,
OS: Deref<Target = OutputSweeper<T, D, F, CF, KVStoreSyncWrapper<K>, L, O>>,
OS: Deref<Target = OutputSweeperSync<T, D, F, CF, K, L, O>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear to me now why I didn't do this in the first place.

@TheBlueMatt
TheBlueMatt merged commit b6f3705 into lightningdevkit:mainSep 25, 2025
25 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@TheBlueMatt@ldk-reviews-bot@tnull@joostjager