Skip to content

Fix outbound payments probe leak - #2037

Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged

Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom
valentinewallace:2023-02-fix-probe-leak

Conversation

@valentinewallace

@valentinewallacevalentinewallace commented Feb 15, 2023

Copy link
Copy Markdown
Contributor

Fixes#2036.

Based on #2008.

@valentinewallacevalentinewallace added this to the 0.0.114 milestone Feb 15, 2023
@jkczyz
jkczyz self-requested a review February 15, 2023 23:02
@codecov-commenter

codecov-commenter commented Feb 15, 2023•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️

Coverage data is based on head (2f14891) compared to base (30b9d9f).
Patch coverage: 92.70% of modified lines in pull request are covered.

❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@TheBlueMatt

TheBlueMatt commented Feb 16, 2023

Copy link
Copy Markdown
Collaborator

Needs rebase now that #2008 landed.

@TheBlueMatt

TheBlueMatt commented Feb 16, 2023

Copy link
Copy Markdown
Collaborator

Change LGTM, though.

@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt merged commit a170478 into lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution

4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
Fix outbound payments probe leak by valentinewallace · Pull Request #2037 · lightningdevkit/rust-lightning · GitHub
Skip to content
Fix outbound payments probe leak  - #2037
Merged
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Feb 16, 2023
Merged
Fix outbound payments probe leak #2037
TheBlueMatt merged 1 commit into
lightningdevkit:mainfrom 
valentinewallace:2023-02-fix-probe-leak
Conversation
@valentinewallace
@valentinewallacevalentinewallace
commented
Feb 15, 2023

Copy link


Copy Markdown

Contributor
Fixes#2036.
Based on #2008.
@valentinewallacevalentinewallace
added this to the 0.0.114 milestone
Feb 15, 2023
@jkczyz
jkczyz
self-requested a review
February 15, 2023 23:02
@codecov-commenter
codecov-commenter
commented
Feb 15, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Base: 87.21% // Head: 87.02% // Decreases project coverage by -0.20%⚠️
Coverage data is based on head (2f14891) compared to base (30b9d9f).

Patch coverage: 92.70% of modified lines in pull request are covered.
❗ Current head 2f14891 differs from pull request most recent head a2b956d. Consider uploading reports for the commit a2b956d to get more accurate results
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@ Coverage Diff @@## main #2037 +/- ##
==========================================
- Coverage 87.21% 87.02% -0.20% 
==========================================
Files 100 100 Lines 44023 44543 +520 Branches 44023 44543 +520 ==========================================
+ Hits 38396 38762 +366 - Misses 5627 5781 +154 
Impacted FilesCoverage Δ
lightning/src/ln/channelmanager.rs85.84% <ø> (ø)
lightning/src/routing/router.rs92.79% <ø> (ø)
lightning/src/util/events.rs23.63% <ø> (-0.58%)⬇️
lightning/src/ln/payment_tests.rs95.94% <82.05%> (+<0.01%)⬆️
lightning/src/ln/outbound_payment.rs80.40% <84.21%> (+0.11%)⬆️
lightning/src/ln/chanmon_update_fail_tests.rs97.31% <100.00%> (ø)
lightning/src/ln/functional_test_utils.rs88.50% <100.00%> (ø)
lightning/src/ln/functional_tests.rs97.29% <100.00%> (+0.01%)⬆️
lightning/src/ln/monitor_tests.rs98.73% <100.00%> (ø)
lightning/src/ln/onion_route_tests.rs95.45% <100.00%> (ø)
... and 11 more
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
☔ View full report at Codecov.

📢 Do you have feedback about the report comment? Let us know in this issue.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Needs rebase now that #2008 landed.
@TheBlueMatt
TheBlueMatt
commented
Feb 16, 2023

Copy link


Copy Markdown

Collaborator
Change LGTM, though.
@valentinewallace
Remove pending probes on update_fail
a2b956d
Previously we had a memory leak where probes would not be removed from
outbound_payments on htlc fail
@TheBlueMatt
TheBlueMatt
merged commit a170478
into
lightningdevkit:mainFeb 16, 2023
PXplod pushed a commit
to bitlightlabs/rust-lightning
that referenced
this pull request
Sep 30, 2024
@zoedberg
Merge tag 'v0.0.114' intoe496878
7784086
0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries"
API Updates
===========
* `InvoicePayer` has been removed and its features moved directly into
`ChannelManager`. As such it now requires a simplified `Router` and supports
`send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was
removed in favor of the automated retries. Invoice payment utilities in
`lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc).
* `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner`
split out in anticipation of future schnorr/taproot support (lightningdevkit#1967).
* The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`,
and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930).
* `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key`
and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070).
* A `lightning-transaction-sync` crate has been added which implements keeping
LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can
only be used on nodes that *never* ran a previous version of LDK.
* `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996).
* `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980).
* BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as
associated builders have been added. Such invoices cannot yet be paid due to
missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926).
* A `lightning-custom-message` crate has been added to make combining multiple
custom messages into one enum/handler easier (lightningdevkit#1832).
* `Event::PaymentPathFailure` is now generated for failure to send an HTLC
over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043).
* `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968).
* `ChannelManager::list_recent_payments` was added (lightningdevkit#1873).
* `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962).
* `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985).
* The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878)
* bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057).
* Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961).
* `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948).
* Macros which implement serialization using TLVs or straight writing of struct
fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977).
Backwards Compatibility
=======================
* Any inbound payments with a custom final CLTV delta will be rejected by LDK
if you downgrade prior to receipt (lightningdevkit#1878).
* `Event::PaymentPathFailed::network_update` will always be `None` if an
0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043).
* `Event::PaymentPathFailed::all_paths_removed` will always be false if an
0.0.114-generated event is read by a prior version of LDK. Users who rely on
it to determine payment retries should migrate to `Event::PaymentFailed`, in
a separate release prior to upgrading to LDK 0.0.114 if downgrading is
supported (lightningdevkit#2043).
Performance Improvements
========================
* Channel data is now stored per-peer and channel updates across multiple
peers can be operated on simultaneously (lightningdevkit#1507).
* Routefinding is roughly 1.5x faster (lightningdevkit#1799).
* Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016).
* Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040).
* `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974).
Bug Fixes
=========
* Fixed a bug where a delay in processing a `PaymentSent` event longer than the
time taken to persist a `ChannelMonitor` update, when occurring immediately
prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048).
* Fixed spurious rejections of rapid gossip sync data when the graph has been
updated by other means between gossip syncs (lightningdevkit#2046).
* Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos`
is set (lightningdevkit#1935).
* Resolved an issue where the `ChannelManager::get_persistable_update_future`
future would fail to wake until a second notification occurs (lightningdevkit#2064).
* Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037).
* Fixed a deadlock on some platforms at least when using async `ChannelMonitor`
updating (lightningdevkit#2006).
* Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964).
* In some cases when payment sending fails on our local channel retries no
longer take the same path and thus never succeed (lightningdevkit#2014).
* Retries for spontaneous payments have been fixed (lightningdevkit#2002).
* Return an `Err` if `lightning-persister` fails to read the directory listing
rather than panicing (lightningdevkit#1943).
* `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035)
Security
========
0.0.114 fixes several denial-of-service vulnerabilities which are reachable from
untrusted input from channel counterparties or in deployments accepting inbound
connections or channels. It also fixes a denial-of-service vulnerability in rare
cases in the route finding logic.
* The number of pending un-funded channels as well as peers without funded
channels is now limited to avoid denial of service (lightningdevkit#1988).
* A second `channel_ready` message received immediately after the first could
lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf
support in LDK 0.0.107.
* A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount
being sent (including previous-hop fees) is equal to a channel's capacity
while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with
historical data tracking in LDK 0.0.112.
In total, this release features 130 files changed, 21457 insertions, 10113
deletions in 343 commits from 18 authors, in alphabetical order:
* Alec Chen
* Allan Douglas R. de Oliveira
* Andrei
* Arik Sosman
* Daniel Granhão
* Duncan Dean
* Elias Rohrer
* Jeffrey Czyz
* John Cantrell
* Kurtsley
* Matt Corallo
* Max Fang
* Omer Yacine
* Valentine Wallace
* Viktor Tigerström
* Wilmer Paulino
* benthecarman
* jurvis
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TheBlueMattTheBlueMattTheBlueMatt approved these changes
@jkczyzjkczyzjkczyz approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.114

Development

Successfully merging this pull request may close these issues.

Remove pending outbound probes on resolution


4 participants

@valentinewallace@codecov-commenter@TheBlueMatt@jkczyz