Do not fail to apply RGS updates for removed channels by TheBlueMatt · Pull Request #2046 · lightningdevkit/rust-lightning

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt · 2023-02-23T19:27:02Z

If we receive a Rapid Gossip Sync update for channels where we are
missing the existing channel data, we should ignore the missing
channel. This can happen in a number of cases, whether because we
received updated channel information via an onion error from an
HTLC failure or because we've partially synced the graph from a
peer over the standard lightning P2P protocol.

codecov-commenter · 2023-02-23T20:15:37Z

Codecov Report

Patch coverage: 58.33% and project coverage change: +1.21 🎉

Comparison is base (b5e5435) 87.25% compared to head (d2f5dc0) 88.47%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@ Coverage Diff @@## main #2046 +/- ##
==========================================
+ Coverage 87.25% 88.47% +1.21% 
==========================================
Files 100 100 Lines 44480 50694 +6214 Branches 44480 50694 +6214 ==========================================
+ Hits 38810 44850 +6040 - Misses 5670 5844 +174

Impacted Files	Coverage Δ
lightning-rapid-gossip-sync/src/lib.rs	`77.61% <0.00%> (ø)`
lightning/src/util/macro_logger.rs	`91.48% <ø> (ø)`
lightning-rapid-gossip-sync/src/processing.rs	`91.70% <60.00%> (-1.13%)`	⬇️
lightning-background-processor/src/lib.rs	`81.88% <100.00%> (ø)`
lightning/src/sync/nostd_sync.rs	`42.10% <0.00%> (-0.40%)`	⬇️
lightning/src/sync/mod.rs	`50.00% <0.00%> (ø)`
lightning/src/sync/test_lockorder_checks.rs	`100.00% <0.00%> (ø)`
lightning/src/ln/functional_tests.rs	`97.53% <0.00%> (+0.24%)`	⬆️
lightning/src/ln/functional_test_utils.rs	`88.74% <0.00%> (+0.31%)`	⬆️
lightning/src/routing/router.rs	`93.75% <0.00%> (+0.97%)`	⬆️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

naumenkogs · 2023-02-24T11:45:33Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

wpaulino · 2023-02-24T19:56:03Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Somewhat. This is on the incremental update path, so the payload will only include the fields that were mutated based on the prior update. If you don't have a prior update, then it's possible (most likely?) you end up with an incorrect policy for said channel, allowing pathfinding to try the channel, only to end up with a failure. If we skip applying the update, we won't try the channel at all until we receive the full policy.

If you keep syncing the graph via RGS though, I would imagine you won't get the full update again, since you would just continue to receive incremental updates. If we were to apply the channel update, even with a potentially incorrect policy, we'll at least have a chance at obtaining the full policy once we try to route a HTLC over it and it fails.

TheBlueMatt · 2023-02-24T20:27:05Z

Doesn't this change simply reduces the amount of information provided to the caller? Not sure I understand why this is useful.

Well, we change from returning an Err and refusing to process further updates from the RGS file to continuing to process further updates and just log it. I suppose we could keep the Err return value and still process further updates, but its not really an error - it may just mean the user synced part of the graph from some other source.

arik-so · 2023-02-24T20:54:33Z

This change is actually more fine-grained than that. Previously we were skipping missing updates if we didn't have the channel. However, if we had the channel but were missing data for the given direction, we would previously fail, whereas with this change we now skip it.

My question, however, is this: Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction. If both directions' data are purged, do we then automatically also purge the channel from the network graph?

arik-so · 2023-02-24T20:51:59Z

- action: ErrorAction::IgnoreError,
- })?;
-
+ if let Some(directional_info) =


it's probably just me, but I think the right-hand-side of an if let expression shouldn't have 4 chained expressions, plus one nested one. Can we extract a variable or two?

TheBlueMatt · 2023-02-24T22:42:45Z

Under which circumstances can we lose only a channel's directional data, but either retain the channel or at the very least retain the other direction.

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

naumenkogs · 2023-02-27T12:16:20Z

The change looks good.
ACK b4f5f9b

TheBlueMatt · 2023-02-27T17:59:30Z

Rebased to fix some "the neighboring line changed" issues.

wpaulino · 2023-02-27T18:10:03Z

In remove_stale_channels_and_tracking_with_time we remove directions individually when that direction has gone stale. We only remove the full channel data when both directions are stale.

Any reason we don't remove upon only one direction being stale (ref lightning/bolts#767)?

TheBlueMatt · 2023-02-27T22:40:41Z

Oh, I'm sorry, I misread remove_stale_channels_and_tracking_with_time, we do indeed remove if either direction is stale. Still, we only do so if we didn't receive the channel recently (otherwise we'd prune stuff while syncing, which would be bad), so you can still hit this issue.

arik-so · 2023-02-28T03:57:25Z

Sorry, can you elaborate on the circumstances under which we'd hit this issue? When exactly would just one direction get pruned?

TheBlueMatt · 2023-02-28T04:46:06Z

So remove_stale_channels_and_tracking_with_time prunes one side at a time. It then removes the full channel data if either side has been removed but only if we've known about the channel for a while. If we haven't known about the channel for a while, we assume it's because we're currently syncing the graph (and we don't want to remove it - we'll learn about the other side of the channel in a minute or two). If we're doing P2P sync, this can definitely cause some channels with only one side.

arik-so · 2023-02-28T04:49:36Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

I thought some folks complained that when they downloaded an incremental update, it hit this snag after only applying the original full sync.

If we receive a Rapid Gossip Sync update for channels where we are missing the existing channel data, we should ignore the missing channel. This can happen in a number of cases, whether because we received updated channel information via an onion error from an HTLC failure or because we've partially synced the graph from a peer over the standard lightning P2P protocol.

... by using explicit paths rather than requiring imports.

TheBlueMatt · 2023-02-28T18:08:10Z

So that basically means that this error should only have been occurring for people who mixed RGS with p2p syncing?

It looks like we currently only prune there, so as long as the RGS server always provides both sides of a channel RGS-only shouldn't cause it. However, if you get one side of a channel updated via a channel failure, it may have a newer timestamp than the other side, which could cause this as well. Finally, I'm not super convinced RGS will never generate such updates - I don't see any explicit filtering code for this, and while the LDK removal of stale directional updates should make it less likely, but not impossible.

Rebased on upstream to fix CI and included one (squashed fix):

diff --git a/lightning-rapid-gossip-sync/src/processing.rs b/lightning-rapid-gossip-sync/src/processing.rs
index 342517bc0..f1072e26b 100644
--- a/lightning-rapid-gossip-sync/src/processing.rs+++ b/lightning-rapid-gossip-sync/src/processing.rs@@ -171,7 +171,7 @@ impl<NG: Deref<Target=NetworkGraph<L>>, L: Deref> RapidGossipSync<NG, L> where L
let read_only_network_graph = network_graph.read_only();
if let Some(directional_info) =
read_only_network_graph.channels().get(&short_channel_id)
- .map(|channel| channel.get_directional_info(channel_flags)).unwrap_or(None)+ .and_then(|channel| channel.get_directional_info(channel_flags))
{
synthetic_update.cltv_expiry_delta = directional_info.cltv_expiry_delta;
synthetic_update.htlc_minimum_msat = directional_info.htlc_minimum_msat;

0.0.114 - Mar 3, 2023 - "Faster Async BOLT12 Retries" API Updates =========== * `InvoicePayer` has been removed and its features moved directly into `ChannelManager`. As such it now requires a simplified `Router` and supports `send_payment_with_retry` (and friends). `ChannelManager::retry_payment` was removed in favor of the automated retries. Invoice payment utilities in `lightning-invoice` now call the new code (lightningdevkit#1812, lightningdevkit#1916, lightningdevkit#1929, lightningdevkit#2007, etc). * `Sign`/`BaseSign` has been renamed `ChannelSigner`, with `EcdsaChannelSigner` split out in anticipation of future schnorr/taproot support (lightningdevkit#1967). * The catch-all `KeysInterface` was split into `EntropySource`, `NodeSigner`, and `SignerProvider`. `KeysManager` implements all three (lightningdevkit#1910, lightningdevkit#1930). * `KeysInterface::get_node_secret` is now `KeysManager::get_node_secret_key` and is no longer required for external signers (lightningdevkit#1951, lightningdevkit#2070). * A `lightning-transaction-sync` crate has been added which implements keeping LDK in sync with the chain via an esplora server (lightningdevkit#1870). Note that it can only be used on nodes that *never* ran a previous version of LDK. * `Score` is updated in `BackgroundProcessor` instead of via `Router` (lightningdevkit#1996). * `ChainAccess::get_utxo` (now `UtxoAccess`) can now be resolved async (lightningdevkit#1980). * BOLT12 `Offer`, `InvoiceRequest`, `Invoice` and `Refund` structs as well as associated builders have been added. Such invoices cannot yet be paid due to missing support for blinded path payments (lightningdevkit#1927, lightningdevkit#1908, lightningdevkit#1926). * A `lightning-custom-message` crate has been added to make combining multiple custom messages into one enum/handler easier (lightningdevkit#1832). * `Event::PaymentPathFailure` is now generated for failure to send an HTLC over the first hop on our local channel (lightningdevkit#2014, lightningdevkit#2043). * `lightning-net-tokio` no longer requires an `Arc` on `PeerManager` (lightningdevkit#1968). * `ChannelManager::list_recent_payments` was added (lightningdevkit#1873). * `lightning-background-processor` `std` is now optional in async mode (lightningdevkit#1962). * `create_phantom_invoice` can now be used in `no-std` (lightningdevkit#1985). * The required final CLTV delta on inbound payments is now configurable (lightningdevkit#1878) * bitcoind RPC error code and message are now surfaced in `block-sync` (lightningdevkit#2057). * Get `historical_estimated_channel_liquidity_probabilities` was added (lightningdevkit#1961). * `ChannelManager::fail_htlc_backwards_with_reason` was added (lightningdevkit#1948). * Macros which implement serialization using TLVs or straight writing of struct fields are now public (lightningdevkit#1823, lightningdevkit#1976, lightningdevkit#1977). Backwards Compatibility ======================= * Any inbound payments with a custom final CLTV delta will be rejected by LDK if you downgrade prior to receipt (lightningdevkit#1878). * `Event::PaymentPathFailed::network_update` will always be `None` if an 0.0.114-generated event is read by a prior version of LDK (lightningdevkit#2043). * `Event::PaymentPathFailed::all_paths_removed` will always be false if an 0.0.114-generated event is read by a prior version of LDK. Users who rely on it to determine payment retries should migrate to `Event::PaymentFailed`, in a separate release prior to upgrading to LDK 0.0.114 if downgrading is supported (lightningdevkit#2043). Performance Improvements ======================== * Channel data is now stored per-peer and channel updates across multiple peers can be operated on simultaneously (lightningdevkit#1507). * Routefinding is roughly 1.5x faster (lightningdevkit#1799). * Deserializing a `NetworkGraph` is roughly 6x faster (lightningdevkit#2016). * Memory usage for a `NetworkGraph` has been reduced substantially (lightningdevkit#2040). * `KeysInterface::get_secure_random_bytes` is roughly 200x faster (lightningdevkit#1974). Bug Fixes ========= * Fixed a bug where a delay in processing a `PaymentSent` event longer than the time taken to persist a `ChannelMonitor` update, when occurring immediately prior to a crash, may result in the `PaymentSent` event being lost (lightningdevkit#2048). * Fixed spurious rejections of rapid gossip sync data when the graph has been updated by other means between gossip syncs (lightningdevkit#2046). * Fixed a panic in `KeysManager` when the high bit of `starting_time_nanos` is set (lightningdevkit#1935). * Resolved an issue where the `ChannelManager::get_persistable_update_future` future would fail to wake until a second notification occurs (lightningdevkit#2064). * Resolved a memory leak when using `ChannelManager::send_probe` (lightningdevkit#2037). * Fixed a deadlock on some platforms at least when using async `ChannelMonitor` updating (lightningdevkit#2006). * Removed debug-only assertions which were reachable in threaded code (lightningdevkit#1964). * In some cases when payment sending fails on our local channel retries no longer take the same path and thus never succeed (lightningdevkit#2014). * Retries for spontaneous payments have been fixed (lightningdevkit#2002). * Return an `Err` if `lightning-persister` fails to read the directory listing rather than panicing (lightningdevkit#1943). * `peer_disconnected` will now never be called without `peer_connected` (lightningdevkit#2035) Security ======== 0.0.114 fixes several denial-of-service vulnerabilities which are reachable from untrusted input from channel counterparties or in deployments accepting inbound connections or channels. It also fixes a denial-of-service vulnerability in rare cases in the route finding logic. * The number of pending un-funded channels as well as peers without funded channels is now limited to avoid denial of service (lightningdevkit#1988). * A second `channel_ready` message received immediately after the first could lead to a spurious panic (lightningdevkit#2071). This issue was introduced with 0conf support in LDK 0.0.107. * A division-by-zero issue was fixed in the `ProbabilisticScorer` if the amount being sent (including previous-hop fees) is equal to a channel's capacity while walking the graph (lightningdevkit#2072). The division-by-zero was introduced with historical data tracking in LDK 0.0.112. In total, this release features 130 files changed, 21457 insertions, 10113 deletions in 343 commits from 18 authors, in alphabetical order: * Alec Chen * Allan Douglas R. de Oliveira * Andrei * Arik Sosman * Daniel Granhão * Duncan Dean * Elias Rohrer * Jeffrey Czyz * John Cantrell * Kurtsley * Matt Corallo * Max Fang * Omer Yacine * Valentine Wallace * Viktor Tigerström * Wilmer Paulino * benthecarman * jurvis

TheBlueMatt added this to the 0.0.114 milestone Feb 23, 2023

arik-so self-requested a review February 24, 2023 20:06

arik-so self-assigned this Feb 24, 2023

arik-so reviewed Feb 24, 2023
View reviewed changes

naumenkogs previously approved these changes Feb 27, 2023
View reviewed changes

TheBlueMatt dismissed naumenkogs’s stale review via 55fff0aFebruary 27, 2023 17:59

TheBlueMatt force-pushed the 2023-02-rgs-robust-and-log branch from b4f5f9b to 55fff0aCompare February 27, 2023 17:59

wpaulino previously approved these changes Feb 27, 2023
View reviewed changes

benthecarman reviewed Feb 28, 2023
View reviewed changes

Comment threadlightning-rapid-gossip-sync/src/processing.rs Outdated

TheBlueMatt added 3 commits February 28, 2023 17:56

Make log macros more usable outside of the lightning crate
df20bcf
... by using explicit paths rather than requiring imports.

Add some basic logging to Rapid Gossip Sync processing
d2f5dc0

TheBlueMatt dismissed wpaulino’s stale review via d2f5dc0February 28, 2023 18:08

TheBlueMatt force-pushed the 2023-02-rgs-robust-and-log branch from 55fff0a to d2f5dc0Compare February 28, 2023 18:08

arik-so approved these changes Feb 28, 2023
View reviewed changes

wpaulino approved these changes Feb 28, 2023
View reviewed changes

wpaulino merged commit 0b1a64f into lightningdevkit:mainFeb 28, 2023