Route blinding MVP by valentinewallace · Pull Request #2413 · lightningdevkit/rust-lightning

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace · 2023-07-13T19:32:56Z

Support sending to blinded payment paths and receiving to 1-hop paths. Partially addresses #1970. Error handling, forwarding and receiving to multi-hop blinded paths will be completed in follow-up.

finish tests

~~Based on #2411, #2412, #2128, #2459, #2514, #2503~~

codecov-commenter · 2023-08-26T21:33:27Z

Codecov Report

❌ Patch coverage is 93.42105% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.61%. Comparing base (448b191) to head (ebdc4ae).
⚠️ Report is 5389 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/onion_utils.rs	76.00%	6 Missing ⚠️
lightning/src/ln/msgs.rs	92.30%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2413 +/- ##
==========================================
+ Coverage 90.45% 90.61% +0.15% 
==========================================
Files 112 113 +1 Lines 58564 59002 +438 Branches 58564 59002 +438 ==========================================
+ Hits 52976 53466 +490 + Misses 5588 5536 -52

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

valentinewallace · 2023-08-30T20:21:37Z

Rather than splitting the pr into handle-it-all-then-add-error-handling, can we split it into "handle sending, with all error handling, handle receiving, with all error handling, and then handle forwarding, with all error handling"?

I'm still looking into this but we can't materially test any error handling until we have forwarding, because 1-hop blinded paths are supposed to error the same as unblinded payments, i.e. not wipe the error or return malformed. Another option would be to split forwarding+error handling off into its own PR and land sending/1-hop receiving first.

valentinewallace · 2023-08-31T21:39:09Z

Removed forwarding support for now, so all forwarding + error handling will be done in #2540 (should be updated tomorrow).

valentinewallace · 2023-09-11T15:00:57Z

Rebased and removed support for receiving to multi-hop blinded paths. This way we can ship an MVP BOLT 12 in 117 and complete route blinding support in 118, to avoid holding up the current release.

jkczyz

Looks good to squash!

This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

Useful until forwarding and receiving to multi-hop blinded paths is supported.

jkczyz · 2023-09-13T18:36:14Z

+	let ev = remove_first_msg_event_to_node(&nodes[1].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[0], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), false, None);
+
+	let ev = remove_first_msg_event_to_node(&nodes[2].node.get_our_node_id(), &mut events);
+	pass_along_path(&nodes[0], expected_route[1], amt_msat, payment_hash.clone(),
+ Some(payment_secret), ev.clone(), true, None);
+	claim_payment_along_route(&nodes[0], expected_route, false, payment_preimage);


nit: ev clones not needed

TheBlueMatt · 2023-09-13T20:17:02Z

+ amt_msat: u64,
+ total_msat: u64,


Heh, can we get away from the legacy names and use more descriptive names since we're adding new code here? Something about mpp_part_amt_msat and total_payment_msat?

TheBlueMatt · 2023-09-13T20:21:31Z

 }
- if recipient_onion.payment_secret.is_none() && route.paths.len() > 1 {
+ if recipient_onion.payment_secret.is_none() && route.paths.len() > 1
+ && !route.paths.iter().any(|p| p.blinded_tail.is_some())


Shouldnt this be all?

TheBlueMatt · 2023-09-13T20:42:18Z

+ return Err(DecodeError::InvalidValue)
+ }
+ let enc_tlvs = encrypted_tlvs_opt.ok_or(DecodeError::InvalidValue)?.0;
+ let enc_tlvs_ss = node_signer.ecdh(Recipient::Node, &blinding_point, None)


I feel like we should do blinded paths to a random key and not reuse the node id. That wouldn't avoid the new trait pass cause we need it for forwards, but for nodes with the node id on a separate hardware device/VLS we'd avoid the need to call out to the hardware device here for receives, which I think would be very nice.

0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks" API Updates =========== * BOLT12 sending and receiving is now supported as an alpha feature. You may run into unexpected issues and will need to have a direct connection with the offer's blinded path introduction points as messages are not yet routed. We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039). * `ConfirmationTarget` has been rewritten to provide information about the specific use LDK needs the feerate estimate for, rather than the generic low-, medium-, and high-priority estimates. This allows LDK users to more accurately target their feerate estimates (lightningdevkit#2660). For those wishing to retain their existing behavior, see the table below for conversion. * `ChainHash` is now used in place of `BlockHash` where it represents the genesis block (lightningdevkit#2662). * `lightning-invoice` payment utilities now take a `Deref` to `AChannelManager` (lightningdevkit#2652). * `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599). * `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670) * `Display` is now implemented for `OutPoint` (lightningdevkit#2649). * `Features::from_be_bytes` is now provided (lightningdevkit#2640). For those moving to the new `ConfirmationTarget`, the new variants in terms of the old mempool/low/medium/high priorities are as follows: * `OnChainSweep` = `HighPriority` * `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)` * `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum` * `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250` * `AnchorChannelFee` = `Background` * `NonAnchorChannelFee` = `Normal` * `ChannelCloseMinimum` = `Background` Bug Fixes ========= * Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a channel which did not exist would immediately hang holding several key `ChannelManager`-internal locks (lightningdevkit#2657). * Channel information updates received from a failing HTLC are no longer applied to our `NetworkGraph`. This prevents a node which we attempted to route a payment through from being able to learn the sender of the payment. In some rare cases, this may result in marginally reduced payment success rates (lightningdevkit#2666). * Anchor outputs are now properly considered when calculating the amount available to send in HTLCs. This can prevent force-closes in anchor channels when sending payments which overflow the available balance (lightningdevkit#2674). * A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC, then reconnects prior to sending a `commitment_signed` (thus retransmitting their `update_fulfill_htlc`) may result in the channel stalling and being unable to make progress (lightningdevkit#2661). * In exceedingly rare circumstances, messages intended to be sent to a peer prior to reconnection can be sent after reconnection. This could result in undefined channel state and force-closes (lightningdevkit#2663). Backwards Compatibility ======================= * Creating a blinded path to receive a payment then downgrading to LDK prior to 0.0.117 may result in failure to receive the payment (lightningdevkit#2413). * Calling `ChannelManager::pay_for_offer` or `ChannelManager::create_refund_builder` may prevent downgrading to LDK prior to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039). Node Compatibility ================== * LDK now sends a bogus `channel_reestablish` message to peers when they ask to resume an unknown channel. This should cause LND nodes to force-close and broadcast the latest channel state to the chain. In order to trigger this when we wish to force-close a channel, LDK now disconnects immediately after sending a channel-closing `error` message. This should result in cooperative peers also working to confirm the latest commitment transaction when we wish to force-close (lightningdevkit#2658). Security ======== 0.0.118 expands mitigations against transaction cycling attacks to non-anchor channels, though note that no mitigations which exist today are considered robust to prevent the class of attacks. * In order to mitigate against transaction cycling attacks, non-anchor HTLC transactions are now properly re-signed before broadcasting (lightningdevkit#2667). In total, this release features 61 files changed, 3470 insertions, 1503 deletions in 85 commits from 12 authors, in alphabetical order: * Antonio Yang * Elias Rohrer * Evan Feenstra * Fedeparma74 * Gursharan Singh * Jeffrey Czyz * Matt Corallo * Sergi Delgado Segura * Vladimir Fomene * Wilmer Paulino * benthecarman * slanesuke

valentinewallace mentioned this pull request Jul 13, 2023
BOLT12 Offers Tracking Issue #1970
Open
60 tasks

valentinewallace added this to the 0.0.117 milestone Jul 13, 2023

This was referenced Jul 13, 2023
Add API for constructing blinded payment paths #2412
Merged
Support constructing blinded path onion keys #2411
Merged

valentinewallace added the blocked on dependent pr label Jul 27, 2023

valentinewallace mentioned this pull request Aug 25, 2023
Aggregate BlindedPayInfo for blinded routes #2514
Merged

valentinewallace force-pushed the 2023-07-route-blinding branch 2 times, most recently from f4dc1aa to 8ed2025Compare August 26, 2023 20:28

valentinewallace force-pushed the 2023-07-route-blinding branch from 8ed2025 to 223e2deCompare August 29, 2023 18:07

valentinewallace marked this pull request as ready for review August 29, 2023 18:28

TheBlueMatt reviewed Aug 30, 2023
View reviewed changes

Comment threadlightning/src/ln/outbound_payment.rs

jkczyz self-requested a review August 31, 2023 17:45

valentinewallace force-pushed the 2023-07-route-blinding branch from 223e2de to 952e537Compare August 31, 2023 21:36

valentinewallace mentioned this pull request Aug 30, 2023
Route blinding: support forwarding as the intro node #2540
Merged

valentinewallace force-pushed the 2023-07-route-blinding branch 3 times, most recently from c2715bf to 80405b4Compare September 8, 2023 18:12

valentinewallace changed the title ~~Route blinding~~Route blinding sending and receivingSep 8, 2023

valentinewallace force-pushed the 2023-07-route-blinding branch from 80405b4 to 43a54c4Compare September 11, 2023 14:46

valentinewallace changed the title ~~Route blinding sending and receiving~~Route blinding MVPSep 11, 2023

valentinewallace removed the blocked on dependent pr label Sep 11, 2023

jkczyz reviewed Sep 12, 2023
View reviewed changes

Comment threadlightning/src/ln/onion_utils.rs
Comment threadlightning/src/ln/onion_utils.rs
Comment threadlightning/src/blinded_path/payment.rs Outdated
Comment threadlightning/src/ln/msgs.rs
Comment threadlightning/src/blinded_path/mod.rs Outdated

valentinewallace force-pushed the 2023-07-route-blinding branch from 43a54c4 to 4b39220Compare September 12, 2023 20:52

jkczyz reviewed Sep 12, 2023
View reviewed changes

Comment threadlightning/src/ln/onion_utils.rs
Comment threadlightning/src/ln/onion_utils.rs

valentinewallace added 2 commits September 12, 2023 18:11

Support paying blinded paths.
7b1e091

Parameterize InboundPayload reads with NodeSigner
154841b
This will be used in the next commit to deserialize encrypted TLVs for receiving to 1-hop blinded paths.

valentinewallace added 3 commits September 12, 2023 18:11

Support receiving to 1-hop blinded payment paths.
070f7e0

Test sending and receiving to 1-hop blinded paths
3e377a1

Only allow creating 1-hop blinded paths.
ebdc4ae
Useful until forwarding and receiving to multi-hop blinded paths is supported.

valentinewallace force-pushed the 2023-07-route-blinding branch from 4b39220 to ebdc4aeCompare September 12, 2023 22:12

jkczyz approved these changes Sep 13, 2023
View reviewed changes

TheBlueMatt approved these changes Sep 13, 2023
View reviewed changes

TheBlueMatt merged commit daf79f5 into lightningdevkit:mainSep 13, 2023