Sending to `Offer` without `signing_pubkey` by jkczyz · Pull Request #3017 · lightningdevkit/rust-lightning

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz · 2024-04-24T14:48:35Z

If an Offer contains a path, the blinded_node_id of the path's final hop can be used when signing an invoice. This helps to further reduce offer QR code size by making offer_node_id optional when offer_paths is set. Allow parsing (and thus sending to) such offers. Also replaces Refund's use of offer_paths with a new invreq_paths TLV in order to differentiate a Bolt12Invoice for an Offer from one for a Refund.

Receiving to an Offer without a signing_pubkey is not supported yet.

codecov-commenter · 2024-04-24T15:41:02Z

Codecov Report

Attention: Patch coverage is 92.42424% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 89.49%. Comparing base (2c0fcf2) to head (b7635c4).
Report is 29 commits behind head on main.

Files	Patch %	Lines
lightning/src/offers/invoice.rs	95.34%	2 Missing and 2 partials ⚠️
lightning/src/offers/invoice_request.rs	87.50%	4 Missing ⚠️
lightning/src/offers/offer.rs	90.47%	3 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs	81.81%	2 Missing ⚠️
lightning/src/offers/refund.rs	95.23%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@ Coverage Diff @@## main #3017 +/- ##
==========================================
+ Coverage 89.13% 89.49% +0.35% 
==========================================
Files 118 118 Lines 97492 100077 +2585 Branches 97492 100077 +2585 ==========================================
+ Hits 86903 89564 +2661 + Misses 8349 8261 -88 - Partials 2240 2252 +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TheBlueMatt · 2024-04-26T21:28:13Z

Feel free to squash imo

If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

Instead of reusing OfferTlvStream::paths, add a dedicated paths TLV to InvoiceRequestTlvStream such that it can be used in Refund. This allows for an Offer without a signing_pubkey and still be able to differentiate whether an invoice is for an offer or a refund.

When parsing a Bolt12Invoice use both the Offer's signing_pubkey and paths to determine if it is for an Offer or a Refund. Previously, an Offer was required to have a signing_pubkey. But now that it is optional, the Offers paths can be used to make the determination. Additionally, check that the invoice matches one of the blinded node ids from the paths' last hops.

valentinewallace · 2024-04-29T15:13:37Z

+ .iter()
+ .filter_map(|path| path.blinded_hops.last())
+ .any(|last_hop| fields.signing_pubkey == last_hop.blinded_node_id)


In the spec, it looks like we MUST verify that the signing pubkey matches a path we sent an invoice request to. I'm not certain it matters or if the spec should change but it looks like we don't specifically check that at the moment.

Yeah, this is a slight deviation as we don't have the blinded node id for the path that the invoice was sent over when parsing. We could do some sort of check at the handler level, but even that would require piping that data through.

TheBlueMatt · 2024-04-29T18:46:25Z

+ );
+ pending_offers_messages.push(message);
+ } else {
+ debug_assert!(false);


Its usually worth having a comment on these kinds of assertions as to why we think its not reachable.

…pubkey Sending to `Offer` without `signing_pubkey`

v0.0.123 - May 08, 2024 - "BOLT12 Dust Sweeping" API Updates =========== * To reduce risk of force-closures and improve HTLC reliability the default dust exposure limit has been increased to `MaxDustHTLCExposure::FeeRateMultiplier(10_000)`. Users with existing channels might want to consider using `ChannelManager::update_channel_config` to apply the new default (lightningdevkit#3045). * `ChainMonitor::archive_fully_resolved_channel_monitors` is now provided to remove from memory `ChannelMonitor`s that have been fully resolved on-chain and are now not needed. It uses the new `Persist::archive_persisted_channel` to inform the storage layer that such a monitor should be archived (lightningdevkit#2964). * An `OutputSweeper` is now provided which will automatically sweep `SpendableOutputDescriptor`s, retrying until the sweep confirms (lightningdevkit#2825). * After initiating an outbound channel, a peer disconnection no longer results in immediate channel closure. Rather, if the peer is reconnected before the channel times out LDK will automatically retry opening it (lightningdevkit#2725). * `PaymentPurpose` now has separate variants for BOLT12 payments, which include fields from the `invoice_request` as well as the `OfferId` (lightningdevkit#2970). * `ChannelDetails` now includes a list of in-flight HTLCs (lightningdevkit#2442). * `Event::PaymentForwarded` now includes `skimmed_fee_msat` (lightningdevkit#2858). * The `hashbrown` dependency has been upgraded and the use of `ahash` as the no-std hash table hash function has been removed. As a consequence, LDK's `Hash{Map,Set}`s no longer feature several constructors when LDK is built with no-std; see the `util::hash_tables` module instead. On platforms that `getrandom` supports, setting the `possiblyrandom/getrandom` feature flag will ensure hash tables are resistant to HashDoS attacks, though the `possiblyrandom` crate should detect most common platforms (lightningdevkit#2810, lightningdevkit#2891). * `ChannelMonitor`-originated requests to the `ChannelSigner` can now fail and be retried using `ChannelMonitor::signer_unblocked` (lightningdevkit#2816). * `SpendableOutputDescriptor::to_psbt_input` now includes the `witness_script` where available as well as new proprietary data which can be used to re-derive some spending keys from the base key (lightningdevkit#2761, lightningdevkit#3004). * `OutPoint::to_channel_id` has been removed in favor of `ChannelId::v1_from_funding_outpoint` in preparation for v2 channels with a different `ChannelId` derivation scheme (lightningdevkit#2797). * `PeerManager::get_peer_node_ids` has been replaced with `list_peers` and `peer_by_node_id`, which provide more details (lightningdevkit#2905). * `Bolt11Invoice::get_payee_pub_key` is now provided (lightningdevkit#2909). * `Default[Message]Router` now take an `entropy_source` argument (lightningdevkit#2847). * `ClosureReason::HTLCsTimedOut` has been separated out from `ClosureReason::HolderForceClosed` as it is the most common case (lightningdevkit#2887). * `ClosureReason::CooperativeClosure` is now split into `{Counterparty,Locally}Initiated` variants (lightningdevkit#2863). * `Event::ChannelPending::channel_type` is now provided (lightningdevkit#2872). * `PaymentForwarded::{prev,next}_user_channel_id` are now provided (lightningdevkit#2924). * Channel init messages have been refactored towards V2 channels (lightningdevkit#2871). * `BumpTransactionEvent` now contains the channel and counterparty (lightningdevkit#2873). * `util::scid_utils` is now public, with some trivial utilities to examine short channel ids (lightningdevkit#2694). * `DirectedChannelInfo::{source,target}` are now public (lightningdevkit#2870). * Bounds in `lightning-background-processor` were simplified by using `AChannelManager` (lightningdevkit#2963). * The `Persist` impl for `KVStore` no longer requires `Sized`, allowing for the use of `dyn KVStore` as `Persist` (lightningdevkit#2883, lightningdevkit#2976). * `From<PaymentPreimage>` is now implemented for `PaymentHash` (lightningdevkit#2918). * `NodeId::from_slice` is now provided (lightningdevkit#2942). * `ChannelManager` deserialization may now fail with `DangerousValue` when LDK's persistence API was violated (lightningdevkit#2974). Bug Fixes ========= * Excess fees on counterparty commitment transactions are now included in the dust exposure calculation. This lines behavior up with some cases where transaction fees can be burnt, making them effectively dust exposure (lightningdevkit#3045). * `Future`s used as an `std::...::Future` could grow in size unbounded if it was never woken. For those not using async persistence and using the async `lightning-background-processor`, this could cause a memory leak in the `ChainMonitor` (lightningdevkit#2894). * Inbound channel requests that fail in `ChannelManager::accept_inbound_channel` would previously have stalled from the peer's perspective as no `error` message was sent (lightningdevkit#2953). * Blinded path construction has been tuned to select paths more likely to succeed, improving BOLT12 payment reliability (lightningdevkit#2911, lightningdevkit#2912). * After a reorg, `lightning-transaction-sync` could have failed to follow a transaction that LDK needed information about (lightningdevkit#2946). * `RecipientOnionFields`' `custom_tlvs` are now propagated to recipients when paying with blinded paths (lightningdevkit#2975). * `Event::ChannelClosed` is now properly generated and peers are properly notified for all channels that as a part of a batch channel open fail to be funded (lightningdevkit#3029). * In cases where user event processing is substantially delayed such that we complete multiple round-trips with our peers before a `PaymentSent` event is handled and then restart without persisting the `ChannelManager` after having persisted a `ChannelMonitor[Update]`, on startup we may have `Err`d trying to deserialize the `ChannelManager` (lightningdevkit#3021). * If a peer has relatively high latency, `PeerManager` may have failed to establish a connection (lightningdevkit#2993). * `ChannelUpdate` messages broadcasted for our own channel closures are now slightly more robust (lightningdevkit#2731). * Deserializing malformed BOLT11 invoices may have resulted in an integer overflow panic in debug builds (lightningdevkit#3032). * In exceedingly rare cases (no cases of this are known), LDK may have created an invalid serialization for a `ChannelManager` (lightningdevkit#2998). * Message processing latency handling BOLT12 payments has been reduced (lightningdevkit#2881). * Latency in processing `Event::SpendableOutputs` may be reduced (lightningdevkit#3033). Node Compatibility ================== * LDK's blinded paths were inconsistent with other implementations in several ways, which have been addressed (lightningdevkit#2856, lightningdevkit#2936, lightningdevkit#2945). * LDK's messaging blinded paths now support the latest features which some nodes may begin relying on soon (lightningdevkit#2961). * LDK's BOLT12 structs have been updated to support some last-minute changes to the spec (lightningdevkit#3017, lightningdevkit#3018). * CLN v24.02 requires the `gossip_queries` feature for all peers, however LDK by default does not set it for those not using a `P2PGossipSync` (e.g. those using RGS). This change was reverted in CLN v24.02.2 however for now LDK always sets the `gossip_queries` feature. This change is expected to be reverted in a future LDK release (lightningdevkit#2959). Security ======== 0.0.123 fixes a denial-of-service vulnerability which we believe to be reachable from untrusted input when parsing invalid BOLT11 invoices containing non-ASCII characters. * BOLT11 invoices with non-ASCII characters in the human-readable-part may cause an out-of-bounds read attempt leading to a panic (lightningdevkit#3054). Note that all BOLT11 invoices containing non-ASCII characters are invalid. In total, this release features 150 files changed, 19307 insertions, 6306 deletions in 360 commits since 0.0.121 from 17 authors, in alphabetical order: * Arik Sosman * Duncan Dean * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * Keyue Bao * Matt Corallo * Orbital * Sergi Delgado Segura * Valentine Wallace * Willem Van Lint * Wilmer Paulino * benthecarman * jbesraa * olegkubrakov * optout * shaavan

jkczyz force-pushed the 2024-04-optional-pubkey branch from 94d12f9 to 825eb2fCompare April 24, 2024 21:07

jkczyz marked this pull request as ready for review April 24, 2024 21:11

jkczyz mentioned this pull request Apr 24, 2024
Optional description for Offer and Refund#3018
Merged

jkczyz requested a review from TheBlueMatt April 25, 2024 14:08

TheBlueMatt added this to the 0.0.123 milestone Apr 25, 2024

TheBlueMatt reviewed Apr 25, 2024
View reviewed changes

Comment threadlightning/src/offers/invoice_request.rs Outdated
Comment threadlightning/src/offers/invoice_request.rs

TheBlueMatt mentioned this pull request Apr 26, 2024
Introduce ResponseInstructions for OnionMessage Handling #2907
Merged

jkczyz force-pushed the 2024-04-optional-pubkey branch from 825eb2f to a70af2bCompare April 26, 2024 15:35

jkczyz added 4 commits April 26, 2024 18:14

Optional OfferContents::signing_pubkey
520fecf
If an Offer contains a path, the blinded_node_id of the path's final hop can be used as the signing pubkey. Make Offer::signing_pubkey and OfferContents::signing_pubkey return an Option to support this. Upcoming commits will implement this behavior.

Allow parsing Offer without signing_pubkey
94d5af6
If an offer has at least one path, it may omit the signing pubkey and use the blinded node id of the last hop of a path to sign an invoice. Allow parsing such offers but not yet creating them.

jkczyz force-pushed the 2024-04-optional-pubkey branch from a70af2b to b7635c4Compare April 26, 2024 23:14

valentinewallace self-requested a review April 29, 2024 14:20

valentinewallace approved these changes Apr 29, 2024
View reviewed changes

TheBlueMatt approved these changes Apr 29, 2024
View reviewed changes

TheBlueMatt merged commit 2b14cc4 into lightningdevkit:mainApr 29, 2024

TheBlueMatt added a commit to TheBlueMatt/rust-lightning that referenced this pull request May 6, 2024
Merge pull requestlightningdevkit#3017from jkczyz/2024-04-optional-…
c0986f2
…pubkey Sending to `Offer` without `signing_pubkey`