Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); BOLT 12 Offers utilities by jkczyz · Pull Request #2578 · lightningdevkit/rust-lightning · GitHub
Skip to content

BOLT 12 Offers utilities - #2578

Merged
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities
Oct 19, 2023
Merged

BOLT 12 Offers utilities#2578
TheBlueMatt merged 15 commits into
lightningdevkit:mainfrom
jkczyz:2023-09-offer-utilities

Conversation

@jkczyz

@jkczyzjkczyz commented Sep 14, 2023

Copy link
Copy Markdown
Contributor

Add utility methods to ChannelManager for:

  • creating OfferBuilder such that derived keys are used for the signing pubkey,
  • creating RefundBuilder such that derived keys are used for the payer id, and
  • sending an InvoiceRequest for an Offer such that derived keys are used for the payer id

When the ChannelManager is the payer (i.e., when creating Refund and InvoiceRequest), a PaymentId is required and used for tracking the outbound payment once the Bolt12Invoice is received.

One-hop blinded paths are used throughout until multi-hop blinded paths are supported.

Offer message handling and sending a Bolt12Invoice for a Refund such that derived keys are used for the signing pubkey are in #2039, which is based on this PR.

@jkczyz
jkczyz marked this pull request as ready for review September 14, 2023 22:44
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from be8bf45 to 0ff5708CompareSeptember 15, 2023 21:12
@jkczyzjkczyz mentioned this pull request Sep 15, 2023
/// The message contents to send in an [`OnionMessage`].
///
/// [`OnionMessage`]: crate::ln::msgs::OnionMessage
pub contents: T,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we leave a totally unbounded type that's gonna be a problem for bindings. Can we bound this by CustomOnionMessageContents (or if we need a general bound we could create an OnionMessageContents trait that we impl for BOLT12 messages as well)?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need a more general bound since OffersMessage is used, too.

Comment threadlightning/src/ln/channelmanager.rs Outdated
/// [`InvoiceRequest`]: crate::offers::invoice_request::InvoiceRequest
/// [`InvoiceRequestBuilder`]: crate::offers::invoice_request::InvoiceRequestBuilder
/// [`Bolt12Invoice`]: crate::offers::invoice::Bolt12Invoice
pub fn request_invoice(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? I'm not sure what the use is for this outside of simply paying an offer, which is presumably a different method.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is the "paying for an offer" method. I guess that I can rename it to be more user- rather than implementation-focused.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 0ff5708 to ff2702cCompareSeptember 18, 2023 19:04
}

/// A type used within a variant of [`OnionMessageContents`].
pub trait OnionMessageContentsType {}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an empty trait, should we just rename CustomOnionMessageContents to OnionMessageContents and impl it for Bolt12/etc? Its already basically what we want - extends Write and has a tlv_type.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we already have an OnionMessageContents enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then lets use that in PendingOnionMessage instead of a generic type T?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also seems arbitrary that a custom message handler isn't allowed to send BOLT12 messages.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't want a BOLT 12 handler to be able to send arbitrary ones. A custom handler can alway have their type wrap OffersMessage in a variant.

@TheBlueMattTheBlueMattSep 18, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean...why not? We could also drop OnionMessageContents or make it priv, and replace most usages with a trait.

@jkczyzjkczyzSep 19, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline and made the following changes:

  • Renamed OnionMessageContents to ParsedOnionMessageContents and gave it pub(super) visibility
  • Renamed CustomOnionMessageContents to OnionMessageContents and used it as a trait bound on most methods
  • Changed Payload to hold T instead of ParsedOnionMessageContents<T> where T: OnionMessageContents
  • Implemented OnionMessageContents for OffersMessage and ParsedOnionMessageContents. The latter is needed so that arbitrary Payload contents can be read and specific contents can be written

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 087941d to 8116d6eCompareSeptember 19, 2023 20:33
@codecov-commenter

codecov-commenter commented Sep 19, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 91 lines in your changes are missing coverage. Please review.

Comparison is base (0357caf) 89.04% compared to head (905028b) 88.96%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2578 +/- ##
==========================================
- Coverage 89.04% 88.96% -0.09% 
==========================================
Files 112 112 Lines 87527 87632 +105 Branches 87527 87632 +105 ==========================================
+ Hits 77942 77963 +21 - Misses 7345 7435 +90 + Partials 2240 2234 -6 
FilesCoverage Δ
lightning/src/events/mod.rs28.97% <ø> (-4.86%)⬇️
lightning/src/ln/msgs.rs76.81% <ø> (ø)
lightning/src/offers/offer.rs94.65% <100.00%> (+0.01%)⬆️
lightning/src/offers/parse.rs92.30% <ø> (ø)
lightning/src/offers/refund.rs93.67% <100.00%> (+0.02%)⬆️
lightning/src/onion_message/functional_tests.rs96.95% <100.00%> (+0.07%)⬆️
lightning/src/routing/router.rs94.07% <ø> (ø)
lightning/src/ln/outbound_payment.rs87.69% <90.32%> (+0.16%)⬆️
lightning/src/ln/peer_handler.rs58.79% <25.00%> (-0.10%)⬇️
lightning/src/blinded_path/mod.rs70.14% <33.33%> (-5.66%)⬇️
... and 4 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial look, just nits


#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind too much, but why remove this?

@jkczyzjkczyzSep 20, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored it but now the test needs to check against InvalidFirstHop, which I'm not sure if that is intended?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will reexamine this but want to get the other changes pushed for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'd also be fine with saving this for follow-up if you want to note it in #1970.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, other than the introduction node being wrong, when hardcoding a fix I'm also seeing this logging:

Received an onion message with path_id Some([]) and a reply_path

Instead of the expected:

Received an onion message with path_id None and a reply_path

But I don't understand how my change would affect the path_id. Any ideas?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nevermind on this. That was just because I had been building using some unpublished commits instead of during the rebase. 🤦‍♂️

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #2671

Comment threadlightning/src/blinded_path/mod.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 8116d6e to 2508211CompareSeptember 20, 2023 16:41
Comment threadlightning/src/blinded_path/mod.rs Outdated
@jkczyzjkczyz added this to the 0.0.117 milestone Sep 20, 2023
@jkczyzjkczyz mentioned this pull request Sep 18, 2023
60 tasks
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMattTheBlueMatt modified the milestones: 0.0.117, 0.0.118Sep 25, 2023
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 2508211 to 7ddbd57CompareOctober 11, 2023 14:37
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Rebased as there were a lot of merge conflicts from the last release.

Comment threadlightning/src/onion_message/messenger.rs Outdated

#[test]
fn test_no_onion_message_breakage() {
let one_hop_om = "020000000000000000000000000000000000000000000000000000000000000e01055600020000000000000000000000000000000000000000000000000000000000000e01ae0276020000000000000000000000000000000000000000000000000000000000000002020000000000000000000000000000000000000000000000000000000000000e0101022a0000000000000000000000000000014551231950b75fc4402da1732fc9bebf00109500000000000000000000000000000004106d000000000000000000000000000000fd1092202a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a2a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005600000000000000000000000000000000000000000000000000000000000000";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm that shouldn't be intended. It looks like that's because the first-hop peer in the reply path isn't connected.

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/offers.rs Outdated
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7ddbd57 to bd42eb0CompareOctober 13, 2023 22:46

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look into addressing the fuzz test comment still.

Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/onion_message/offers.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I think this LGTM, I guess its now based on #2662, but once that lands feel free to squash.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

There's probably gonna be some conflicts between this and #2599

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch 2 times, most recently from 4f520cc to 6b96866CompareOctober 16, 2023 13:54
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from e4e9fd7 to 7bf0469CompareOctober 18, 2023 19:43
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Speaking of timeouts, we should check if the payment has timed out per the AwaitingInvoice::retry_stragety (assuming its a time-based one) in the timer tick too.

Discussed offline. It would be better to keep the retry strategy separate from when to expire waiting for an invoice.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Only comment left form my end is #2578 (comment), which you can fix or not, but either way lets squash and I'll ack :)

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Sorry needs a small-ish rebase after #2599.

@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7bf0469 to 7a198a8CompareOctober 18, 2023 21:53
OnionMessageProvider is a super-trait of OnionMessageHandler, but they
don't need to be used separately. Additionally, the former is misplaced
in the events module. Remove OnionMessageProvider and add it's only
method, next_onion_message_for_peer, into OnionMessageHandler.
OnionMessenger can send onion message responses from its handlers using
respond_with_onion_message, which finds a path to the destination and
enqueues the response for sending. Generalize this as it can be used not
only for responses but for initial sends as well.
In preparation for needing the name OnionMessageContents for a trait to
bound methods, rename it to ParsedOnionMessageContents. In the next
commit, it's use will be limited to reading only, and the new trait will
be a bound on method parameters instead.

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after rebase

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs
Comment threadlightning/src/onion_message/messenger.rs Outdated
Rename CustomOnionMessageContents to OnionMessageContents and use it as
a trait bound on messages passed to OnionMessenger methods. This allows
using the trait in an upcoming commit as a bound on the contents of
PendingOnionMessage.
Also, make ParsedOnionMessageContent implement OnionMessageContents so
that Payload can be bounded by OnionMessageContents directly, but used
when either reading a ParsedOnionMessageContent or writing a specific
type of OnionMessageContents (e.g., OffersMessage).
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from 7a198a8 to bf980c9CompareOctober 18, 2023 23:25
When constructing onion messages to send initially (opposed to replying
to one from a handler), the user must construct an OnionMessagePath first
before calling OnionMessener::send_onion_message. Additionally, having a
reference to OnionMessener isn't always desirable. For instance, in an
upcoming commit, ChannelManager will implement OffersMessageHandler,
which OnionMessenger needs a reference to. If ChannelManager had a
reference to OnionMessenger, too, there would be a dependency cycle.
Instead, modify OffersMessageHandler and CustomOnionMessageHandler's
interfaces to include a method for releasing pending onion messages.
That way, ChannelManager may, for instance, construct and enqueue an
InvoiceRequest for sending without needing a reference to
OnionMessenger.
Additionally, OnionMessenger has responsibility for path finding just as
it does when replying to messages from a handler. It performs this when
extracting messages from the handlers before returning the next message
to send to a peer.
This allows for specifying the introduction node as the message
recipient.
PendingOutboundPayment::AwaitingInvoice counts the number of timer ticks
that have passed awaiting a Bolt12Invoice for an InvoiceRequest. When a
constant INVOICE_REQUEST_TIMEOUT_TICKS has passed, the payment is
forgotten. However, this mechanism is insufficient for the Refund
scenario, where the Refund's expiration should be used instead.
Change AwaitingInvoice to store an absolute expiry instead. When
removing stale payments, pass the `SystemTime` in `std` and the highest
block time minus two hours in `no-std`.
Add utility functions to ChannelManager for creating OfferBuilder,
and RefundBuilder such that derived keys are used for the signing
pubkey and payer id, respectively. This allows for stateless
verification of any InvoiceRequest and Invoice messages.
Later, blinded paths can be included in the returned builders.
Also tracks future payments using the given PaymentId such that the
corresponding Invoice is paid only once.
While this doesn't add much privacy over not including any blinded
paths, it allows us to exercise code for receiving on blinded paths.
For consistency with other functions and doc cleanliness.
@jkczyz
jkczyzforce-pushed the 2023-09-offer-utilities branch from bf980c9 to 905028bCompareOctober 18, 2023 23:33
@jkczyz

Copy link
Copy Markdown
ContributorAuthor

Sorry needs a small-ish rebase after #2599.

Rebased. Needed to expose ParsedOnionMessageContents because of this.

/// Creates a [`RefundBuilder`] such that the [`Refund`] it builds is recognized by the
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to mention the two-hour thing here, its a bit awkward its buried in the timer_tick_occurred docs. Can happen in a followup, though.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

@TheBlueMatt
TheBlueMatt merged commit 1852715 into lightningdevkit:mainOct 19, 2023
/// [`ChannelManager`] when handling [`Bolt12Invoice`] messages for the refund. The builder will
/// have the provided expiration set. Any changes to the expiration on the returned builder will
/// not be honored by [`ChannelManager`].
///

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh also we should probably say something about how you can use abandon_payment to revoke a refund once issued (if it hasn't been paid). Its somewhat non-intuitive to me that that is true.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #2039.

PXplod pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.118 - Oct 23, 2023 - "Just the Twelve Sinks"
API Updates
===========
* BOLT12 sending and receiving is now supported as an alpha feature. You may
run into unexpected issues and will need to have a direct connection with
the offer's blinded path introduction points as messages are not yet routed.
We are seeking feedback from early testers (lightningdevkit#2578, lightningdevkit#2039).
* `ConfirmationTarget` has been rewritten to provide information about the
specific use LDK needs the feerate estimate for, rather than the generic
low-, medium-, and high-priority estimates. This allows LDK users to more
accurately target their feerate estimates (lightningdevkit#2660). For those wishing to
retain their existing behavior, see the table below for conversion.
* `ChainHash` is now used in place of `BlockHash` where it represents the
genesis block (lightningdevkit#2662).
* `lightning-invoice` payment utilities now take a `Deref` to
`AChannelManager` (lightningdevkit#2652).
* `peel_onion` is provided to statelessly decode an `OnionMessage` (lightningdevkit#2599).
* `ToSocketAddrs` + `Display` are now impl'd for `SocketAddress` (lightningdevkit#2636, lightningdevkit#2670)
* `Display` is now implemented for `OutPoint` (lightningdevkit#2649).
* `Features::from_be_bytes` is now provided (lightningdevkit#2640).
For those moving to the new `ConfirmationTarget`, the new variants in terms of
the old mempool/low/medium/high priorities are as follows:
* `OnChainSweep` = `HighPriority`
* `MaxAllowedNonAnchorChannelRemoteFee` = `max(25 * 250, HighPriority * 10)`
* `MinAllowedAnchorChannelRemoteFee` = `MempoolMinimum`
* `MinAllowedNonAnchorChannelRemoteFee` = `Background - 250`
* `AnchorChannelFee` = `Background`
* `NonAnchorChannelFee` = `Normal`
* `ChannelCloseMinimum` = `Background`
Bug Fixes
=========
* Calling `ChannelManager::close_channel[_with_feerate_and_script]` on a
channel which did not exist would immediately hang holding several key
`ChannelManager`-internal locks (lightningdevkit#2657).
* Channel information updates received from a failing HTLC are no longer
applied to our `NetworkGraph`. This prevents a node which we attempted to
route a payment through from being able to learn the sender of the payment.
In some rare cases, this may result in marginally reduced payment success
rates (lightningdevkit#2666).
* Anchor outputs are now properly considered when calculating the amount
available to send in HTLCs. This can prevent force-closes in anchor channels
when sending payments which overflow the available balance (lightningdevkit#2674).
* A peer that sends an `update_fulfill_htlc` message for a forwarded HTLC,
then reconnects prior to sending a `commitment_signed` (thus retransmitting
their `update_fulfill_htlc`) may result in the channel stalling and being
unable to make progress (lightningdevkit#2661).
* In exceedingly rare circumstances, messages intended to be sent to a peer
prior to reconnection can be sent after reconnection. This could result in
undefined channel state and force-closes (lightningdevkit#2663).
Backwards Compatibility
=======================
* Creating a blinded path to receive a payment then downgrading to LDK prior to
0.0.117 may result in failure to receive the payment (lightningdevkit#2413).
* Calling `ChannelManager::pay_for_offer` or
`ChannelManager::create_refund_builder` may prevent downgrading to LDK prior
to 0.0.118 until the payment times out and has been removed (lightningdevkit#2039).
Node Compatibility
==================
* LDK now sends a bogus `channel_reestablish` message to peers when they ask to
resume an unknown channel. This should cause LND nodes to force-close and
broadcast the latest channel state to the chain. In order to trigger this
when we wish to force-close a channel, LDK now disconnects immediately after
sending a channel-closing `error` message. This should result in cooperative
peers also working to confirm the latest commitment transaction when we wish
to force-close (lightningdevkit#2658).
Security
========
0.0.118 expands mitigations against transaction cycling attacks to non-anchor
channels, though note that no mitigations which exist today are considered robust
to prevent the class of attacks.
* In order to mitigate against transaction cycling attacks, non-anchor HTLC
transactions are now properly re-signed before broadcasting (lightningdevkit#2667).
In total, this release features 61 files changed, 3470 insertions, 1503
deletions in 85 commits from 12 authors, in alphabetical order:
* Antonio Yang
* Elias Rohrer
* Evan Feenstra
* Fedeparma74
* Gursharan Singh
* Jeffrey Czyz
* Matt Corallo
* Sergi Delgado Segura
* Vladimir Fomene
* Wilmer Paulino
* benthecarman
* slanesuke
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jkczyz@codecov-commenter@TheBlueMatt@valentinewallace@benthecarman