Support paying < total MPP value in BOLT 12 flow

[TheBlueMatt]

In #4373 I'm proposing supporting paying a partial payment (with the payment to be completed by another sender) in the BOLT 11 flow. It should also be supported in the BOLT 12 flow, though it'll likely require manual BOLT12 invoice handling (as we have to pay the blinded paths from the same invoice). May not get to this for 0.3, but def should for 0.4.

[Alkamal01]

I'd like to pick this up. My plan is to mirror what #4373 did for BOLT 11, add an optional params struct to send_payment_for_bolt12_invoice that accepts a declared_total_mpp_value_msat_override and a partial amount_msats, wiring the declared total into RecipientOnionFields::total_mpp_amount_msat while routing only the partial amount. Since all partial senders must use blinded paths from the same invoice, coordination of the invoice itself is left to the caller.

Happy to hear if there's a different approach in mind before I start.

[TheBlueMatt]

Hmm, not entirely sure if that'll work - send_payment_for_bolt12_invoice is really tied to UserConfig::manually_handle_bolt12_invoices and assumes that we requested the invoice, calling verify_bolt12_invoice which requires that the metadata used in the invreq matches our local cryptographic material. I think we should think about dropping UserConfig::manually_handle_bolt12_invoices now, forcing those who want to use that to go through the OffersMessageFlow by hand and then adapt send_payment_for_bolt12_invoice to both not require the cryptographic material match our local state and also not require that there be a matching pending payment. @jkczyz/ @shaavan are there any remaining uses of the manual handling logic that can't go through the flow at this point?

[Alkamal01]

Hmm, not entirely sure if that'll work - send_payment_for_bolt12_invoice is really tied to UserConfig::manually_handle_bolt12_invoices and assumes that we requested the invoice, calling verify_bolt12_invoice which requires that the metadata used in the invreq matches our local cryptographic material. I think we should think about dropping UserConfig::manually_handle_bolt12_invoices now, forcing those who want to use that to go through the OffersMessageFlow by hand and then adapt send_payment_for_bolt12_invoice to both not require the cryptographic material match our local state and also not require that there be a matching pending payment. @jkczyz/ @shaavan are there any remaining uses of the manual handling logic that can't go through the flow at this point?

Thanks for the clarification. Will hold off until @jkczyz and @shaavan weigh in before proceeding.

[jkczyz]

The biggest obstacles to using OffersMessageFlow are (1) reproducing the OffersMessageHandler logic implemented by ChannelManager and (2) accessing anything ChannelManager provided if OffersMessageFlow is not going to be used independent of ChannelManager.

Other than the boilerplate of handling messages, calling to OffersMessageFlow, and error handling (maybe that can be extracted into a re-usable utility?), this includes:

• pushing StaticInvoiceRequested event into ChannelManager's event queue, which is private
• calling ChannelManager::create_inbound_payment, which is public
• calling ChannelManager::initiate_async_payment, which is private
• calling ChannelManager::abandon_payment_with_reason, which is private
• accessing anything called by ChannelManager::send_payment_for_verified_bolt12_invoice (i.e., pending_outbound_payments).

See:

rust-lightning/lightning/src/ln/channelmanager.rs

Line 17037 in 423c1dc

> OffersMessageHandler for ChannelManager<M, T, ES, NS, SP, F, R, MR, L>

That is, if you just want to add some custom logic but still use ChannelManager, there's some boilerplate that needs to be written and that would need access to some of ChannelManger's internals.

[shaavan]

I agree with Jeff’s point.

To restate, OffersMessageFlow provides every utility needed for working with BOLT12 messages that are agnostic to any pending payment tracking. That’s it.

The original goal was to support non-ChannelManager users in implementing their own OffersMessageHandler, by handling everything except the payment storage itself.

However, building the pending_outbound_payment-like storage and tracking remains a non-trivial part for anyone implementing a custom OffersMessageHandler.

In addition, from the recurrence perspective, this adds another layer. When a subsequent invoice for an active recurrence session is received by LDK’s ChannelManager, it needs to rely on the LDK node’s outbound recurrence tracker to validate the invoice before proceeding with payment.

One mechanism that integrates cleanly with the current LDK design is to enable UserConfig::manually_handle_bolt12_invoices, allowing recurrence-related invoices to be forwarded to the LDK node for manual handling and validation.

[Alkamal01]

The two concrete changes needed to unblock the multi-wallet case without touching manually_handle_bolt12_invoices are:

1. Allow send_payment_for_bolt12_invoice to accept an externally-provided payment_id and skip verify_bolt12_invoice for callers that have already validated the invoice out-of-band.
2. Handle the Vacant case in send_payment_for_bolt12_invoice_internal (currently returns DuplicateInvoice) to allow initiating a payment without a pre-existing AwaitingInvoice entry.

Given @jkczyz and @shaavan points about recurrence and the private internals, dropping manually_handle_bolt12_invoices seems like a separate and larger effort. These two changes feel like the minimal path forward for this issue specifically.

[Afoxcute]

The two concrete changes needed to unblock the multi-wallet case without touching manually_handle_bolt12_invoices are:

1. Allow send_payment_for_bolt12_invoice to accept an externally-provided payment_id and skip verify_bolt12_invoice for callers that have already validated the invoice out-of-band.
2. Handle the Vacant case in send_payment_for_bolt12_invoice_internal (currently returns DuplicateInvoice) to allow initiating a payment without a pre-existing AwaitingInvoice entry.

Given @jkczyz and @shaavan points about recurrence and the private internals, dropping manually_handle_bolt12_invoices seems like a separate and larger effort. These two changes feel like the minimal path forward for this issue specifically.

Thanks for spelling out a minimal path. I agree it’s reasonable to decouple partial-MPP / routing work from the larger “drop manually_handle_bolt12_invoices” discussion Jeff and Shaavan raised.

A few details worth tightening before implementation:

Vacant vs error type: In send_payment_for_bolt12_invoice_internal, a Vacant payment_id currently returns UnexpectedInvoice, not DuplicateInvoice (DuplicateInvoice is more the “occupied but wrong PendingOutboundPayment variant” case). Minor naming fix only.

Vacant isn’t enough by itself: Today mark_invoice_received_and_get_details (and the flow into send_payment_for_bolt12_invoice_internal) assumes state created by the normal outbound flow—e.g. retry_strategy and route_params_config from AwaitingInvoice / InvoiceReceived. If we allow paying without that row, the API needs to say where those come from (explicit params vs defaults), not only “allow Vacant.”

Skip verify_bolt12_invoice: Understandable for multi-wallet / out-of-band validation, but it’s a sharp boundary (today verification ties the invoice to LDK’s payer material). Likely needs a very explicit entry point (separate method or options enum) + docs so callers don’t skip verify by accident.

Partial MPP: Still need the same split as BOLT11 #4373 : route final_value_msat (partial) vs RecipientOnionFields::total_mpp_amount_msat (declared full MPP total). Today BOLT12 sets total_mpp_amount_msat from route_params.final_value_msat, so that wiring is orthogonal to verify/skip.

[jkczyz]

@Afoxcute This looks like an AI summary. Do you have a specific question?

[Alkamal01]

Hey @TheBlueMatt , just checking back in, has there been any consensus on the approach after @jkczyz and @shaavan input? Happy to start once there's a direction.

[TheBlueMatt]

pushing StaticInvoiceRequested event into ChannelManager's event queue, which is private

Right, my point was that for the users using the flow directly they wouldn't go through the event pipeline, so there's no need for this?

calling ChannelManager::create_inbound_payment, which is public
calling ChannelManager::initiate_async_payment, which is private
calling ChannelManager::abandon_payment_with_reason, which is private
accessing anything called by ChannelManager::send_payment_for_verified_bolt12_invoice (i.e., pending_outbound_payments).

My point was that we should move to incorporating this stuff into send_payment_for_bolt12_invoice - today you cannot send a payment for a bolt12 invoice at all. That's fine in that people generally shouldn't be doing that for "normal" usecases, but for cases where they want to manually handle bolt12 invoice request messages because they're using ecash or whatever, we should absolutely support it. IOW instead of the ChannelManager going through the usual "bolt12 invoice requested -> bolt12 invoice received -> pay" flow it would skip to the "bolt12 invoice received" step right when it starts handling the payment.

However, building the pending_outbound_payment-like storage and tracking remains a non-trivial part for anyone implementing a custom OffersMessageHandler.

If the goal is to just track requested invoices and then hand them off to the ChannelManager I don't buy that this is crazy complicated.

In addition, from the recurrence perspective, this adds another layer. When a subsequent invoice for an active recurrence session is received by LDK’s ChannelManager, it needs to rely on the LDK node’s outbound recurrence tracker to validate the invoice before proceeding with payment.

Hmm? For recurrence IIUC the payer initiates the payment flow by requesting a fresh invoice. Thus at that point its "just a normal payment" from the PoV of lightning. I don't understand why this is any different.

One mechanism that integrates cleanly with the current LDK design is to enable UserConfig::manually_handle_bolt12_invoices, allowing recurrence-related invoices to be forwarded to the LDK node for manual handling and validation.

I still maintain that manually_handle_bolt12_invoices should be totally removed. I don't see a strong motivation for retaining it for ecash setups, especially given that we want to enable partial-payments for them.

[Alkamal01]

To make sure I'm scoping this right, when you say remove manually_handle_bolt12_invoices, does send_payment_for_bolt12_invoice need to handle invoices LDK didn't originate (i.e. skip/relax verify_bolt12_invoice)? That's the part I want to nail before touching anything.

[shaavan]

If the goal is to just track requested invoices and then hand them off to the ChannelManager I don't buy that this is crazy complicated.

Fair point. If someone can solve handling payment tracking, along with routing and signing, they should be able to implement a manual BOLT12 invoice handling setup on top.

---

Hmm? For recurrence IIUC the payer initiates the payment flow by requesting a fresh invoice. Thus at that point its "just a normal payment" from the PoV of lightning. I don't understand why this is any different.

That’s true at high level. However, the difference comes from the invoice_recurrence_basetime set by the payee, which the payer must ensure remains consistent across all invoices within a recurrence session.

Since we’re aiming to keep recurrence state tracking at the LDK node layer (see), it becomes tricky to intercept the invoice at the rust-lightning layer and then defer validation and payment to the node without relying on manually_handle_bolt12_invoices.

One possible approach is this: when requesting a fresh invoice from the LDK node, we already have access to the tracked recurrence data. If we encode the expected_invoice_recurrence_basetime into the blinded reply paths, LDK could validate the returned invoice directly, without needing to call back into the node for manual handling.

[TheBlueMatt]

That’s true at high level. However, the difference comes from the invoice_recurrence_basetime set by the payee, which the payer must ensure remains consistent across all invoices within a recurrence session.

That's a weird spec decision, why does the payee have to set that?

One possible approach is this: when requesting a fresh invoice from the LDK node, we already have access to the tracked recurrence data. If we encode the expected_invoice_recurrence_basetime into the blinded reply paths, LDK could validate the returned invoice directly, without needing to call back into the node for manual handling.

Yes, let's do that rather than using invoice request interception.

[Alkamal01]

That’s true at high level. However, the difference comes from the invoice_recurrence_basetime set by the payee, which the payer must ensure remains consistent across all invoices within a recurrence session.

That's a weird spec decision, why does the payee have to set that?

One possible approach is this: when requesting a fresh invoice from the LDK node, we already have access to the tracked recurrence data. If we encode the expected_invoice_recurrence_basetime into the blinded reply paths, LDK could validate the returned invoice directly, without needing to call back into the node for manual handling.

Yes, let's do that rather than using invoice request interception.

Thanks for the clarity. So the path forward is:

• Encode the recurrence expectations into the blinded reply paths
• LDK validates on invoice receipt internally
• No need for manually_handle_bolt12_invoices interception

Happy to start working on this.