[RFC] Implement a way to do BOLT 12 Proof of Payment by vincenzopalazzo · Pull Request #3593 · lightningdevkit/rust-lightning

vincenzopalazzo · 2025-02-07T23:14:43Z

This is already possible if we manually_handle_bolt12_invoices, but
this approach requires a significant amount of work from the user.

This commit adds the bolt12 invoice to the PaymentSend event when
the payment is completed. This always gives the user the option to
provide proof of payment.

I am opening this up for an initial review, but I need to look inside the DNSSECProof to add this proof, but
probably be also a follow-up PR

~~Opening in draft for now to make the CI happy~~

Fixes: #3344

codecov · 2025-02-10T20:28:47Z

Codecov Report

Attention: Patch coverage is 89.42308% with 11 lines in your changes missing coverage. Please review.

Project coverage is 89.30%. Comparing base (1fc2726) to head (0cd70fa).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/offers/static_invoice.rs	0.00%	8 Missing ⚠️
lightning/src/ln/offers_tests.rs	92.85%	1 Missing ⚠️
lightning/src/ln/outbound_payment.rs	96.55%	1 Missing ⚠️
lightning/src/util/ser_macros.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3593 +/- ##
==========================================
+ Coverage 89.25% 89.30% +0.05% 
==========================================
Files 155 156 +1 Lines 119708 121825 +2117 Branches 119708 121825 +2117 ==========================================
+ Hits 106845 108796 +1951 - Misses 10250 10388 +138 - Partials 2613 2641 +28

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Thank you for picking this up! Sorry about the delay in review, not sure why this slipped through the cracks.

vincenzopalazzo · 2025-02-16T13:06:50Z

Thanks for the review @TheBlueMatt, I applied the suggestion in the review. In the meanwhile, I rebased on top of main to fix the conflict and ran the CI again

vincenzopalazzo · 2025-03-05T19:40:43Z

Ok @TheBlueMatt this PR is really funny to work! I worked to clean up the PR a bit and divide the code to support both Bolt12 and static invoices. However, I am currently working on supporting the async payment functionality. As far as I understand, the code involving async payments is a bit tricky to support for the following reasons (code available 85c3f73).

The async workflow is initialized with initiate_async_payment, where we pass down the payment_id and the static invoice value. Then we call static_invoice_received, which generates the event PendingOutboundPayment::StaticInvoiceReceived. After that, we iterate over the entry in send_payment_for_static_invoice and follow the Bolt12 pay code path (at this point, we need to pass down the static invoice).

The only way I have to access the static invoice when passing it down inside PendingOutboundPayment::StaticInvoiceReceived breaks the storage code because the static invoice is not available without cfg(async_payment).

Is there any way to use the persistence macros (impl_writeable_tlv_based_enum_upgradable) with a cfg?
If we want to read and pass the static invoice in PendingOutboundPayment::StaticInvoiceReceived, we could store a serialized version of the static invoice (a sequence of bytes), but this seems rather hacky.
I still need to delve into the mechanics of async payment as implemented in LDK, but if I understand correctly, there is a refactoring that I can do to make access to the static invoice a bit easier?

I would prefer the 3rd point, but this requires a follow-up PR to rework a little bit how the async payment is handled and I do not know if there is any plan for that,t or there is other code that will be depending on this architecture in the pipeline.

Probably @valentinewallace will be interested in this conversation

TheBlueMatt · 2025-03-10T14:28:57Z

Is there any way to use the persistence macros (impl_writeable_tlv_based_enum_upgradable) with a cfg?

Generally we just copy and paste the impl with different cfg flags and go back and clean it up after we drop the cfg.

I'll let @valentinewallace comment on the rest.

valentinewallace · 2025-03-10T16:00:42Z

Ok @TheBlueMatt this PR is really funny to work! I worked to clean up the PR a bit and divide the code to support both Bolt12 and static invoices. However, I am currently working on supporting the async payment functionality. As far as I understand, the code involving async payments is a bit tricky to support for the following reasons (code available 85c3f73).
The async workflow is initialized with initiate_async_payment, where we pass down the payment_id and the static invoice value. Then we call static_invoice_received, which generates the event PendingOutboundPayment::StaticInvoiceReceived. After that, we iterate over the entry in send_payment_for_static_invoice and follow the Bolt12 pay code path (at this point, we need to pass down the static invoice).
The only way I have to access the static invoice when passing it down inside PendingOutboundPayment::StaticInvoiceReceived breaks the storage code because the static invoice is not available without cfg(async_payment).
* Is there any way to use the persistence macros (`impl_writeable_tlv_based_enum_upgradable`) with a cfg?
* If we want to read and pass the static invoice in `PendingOutboundPayment::StaticInvoiceReceived`, we could store a serialized version of the static invoice (a sequence of bytes), but this seems rather hacky.
* I still need to delve into the mechanics of async payment as implemented in LDK, but if I understand correctly, there is a refactoring that I can do to make access to the static invoice a bit easier?
I would prefer the 3rd point, but this requires a follow-up PR to rework a little bit how the async payment is handled and I do not know if there is any plan for that,t or there is other code that will be depending on this architecture in the pipeline.
Probably @valentinewallace will be interested in this conversation

Hey sorry I missed this tag. In #3618 right now we actually un-cfg-gate StaticInvoices. Would that solve this? I could try to get conceptual ACKs on that to unblock you if so.

vincenzopalazzo · 2025-03-11T20:52:22Z

Generally we just copy and paste the impl with different cfg flags and go back and clean it up after we drop the cfg.

Yes, but in this case, we need to duplicate all the impl_writeable_tlv_based_enum_upgradable code which sounds a really bad thing, I think I will go to steal the commit 890ce7d and see how the code will looks like :) but I think StaticInvoice out of the cfg is all what I needs

vincenzopalazzo · 2025-04-01T13:01:42Z

Ok at my desk trying to finish this PR ASAP.

I had to rebase the PR to fix the conflicts to be able to run the CI! now waiting for the latter to see if I am forgetting something else to fix.

Thanks for your time in review this PR @valentinewallace

Probably the only thing left to fix Is the lining with the new change at the docs https://github.com/lightningdevkit/rust-lightning/actions/runs/14196272391/job/39772046844?pr=3593

TheBlueMatt

I think you can go ahead and squash and we can probably do final passes?

We need to include static invoices in the public API as part of the onion messages we're adding for static invoice server support. Utilities to create these static invoices and other parts of the async receive API will remain cfg-gated for now. Generally, we can't totally avoid exposing half baked async receive support in the public API because OnionMessenger is parameterized by an async payments message handler, which can't be cfg-gated easily. Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

vincenzopalazzo · 2025-04-05T10:38:38Z

Ok should be ready for another round of review! However, I am not sure why the linter is failing now, looks like unrelated to this PR?

TheBlueMatt

LGTM one nit

valentinewallace

LGTM after Matt's last comment

valentinewallace · 2025-04-07T20:41:11Z

 let length: $crate::util::ser::BigSize = $crate::util::ser::Readable::read(reader)?;
 let mut s = $crate::util::ser::FixedLengthReader::new(reader, length.0);
- let res = $crate::util::ser::Readable::read(&mut s)?;
+ let res = $crate::util::ser::LengthReadable::read_from_fixed_length_buffer(&mut s)?;


It actually still says that this is a temporary workaround, but not blocking

@valentinewallace

When introducing the LengthLimitedRead trait in [1], for some serialization macros we need this change to avoid that we have the following compilation error error[E0277]: the trait bound `Bolt12Invoice: Readable` is not satisfied --> lightning/src/util/ser_macros.rs:1243:35 | 1243 | Ok($st::$tuple_variant_name($crate::util::ser::Readable::read(reader)?)) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Readable` is not implemented for `Bolt12Invoice` | ::: lightning/src/events/mod.rs:2463:1 | 2463 | / impl_writeable_tlv_based_enum_legacy!(PaidInvoice, 2464 | | ; 2465 | | (0, Bolt12Invoice), 2466 | | (2, StaticInvoice) 2467 | | ); | |_- in this macro invocation | = help: the following other types implement trait `Readable`: () (A, B) (A, B, C) (A, B, C, D) (A, B, C, D, E) (A, B, C, D, E, F) (A, B, C, D, E, F, G) AnnouncementSigsState and 200 others = note: this error originates in the macro `impl_writeable_tlv_based_enum_legacy` (in Nightly builds, run with -Z macro-backtrace for more info) This changes are needed since [1] is landed on mainline. [1] lightningdevkit#3640 Co-Developed-by: @valentinewallace Suggested-by: @valentinewallace Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

This commit make two things possible: 1. make persistent BOLT12 invoice through PendingOutboundPayment This commit prepares the code to pass down the BOLT12 invoice inside the `PaymentSent` event. To achieve this, the `bolt12` field has been added to the `PendingOutboundPayment::Retryable` enum, allowing it to be attached to the `PaymentSent` event when the payment is completed. 2. To enable proof of payment, we need to share the bolt12 invoice with the library user. This is already possible if we `manually_handle_bolt12_invoices`, but this approach requires a significant amount of work from the user. This commit adds the bolt12 invoice to the PaymentSend event when the payment is completed. This allows the user to always have the option to perform proof of payment. We also Inject the static invoice inside the PendingOutboundPayment::StaticInvoiceReceived to allow to use the static invoice inside the PendingOutboundPayment::PaymentReceived. Link: lightningdevkit#3344 Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

vincenzopalazzo · 2025-04-07T21:07:08Z

It actually still says that this is a temporary workaround, but not blocking

Sorry I forgot to read also the upper part of the commit body, luckily git rebase -i is free :)

TheBlueMatt

One issue, but I'm gonna go ahead and land this as fixing the issue can be separate.

TheBlueMatt · 2025-04-08T00:28:40Z

 payment_metadata: None, // only used for retries, and we'll never retry on startup
 keysend_preimage: None, // only used for retries, and we'll never retry on startup
 invoice_request: None, // only used for retries, and we'll never retry on startup
+ bolt12_invoice: None, // only used for retries, and we'll never retry on startup!


Oof. Sorry I missed this until now. This is not, in fact, "only used for retries", we use it on claims only, in fact. If a user is relying on the event field for PoP, what this can mean is that we can initiate a send, restart with a stale ChannelManager, notice the payment is pending, then when it claims fail to provide the invoice (only the preimage) to the payer.
In practice, to fix this, we'll need to include the PaidBolt12Invoice in the HTLCSource::OutboundRoute, I believe.

vincenzopalazzo marked this pull request as draft February 7, 2025 23:18

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from c7e9e61 to cc714aeCompare February 7, 2025 23:33

vincenzopalazzo commented Feb 7, 2025
View reviewed changes

Comment threadlightning/src/events/mod.rs

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from a5c211c to 9cdbb44Compare February 10, 2025 16:59

vincenzopalazzo marked this pull request as ready for review February 10, 2025 16:59

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9cdbb44 to 8f5dc5eCompare February 10, 2025 20:19

TheBlueMatt reviewed Feb 13, 2025
View reviewed changes

Comment threadlightning/src/ln/outbound_payment.rs Outdated
Comment threadlightning/src/ln/outbound_payment.rs
Comment threadlightning/src/ln/outbound_payment.rs Outdated
Comment threadlightning/src/events/mod.rs Outdated

vincenzopalazzo mentioned this pull request Feb 16, 2025
BOLT 12 Proof of Payment #3344
Closed

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 7da68f6 to 0e7b503Compare February 16, 2025 13:05

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 0e7b503 to a4897fbCompare February 16, 2025 13:11

vincenzopalazzo requested a review from TheBlueMatt February 16, 2025 13:12

TheBlueMatt reviewed Feb 18, 2025
View reviewed changes

Comment threadlightning/src/ln/outbound_payment.rs Outdated

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from a4897fb to 2c65d6aCompare March 3, 2025 19:15

TheBlueMatt reviewed Mar 5, 2025
View reviewed changes

Comment threadlightning/src/offers/mod.rs Outdated
Comment threadlightning/src/offers/mod.rs Outdated
Comment threadlightning/src/ln/outbound_payment.rs Outdated

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9b72246 to 2c65d6aCompare March 5, 2025 16:31

vincenzopalazzo mentioned this pull request Mar 7, 2025
Multiple InvoiceReceived events for single pay_for_offer call #3653
Closed

TheBlueMatt requested a review from valentinewallace March 10, 2025 14:27

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 7 times, most recently from 734c955 to 525c01eCompare March 12, 2025 15:01

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 3 times, most recently from d69787b to 3929d07Compare April 1, 2025 12:59

vincenzopalazzo requested a review from valentinewallace April 2, 2025 17:41

TheBlueMatt reviewed Apr 4, 2025
View reviewed changes

Comment threadlightning/src/events/mod.rs Outdated
Comment threadlightning/src/events/mod.rs Outdated
Comment threadlightning/src/events/mod.rs Outdated
Comment threadlightning/src/ln/functional_test_utils.rs Outdated
Comment threadlightning/src/offers/invoice_macros.rs Outdated

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 2 times, most recently from 7450529 to 9ed2e06Compare April 4, 2025 16:47

vincenzopalazzo requested a review from TheBlueMatt April 4, 2025 16:47

TheBlueMatt reviewed Apr 4, 2025
View reviewed changes

Comment threadlightning/src/events/mod.rs Outdated

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9ed2e06 to 22f8c50Compare April 5, 2025 10:19

vincenzopalazzo requested a review from TheBlueMatt April 5, 2025 10:35

TheBlueMatt reviewed Apr 7, 2025
View reviewed changes

Comment threadlightning/src/ln/outbound_payment.rs Outdated

valentinewallace reviewed Apr 7, 2025
View reviewed changes

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 2 times, most recently from 2abe14a to 44bb850Compare April 7, 2025 19:09

vincenzopalazzo requested review from TheBlueMatt and valentinewallace April 7, 2025 19:09

valentinewallace approved these changes Apr 7, 2025
View reviewed changes

vincenzopalazzo added 2 commits April 7, 2025 23:05

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 44bb850 to 57772d3Compare April 7, 2025 21:06

TheBlueMatt approved these changes Apr 8, 2025
View reviewed changes

TheBlueMatt mentioned this pull request Apr 8, 2025
Fix (or doc, for 0.2) missing Event::PaymentSent::bolt12_invoice on payments loaded from monitors #3714
Closed

TheBlueMatt merged commit 077df33 into lightningdevkit:mainApr 8, 2025

vincenzopalazzo deleted the macros/proof-of-payment-bolt12 branch April 8, 2025 15:30

vincenzopalazzo mentioned this pull request Jun 5, 2025
storage: store the bolt12invoice invoice lightningdevkit/ldk-node#563
Closed

Uh oh!

Conversation

vincenzopalazzo commented Feb 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

codecovBot commented Feb 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vincenzopalazzo commented Feb 16, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vincenzopalazzo commented Mar 5, 2025

Uh oh!

TheBlueMatt commented Mar 10, 2025

Uh oh!

valentinewallace commented Mar 10, 2025

Uh oh!

vincenzopalazzo commented Mar 11, 2025

Uh oh!

vincenzopalazzo commented Apr 1, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vincenzopalazzo commented Apr 5, 2025

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

valentinewallace left a comment

Choose a reason for hiding this comment

Uh oh!

valentinewallaceApr 7, 2025

Choose a reason for hiding this comment

Uh oh!

vincenzopalazzo commented Apr 7, 2025

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

TheBlueMattApr 8, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vincenzopalazzo commented Feb 7, 2025•
edited
Loading

codecovBot commented Feb 10, 2025•
edited
Loading

vincenzopalazzo commented Feb 16, 2025•
edited
Loading

vincenzopalazzo commented Apr 1, 2025•
edited
Loading