Handle Payment Metadata in the Invoice/Send/Receive Pipelines by TheBlueMatt · Pull Request #1221 · lightningdevkit/rust-lightning

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt · 2021-12-22T06:29:45Z

As specified by lightning/bolts#912. All pretty straightforward, just a lot of plumbing.

TheBlueMatt · 2022-01-26T19:05:11Z

Rebased.

There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

We only use it to check the amount when processing MPP parts, but store the full object (including new payment metadata) in it. Because we now store the amount in the parent structure, there is no need for it at all in the `OnionPayload`. Sadly, for serialization compatibility, we need it to continue to exist, at least temporarily, but we can avoid populating the new fields in that case.

...without exposing it to the public `send_payment` API yet.

TheBlueMatt · 2022-01-26T19:09:21Z

Oops, actually rebased now.

TheBlueMatt · 2022-02-07T17:39:48Z

Actually, I think I want to redo this to pull the metadata, payment secret, and also #1298 data into a struct so we can solve #1298 here as well.

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs (see [discussion](lightningdevkit#1298 (comment)))

This work is a pre-req for [Issue-1298](lightningdevkit#1298). This is a rebase of [PR-1221](lightningdevkit#1221) with an additional commit to move the payment_secret and the payment_metadata into a new struct called RecipientInfo. This will allow us to add custom onion TLVs, see [discussion](lightningdevkit#1298 (comment)) No logic change, just lots of plumbing.

TheBlueMatt · 2022-04-22T15:49:04Z

Superceded by #1445.

TheBlueMatt force-pushed the 2021-12-payment-metadata branch from 12d90c2 to 49055f8Compare December 22, 2021 06:33

TheBlueMatt mentioned this pull request Dec 23, 2021
Support MPP KeySend #1222
Closed

TheBlueMatt force-pushed the 2021-12-payment-metadata branch from 49055f8 to 01f20d7Compare December 30, 2021 18:46

TheBlueMatt added the Seeking Code Review label Dec 30, 2021

jkczyz self-requested a review January 5, 2022 19:28

TheBlueMatt force-pushed the 2021-12-payment-metadata branch from 01f20d7 to 6d898f6Compare January 26, 2022 19:05

TheBlueMatt added 11 commits January 26, 2022 19:08

Define the PaymentMetadata feature to be used in invoices
1c1fc45

Unset the optional bit for a feature when setting the required bit
cb82305
There is no reason to set both, and this currently makes testing the new BOLT invoice tests slightly harder, so we just unset it.

Support reading the new payment_metadata field in invoices
c8e2635

Support setting the new payment metadata field in invoices
2cfa80b

Pass FinalOnionHopData to payment verify by reference, not clone
24c6792

Store total payment amount in ClaimableHTLC explicitly
7c16299
...instead of accessing it via the `OnionPayload::Invoice` form. This may be useful if we add MPP keysend support, but is directly useful to allow us to drop `FinalOnionHopData` from `OnionPayload`.

Deserialize payment metadata fields in the onion final hop data
607cd2e

Provide users the payment metadata via Event::PaymentReceived
c98733b

Pipe payment metadata through the HTLC send pipeline
588337f
...without exposing it to the public `send_payment` API yet.

Expose payment metadata in the sending path via send_payment.
cd7e5ac

TheBlueMatt force-pushed the 2021-12-payment-metadata branch from 6d898f6 to cd7e5acCompare January 26, 2022 19:09

TheBlueMatt removed the Seeking Code Review label Feb 7, 2022

TheBlueMatt self-assigned this Feb 7, 2022

joostjager mentioned this pull request Mar 24, 2022
routing: send payment metadata lightningnetwork/lnd#5810
Merged

TheBlueMatt mentioned this pull request Apr 10, 2022
Custom HTLC TLVs #1298
Closed

andozw mentioned this pull request Apr 22, 2022
Introduce RecipientInfo struct for send_payment #1445
Closed

TheBlueMatt closed this Apr 22, 2022

This was referenced Apr 22, 2022
[sean] Avoid storing a full FinalOnionHopData in OnionPayload::Invoice andozw/rust-lightning#3
Merged
Avoid storing a full FinalOnionHopData in OnionPayload::Invoice #1447
Merged