feat: allow self payment

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

[shruti2522]

fix#2462

• implemented the following in outbound_payment.rs :

  • if the router tells us the payment is to ourselves, we build a simple one-hop route and treat it like a local payment
  • when preimage is available, immediately claim the HTLC

• added test case self_spontaneous_payment_fulfills() to test self payment

[ldk-reviews-bot]

👋 I see @TheBlueMatt was un-assigned.
If you'd like another reviewer assignment, please click here.

[TheBlueMatt]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

[codecov]

Codecov Report

❌ Patch coverage is 92.55319% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.54%. Comparing base (7785360) to head (e5498e0).

Files with missing lines	Patch %	Lines
lightning/src/ln/outbound_payment.rs	92.55%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #4328 +/- ##
==========================================
+ Coverage 86.53% 86.54% +0.01% 
==========================================
Files 158 158 Lines 103188 103280 +92 Branches 103188 103280 +92 ==========================================
+ Hits 89292 89385 +93 
Misses 11471 11471 + Partials 2425 2424 -1 

Flag	Coverage Δ
fuzzing	37.03% <12.76%> (-0.05%)	⬇️
tests	85.84% <92.55%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[shruti2522]

This violates a number of the API constraints on sending payments - specifically, someone could send a payment, then restart and they wouldn't see the pending payment, thinking they hadn't "sent" it. Check out some of the early feedback on the first attempt at this #2573

I reworked this based on feedback in #2573 . We now try routing first and only fall back to a self-payment check if it fails, so explicit self-routes still work for rebalancing. I have also added phantom node support. this implementation reuses existing code without adding new ChannelManager fields. PaymentClaimable is still left since it’s a bigger change, should i implement it here?
please take a look @TheBlueMatt

[TheBlueMatt]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

[shruti2522]

Yea, sadly I really don't think this is all that easy of a task - in order to provide the same semantics we provide for regular payments to self-payments (which imo is a strong requirement to land this) we need to have a disk write involved and keep the pending payment around like we do normal payments. Its not really entirely clear to me how we should do that - for a normal payment the way we do disk writes is via ChannelMonitorUpdates to the channel the payment is going out through. Of course for self-payments we don't intended to use any channel, so there's not a clear connected channel to use. We could of course pick some random channel to use, which is fine and maybe the right answer, but it does seem a bit awkward (I don't see a better answer tho tbh). Once we do that disk write we have to keep the payment around for a while until the events for the payment are processed and then remove it with another monitor update. On startup if we detect a self-payment that wasn't resolved we'll have to regenerate the events and replay them.

thanks for the explanation, I'm willing to implement the channel-based approach you explained, i just wanted to confirm should self payments only work when the node has at least one channel or should we handle the zero-channel case differently?

[TheBlueMatt]

Yea, I think its fine to just only allow self-payments if we have at least one channel. Devs shouldn't expect payments to work if they have no channels anyway.

[ldk-reviews-bot]

Hi @shruti2522,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4328

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:shruti2522/feat/self-payment. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.