Handle network updates from failed payments in BackgroundProcessor

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.

[jkczyz]

MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) into Event::PaymentFailed to be processed by an event handler.

This requires making NetGraphMsgHandler an EventHandler, which is used in BackgroundProcessor to decorate the user-provided EventHandler with the ability to update the NetworkGraph accordingly.

The general concept of intercepting events via a decorator will be used in a follow-up PR for scoring channels and retrying failed payments.

[codecov]

Codecov Report

Merging #1043 (992df51) into main (423f1b1) will increase coverage by 0.21%.
The diff coverage is 91.04%.

@@ Coverage Diff @@## main #1043 +/- ##
==========================================
+ Coverage 90.83% 91.04% +0.21% 
==========================================
Files 65 65 Lines 33281 33762 +481 ==========================================
+ Hits 30232 30740 +508 + Misses 3049 3022 -27 

Impacted Files	Coverage Δ
lightning-net-tokio/src/lib.rs	77.27% <ø> (+0.29%)	⬆️
lightning/src/ln/msgs.rs	89.50% <ø> (+0.39%)	⬆️
lightning/src/ln/peer_handler.rs	45.94% <ø> (+0.16%)	⬆️
lightning/src/util/test_utils.rs	82.55% <ø> (+0.21%)	⬆️
lightning/src/util/events.rs	27.54% <40.00%> (+13.60%)	⬆️
lightning/src/ln/onion_utils.rs	94.91% <75.00%> (ø)
lightning/src/ln/channelmanager.rs	85.90% <84.61%> (+0.05%)	⬆️
lightning-background-processor/src/lib.rs	94.20% <85.71%> (+0.26%)	⬆️
lightning/src/routing/network_graph.rs	92.12% <87.13%> (+0.31%)	⬆️
lightning/src/routing/router.rs	96.25% <92.00%> (-0.10%)	⬇️
... and 15 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 423f1b1...992df51. Read the comment docs.

[TheBlueMatt]

Concept ACK. Only did a high-level skim but I like it.

[TheBlueMatt]

Hmmmmmm, have we considered maybe just allowing events to be handled by multiple handlers? Passing them off to an inner handler works, I suppose, but its kinda awkward. I guess I'm not a huge fan of passing the events by reference instead of value, but most of them have no heap allocation, and eg SpendableOutputs should be pretty rare (requires on-chain txn).

[jkczyz]

The decorator approach here is nice because it allows us to short-circuit handling. This is needed for payment retries where the inner handler is only invoked after the final failure.

[TheBlueMatt]

Hmm, yea, good point. This isn't a trait change, though, so we could do different things for different handlers, right?

[jkczyz]

Not quite sure I follow what you mean by do "different things for different handlers". Do you have a concrete example that may clarify?

[TheBlueMatt]

As in, there's nothing here that requires that we handle ignoring/passing down events the same for NetworkGraph as we do for PaymentRetryer, or whatever, is there?

[jkczyz]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

(see also other comment about removing NetworkGraph from BackgroundProcessor)

We'd have to parameterize NetworkGraph by a Logger if we did that. 😕 How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

Using references allows a bit more flexibility like post-handling event as mentioned above.

[TheBlueMatt]

We'd have to parameterize NetworkGraph by a Logger if we did that.

Eh, we basically already do that for everything, not a big deal IMO.

How about implementing EventHandler for NetGraphMsgHandler? I've been toying around with renaming that NetworkGossip, making the naming less specific about how and that it updates NetworkGraph.

No strong feelings, do what you want for both.

[jkczyz]

Haven't pushed it yet, but I went with implementing it on NetGraphMsgHandler. It's also nice to not have to know about a decorated EventHandler when deserializing a NetworkGraph.

That said, just want to point out a drawback with either is that the guide for setting up a node will need to change. Now event handling (step 16) can no longer be discussed independently of initializing NetGraphMsgHandler (optional step 12).

cc: @valentinewallace Any opinions on this?

[valentinewallace]

Thanks for pointing that out. That is a slight drawback but not a deal breaker IMO

[TheBlueMatt]

I think that's ok. Indeed, ideally we'd have everything be nice and separate, but if we're gonna have something related to the network graph handle events, that's the outcome we're gonna have. It is nice that all the bindings have moved towards using the lightning-background-processor crate, though, so it'll all be common infrastructure in any case.

[valentinewallace]

General call for more docs on the usage of this new API, but this is looking good to me!

[valentinewallace]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414
I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

[TheBlueMatt]

Implemented in the sample: valentinewallace/ldk-sample@ddc9414 I think it looks good, though it does add a clone to each Event and a few more clones of NetworkGraph.

Note that you don't actually need the clone - https://git.bitcoin.ninja/index.cgi?p=ldk-sample;a=commit;h=bc119d8fb23d8e55c91df002ca490062f9cc72a6

[jkczyz]

Updated the docs!

There's a couple open issues with testing:

• test_invalid_channel_announcement seems non-sensical, so maybe just drop it as discussed on Slack
• htlc_fail_async_shutdown fails when performing checks in expect_payment_failed_with_update

For the latter, I commented out the assertion in expect_payment_failed_with_update, which I won't want to merge. So need to figure a workaround. Perhaps just not use the macro. 😕

[TheBlueMatt]

I'd argue the issue here is on the channel end - we fail with chan_disabled instead of permanent_channel_failure. I think we should swap the shutdown checks in Channel::update_add_htlc to instead use 0x4000|8 and then fix channelmanager to handle it (I haven't reviewed this, just hacked it up, so may need more thought):

@@ -3461,33 +3464,32 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
}
let create_pending_htlc_status = |chan: &Channel<Signer>, pending_forward_info: PendingHTLCStatus, error_code: u16| {
- // Ensure error_code has the UPDATE flag set, since by default we send a
- // channel update along as part of failing the HTLC.
- assert!((error_code & 0x1000) != 0);
// If the update_add is completely bogus, the call will Err and we will close,
// but if we've sent a shutdown and they haven't acknowledged it yet, we just
// want to reject the new HTLC and fail it backwards instead of forwarding.
match pending_forward_info {
PendingHTLCStatus::Forward(PendingHTLCInfo { ref incoming_shared_secret, .. }) => {
- let reason = if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
- let mut res = Vec::with_capacity(8 + 128);
- // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
- res.extend_from_slice(&byte_utils::be16_to_array(0));
- res.extend_from_slice(&upd.encode_with_len()[..]);
- res
- }[..])
+ let reason = if (error_code & 0x1000) != 0 {
+ if let Ok(upd) = self.get_channel_update_for_unicast(chan) {
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &{
+ let mut res = Vec::with_capacity(8 + 128);
+ // TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
+ res.extend_from_slice(&byte_utils::be16_to_array(0));
+ res.extend_from_slice(&upd.encode_with_len()[..]);
+ res
+ }[..])
+ } else {
+ // The only case where we'd be unable to successfully get a
+ // channel update is if the channel isn't in the fully-funded
+ // state yet, implying our counterparty is trying to route
+ // payments over the channel back to themselves (cause no one
+ // else should know the short_id is a lightning channel yet).
+ // We should have no problem just calling this
+ // unknown_next_peer (0x4000|10).
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ }
} else {
- // The only case where we'd be unable to
- // successfully get a channel update is if the
- // channel isn't in the fully-funded state yet,
- // implying our counterparty is trying to route
- // payments over the channel back to themselves
- // (cause no one else should know the short_id
- // is a lightning channel yet). We should have
- // no problem just calling this
- // unknown_next_peer (0x4000|10).
- onion_utils::build_first_hop_failure_packet(incoming_shared_secret, 0x4000|10, &[])
+ onion_utils::build_first_hop_failure_packet(incoming_shared_secret, error_code, &[])
};
let msg = msgs::UpdateFailHTLC {
channel_id: msg.channel_id,

[jkczyz]

Seems to work though I had to change which channel is used in the failing test otherwise the short channel id wouldn't match. Take a look at d5ea8e8. I'll properly backport the fixup if all looks well.

[TheBlueMatt]

Needs rebase.

[TheBlueMatt]

One material comment, but once we get resolution on that I'm ACK.

[TheBlueMatt]

What does # do?

[jkczyz]

Hides it from showing up in the documentation. Useful if you have an elaborate setup that may distract from the example. Could leave some of these lines in if you'd like. Need to fix the indentation anyhow.

[TheBlueMatt]

Do you also want to update to just implement EventHandler on the NetworkGraph? Given the background processor already explicitly takes a NetworkGraph, it wouldn't complicate the background processor much at all (and maybe simplify it if we make the network graph there optional).

If we're gonna go down the path of decorators eating and passing events onto inner handlers, I dunno if we need to bother passing events by reference.

[TheBlueMatt]

network_graph needs to be Optional.

[jkczyz]

I think we may actually want to remove this entirely from BackgroundProcessor. With #1059, users will want to decorate InvoicePayer with NetworkUpdateHandler so that updates are applied prior to retrying. Given they'll want to use the InvoicePayer to make payments, we can't construct that here without returning it back to them somehow.

So instead we may want to leave it to the user to set up the event handler to pass to BackgroundProcessor. But we could perhaps provide a utility for decorating their event handler with standard decorators.

[TheBlueMatt]

Yea, I guess that makes sense - you only get those errors if you send a payment and it fails, so requiring a payment retryer in order to handle those errors isn't crazy.

[jkczyz]

I removed the changes to BackgroundProcessor.

[jkczyz]

I removed the commit that made NetworkGraph have shared ownership since it is no longer necessary now that NetGraphMsgHandler is responsible for event handling.

[valentinewallace]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

[valentinewallace]

should there be a TODO to handle the error?

[jkczyz]

Not sure. The code that produces them is reused in other places but likely isn't actionable here. Might be able to log one marked IgnoreAndLog.

[TheBlueMatt]

Needs rebase.

[jkczyz]

I think the docs could be a little more explicit that you're expected to nest EventHandlers for BackgroundProcessor::start now but this is looking slick overall!

Added a short note about decorators in start. Otherwise, we'll likely want some better documentation which could go on a utility used to wrap an event handler with common ones.

[TheBlueMatt]

Moving this (and related stuff) to 0.0.102 because it looks like we may want to ship an 0.0.101 to get some fixes out.

[jkczyz]

Based on an offline conversation with @TheBlueMatt, I removed the decorated EventHandler from NetGraphMsgHandler. Instead, BackgroundProcessor now takes an optional NetGraphMsgHandler which it delegates to before letting the user's event handler run.

This approach is a bit cleaner since it simplifies NetGraphMsgHandler and makes it such that BackgroundProcessor does the right thing for the user.

[TheBlueMatt]

ACK aside from the two comments above.

[valentinewallace]

Nice, LGTM

[valentinewallace]

Hmm, so I checked lightningdevkit.org guides, the sample, process_events docs and I don't think we indicate anywhere that the user should be calling this function perioically (i.e., users do currently rely on BackgroundProcessor to call process_events atm).

Should we update some other docs/the sample, or open an issue to do so?

[TheBlueMatt]

Its primarily called by lightning-net-tokio or by the NioPeerHandler in Java, so I don't think users need to care, at least not normal users.

[TheBlueMatt]

I did open an issue at #1075.