Implement the SCIDAlias Channel Type and provide SCID Privacy

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).

[TheBlueMatt]

This is based on #1311 and implements the SCIDAlias channel type as well as ensures we don't leak "real" SCIDs via HTLC failures (as pointed out by Val at #1311 (comment))

Codecov Report

Merging #1351 (4d9a5c2) into main (ca163c3) will increase coverage by 0.82%.
The diff coverage is 97.79%.

❗ Current head 4d9a5c2 differs from pull request most recent head 952cee4. Consider uploading reports for the commit 952cee4 to get more accurate results

@@ Coverage Diff @@## main #1351 +/- ##
==========================================
+ Coverage 90.65% 91.47% +0.82% 
==========================================
Files 73 73 Lines 40462 48497 +8035 Branches 0 48497 +48497 ==========================================
+ Hits 36682 44365 +7683 - Misses 3780 4132 +352 

Impacted Files	Coverage Δ
lightning/src/ln/onion_route_tests.rs	97.62% <ø> (ø)
lightning/src/util/config.rs	45.83% <0.00%> (-0.98%)	⬇️
lightning/src/util/ser.rs	91.46% <ø> (ø)
lightning/src/ln/channel.rs	90.39% <96.15%> (+1.18%)	⬆️
lightning/src/ln/channelmanager.rs	87.90% <97.00%> (+3.12%)	⬆️
lightning/src/ln/priv_short_conf_tests.rs	97.83% <98.88%> (+0.99%)	⬆️
lightning/src/ln/features.rs	99.44% <100.00%> (+1.13%)	⬆️
lightning/src/ln/functional_test_utils.rs	95.54% <100.00%> (ø)
lightning/src/routing/router.rs	93.21% <100.00%> (+0.81%)	⬆️
lightning/src/util/events.rs	31.14% <100.00%> (-2.31%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca163c3...952cee4. Read the comment docs.

Note first commit is #1331, which we still depend on here.

First, I would say it's more a ChannelConfig flag as it's related to channel metada and not HTLC flow or channel format itself ? Though reading the types of configs comments I'm not sure if we have that strong a reasoning to sort config flags.

Second, do you think we should introduce some config sanitization to avoid invalid flags to be set up and thus detect buggy config ? In that present case, we would disallow negotiate_scid_alias if announced_channel is true, I think ?

edit: I see you do such sanitization in get_initial_channel_type maybe you could log a warning message for the incompatible combination ?

I believe we'd like to move towards ChannelConfig being only update-able fields (ie fees and such), see-also #1270. As for sanitation, yea, we probably should, historically we've just ignored nonsense settings, which we should document/do here as well.

Just to understand better, let's say the HTLC routing topology is Alice -> Bob -> Caroll, here we're returning a error date for the failure on Bob to Caroll as reported by Bob ? If so, it sounds counter-intuitive to use latest_inbound_scid_alias as it's discovered by Bob from Caroll's funding_locked iiuc and never announced by Alice by a routing hint ?

No, the channel here would be the channel from Alice to Bob, note the comment in the docs:

	/// This is for failures on the channel on which the HTLC was *received*, not failures
/// forwarding

Overall looks pretty good. Need to take a more thorough look at the tests and when failing HTLCs.

s/which/that

Rule of thumb: Use "that" if what follows restricts the preceding phrase and hence removing the clause would change its meaning. Use "which" if the clause can be removed without changing the meaning.

s/[ChannelManager]'s read method/[ChannelManager::read]

Sadly the link is broken if I do that since its through a trait - I can do it without [], or I can leave it as is, which do you prefer?

Fine to keep it as is.

Hmmm... this check may be more suitable at the call site. That way it is clear the error message is in response to an open_channel message. Though, I suppose you wouldn't want this called when in a different state. Maybe it would simpler to inline the method in handle_error?

Hmmmm, I really prefer to keep as much channel state machine logic in channel.rs as possible. In general we've failed at this increasingly, but if I ever find time I'm gonna try to push a chunk back down. channelmanager should just be for inter-channel stuff, never have any real knowledge of channel's state machine transitions, though it has to sometimes ask about the current state.

Ah, I see - good point! I guess it seems that ChannelManager::handle_event has state machine logic, though, because it assumes the error message is a response to open_channel. Would it make sense to add a handle_error method to Channel, which ChannelManager could delegate to? That would in turn have this logic to determine whether advance_channel_type_pref should be called or something else in the future depending on the channel state. I suppose it would need to return an event for ChannelManager to enqueue instead of assuming it is SendOpenChannel.

because it assumes the error message is a response to open_channel.

Heh, that's why i put the state check in channel.rs that way channelmanager doesn't really know what the error is in response to, it just tries and lets channel figure it out. I'm gonna rename advance_channel_type_pref to maybe_handle_error_without_close to make it "feel" more generic.

Unless you feel strongly I'm not gonna bother changing its return type, though, currently all of the channel/channelmanager bounday has chanellmanger "know" what type of message is being sent, cause its enforced by the type-checker. I agree in the future we should move some of those to events, but that's also a larger refactor.

Is this subject to change? Seems you thought 50 was the actual desired feature bit: https://github.com/lightning/bolts/pull/910/files#r807405646

Oh, yea, somewhat unclear, I guess we have to wait for this to get merged....

To be clear, are we waiting on clarification on this?

Oh, oops, sorry, no, so the 48/50 thing is the zero_conf feature, not the SCID feature, the SCID feature is consistently 46/47 everywhere in the current PR.

Seems this can't fail and can just return a ChannelUpdate instead of Result?

Hmm, maybe we should check state before returning actually? I guess if the channel isn't live we shouldn't even be there, but in any cast it doesn't cost anything to return the Result, no? its just used in one place.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Think this'd be slightly cleaner as:

let mut res = Vec::with_capacity(8 + 128);
// TODO: underspecified, follow https://github.com/lightningnetwork/lightning-rfc/issues/791
if error_code == 0x1000 | 20 {
res.extend_from_slice(&byte_utils::be16_to_array(0));
}
res.extend_from_slice(&upd.encode_with_len()[..]);

taken from 694ef1e

I wanted to avoid the hard-coded length upper bound assumption, but I can use serialized_length, too, I suppose.

Maybe move the log to after Ok, similar below

Hmm, it should always succeed, and is kinda nice to always log - if we don't see the later generation logs in the get_channel_update* methods we know something is wrong.

My thinking is it's cleaner to not return a result if it's not necessary. Fine if the check is desired though

Feel like downgrade_channel_type would be more accurate

Hmm, is it strictly a "downgrade" always, though? I guess my mental model here was we have a list of channel types we are willing to use, and we're iterating through that list until we find one that works.

Given the deserialization issue for older versions of LDK, should we also refuse inbound channels with scid_alias_required unless the config is set?

Yea, I was trying to avoid adding an explicit config for it by letting users do a generic "check channel type flags" thing, though I admit its definitely forcing users to jump through quite a few hoops for something that they may want. It'll need calling out in the release notes either way, just don't know how much we want to complexify the config objects. We could also complexify them and remove the new option in the next release, but doing the manual accept hops isn't crazy either. I dunno.

To be clear, are we waiting on clarification on this?

Looks good, test coverage sounds correct.

I think you should write better the rational for why we're using get_channel_update_broadcast instead of unicast. I didn't get the why at first read.

Something like "SCID alias aims to mask the real channel SCID from the payment sender. If we return the real channel SCID in a channel update error message, it would allow a payment sender to deanonymize the alias by triggering HTLC failures on the routing fees or HTLC minimum amount".

Good catch, but this doesn't need a comment, this is just wrong!

Also updated the test to hit this case.

I thought if short_channel_id is the real scid for a private channel, we still want to leave the update as None?

We do - the chan.get_channel_type().supports_scid_alias() && *short_channel_id != chan.outbound_scid_alias() check below the chan_update_opt get breaks with None, dropping the chan_update_opt. For clarity I moved the get_channel_update_for_onion call down a few lines.

Missing something here -- if we're attempting to forward over a private channel without an alias, and we break on e.g. if !chan.is_live() 10 lines down, won't we include the private channel update with the current code?

Hmm, so the SCIDAlias feature is supposed to be the "never, ever, ever tell anyone the real SCID" flag (maybe we should rename it scid_privacy in LDK?). I guess in principle we could treat any private channel as an SCIDPrivacy/SCIDAlias channel for the purposes of channel_update generation, but I think eventually after people upgrade a private channel without the flag is rare/indicates some strange use-case where they're gonna use the real SCID for something.

Likely before this PR, though not sure if there is test coverage for that.

When I comment this line out I get failures in ln::priv_short_conf_tests::test_priv_forwarding_rejection and ln::priv_short_conf_tests::test_scid_alias_on_pub_channel

Left one question #1351 (comment) and CI's sad. I'm pretty much ACK otherwise

nit: extra line

Test coverage looks good. Only outstanding comment is #1351 (comment).

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

You can do 0conf without scid_alias, so its not a prereq.

FWIW, the bolt has it as a dependency.

FWIW, the bolt has it as a dependency.

Hmm, that's confusing, I believe that is talking about it in the init flags, the part talking about channel types has no such indication - https://github.com/lightning/bolts/pull/910/files#diff-ed04ca2c673fd6aabde69389511fa9ee60cb44d6b2ef6c88b549ffaa753d6afeR216

Would you disagree with naming the feature scid_privacy and leaving everything else as scid_alias? THat's what I was thinking but I don't feel super duper strongly about it.

Yeah, there and maybe the config option I'd be fine with changing.

SGTM

Squashed down the existing fixup commits, added some new ones to address Jeff's last comment and rename the scid_privacy feature.

I'm ACK after Jeff's comments are addressed

Addressed all the comments, let me know if I should squash.

Addressed all the comments, let me know if I should squash.

Yes, please squash

Pushed an additional commit at the start to bump the CI rustc version in check_commits - for some reason older rustcs refuse a perfectly fine doc link.

ACK c47acd7

(Good with the scid_privacy renaming)

nit: You can precise we only store for now the latest announced SCID alias by our counterparty. That's an implementation details that user might be interested to know (e.g if they would like to rotate scid alias for each invoice)

Good point, but feel like that belongs on ChannelDetails::inbound_scid_alias and ChannelDetails::get_inbound_payment_scid

Added an extra commit with more docs (the relevant code is already upstream).