Attributable errors 2025 by joostjager · Pull Request #3611 · lightningdevkit/rust-lightning

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager · 2025-02-21T13:39:13Z

Work PR for reviving #2256

TheBlueMatt · 2025-02-25T22:40:51Z

Okay, you got totally bitten by some really old serialization logic that was written when upgrade/downgrade support was...not super well thought-through (and which was not upgraded when we made the transition, sadly). Luckily there's fairly little of this code left, but you jumped right into an area where there's a good bit of it :(.

I went ahead and implemented the serialization logic at https://git.bitcoin.ninja/?p=rust-lightning;a=log;h=refs/heads/2025-02-3611-ser-impl note that the attribution_data field has to be optional in OnionErrorPacket to support upgrade. The middle commit I'll go ahead and upstream via a separate PR now.

joostjager · 2025-02-27T15:57:22Z

- 4 => InboundHTLCState::LocalRemoved(Readable::read(reader)?),
+ 4 => {
+ let reason = match <u8 as Readable>::read(reader)? {
+ 0 => InboundHTLCRemovalReason::FailRelay(msgs::OnionErrorPacket {


Update in holding cell, still waiting for next round of commitment dance. Should be tests that test holding cell persistence.
Strategy: break main, see if tests fail.

expect_pending_htlcs_forwardable!(nodes[1]); - in here let node 0 do something else like add another htlc, so that the failure gets into the holding cell

joostjager · 2025-02-27T15:59:29Z

 channel_id.write(writer)?;
 htlc_id.write(writer)?;
 reason.write(writer)?;
+ attribution_data.write(writer)?; // TODO: Make TLV?


Try remove serialization logic, see where it used.
Look at call site, see what context is, and find how to add more data at a higher level (tlv?)

joostjager · 2025-02-27T16:03:03Z

 channel_id: Readable::read(reader)?,
 htlc_id: Readable::read(reader)?,
 reason: Readable::read(reader)?,
+ attribution_data: Readable::read(reader)?,


change write side to use version 2/3 and it will have the length descriptor. can add additional content at the end.
maybe intro tlv stream here?

codecov · 2025-03-07T07:50:25Z

Codecov Report

Attention: Patch coverage is 92.81867% with 40 lines in your changes missing coverage. Please review.

Project coverage is 90.24%. Comparing base (3a5f428) to head (a8316f1).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/ln/channel.rs	79.41%	1 Missing and 20 partials ⚠️
lightning/src/ln/onion_utils.rs	95.75%	11 Missing and 4 partials ⚠️
lightning/src/ln/channelmanager.rs	86.36%	3 Missing ⚠️
lightning/src/ln/onion_payment.rs	80.00%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #3611 +/- ##
==========================================
+ Coverage 89.17% 90.24% +1.06% 
==========================================
Files 155 156 +1 Lines 119379 126680 +7301 Branches 119379 126680 +7301 ==========================================
+ Hits 106462 114326 +7864 + Misses 10312 9818 -494 + Partials 2605 2536 -69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joostjager · 2025-03-11T15:18:08Z

+ .copy_from_slice(&payloads[..payloads.len() - PAYLOAD_LEN]);
+ }
+
+ // Shift hmacs right.


Hmac pruning happening here

Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager · 2025-03-11T15:40:29Z

Closing this work PR, pushed updated and reworked code to original PR #2256

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

joostjager marked this pull request as draft February 21, 2025 13:39

joostjager changed the title ~~Attr errors 2025~~Attributable errors 2025Feb 21, 2025

joostjager force-pushed the attr-errors-2025 branch from 97480b6 to 72a8397Compare February 25, 2025 16:07

joostjager force-pushed the attr-errors-2025 branch from d2a3b6a to 936a2b9Compare February 26, 2025 11:11

joostjager mentioned this pull request Feb 27, 2025
Attribution data (feature 36/37) lightning/bolts#1044
Merged

joostjager commented Feb 27, 2025
View reviewed changes

Comment threadlightning/src/ln/channel.rs

joostjager commented Feb 27, 2025
View reviewed changes

joostjager force-pushed the attr-errors-2025 branch 2 times, most recently from 33c4368 to dc4b841Compare March 4, 2025 11:56

joostjager mentioned this pull request Mar 5, 2025
Add persistence test for htlc in the LocalRemoved state #3645
Merged

joostjager force-pushed the attr-errors-2025 branch 15 times, most recently from fa88d2f to fbd8e91Compare March 6, 2025 09:36

joostjager mentioned this pull request Mar 6, 2025
Attributable failures prefactor #3650
Merged

joostjager force-pushed the attr-errors-2025 branch from fbd8e91 to 2c07c7bCompare March 7, 2025 07:41

joostjager force-pushed the attr-errors-2025 branch 3 times, most recently from a8316f1 to 06b8979Compare March 10, 2025 13:13

joostjager commented Mar 11, 2025
View reviewed changes

joostjager added 2 commits March 11, 2025 16:31

Attributable failures
d282ea8

Attributable failures optional feature bit
e7923e9
Signal to senders that the node will return attributable failures. When the sender makes sure all path nodes supports this, they will be able to attribute every failure that may occur.

joostjager force-pushed the attr-errors-2025 branch from 06b8979 to e7923e9Compare March 11, 2025 15:32

joostjager closed this Mar 11, 2025

Uh oh!

Conversation

joostjager commented Feb 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheBlueMatt commented Feb 25, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjagerFeb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjagerFeb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecovBot commented Mar 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager commented Mar 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

joostjager commented Feb 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheBlueMatt commented Feb 25, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjagerFeb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjagerFeb 27, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecovBot commented Mar 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager commented Mar 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

joostjager commented Feb 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheBlueMatt commented Feb 25, 2025

Uh oh!

Uh oh!

joostjager commented Feb 21, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

codecovBot commented Mar 7, 2025•
edited
Loading

joostjager commented Feb 21, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

codecovBot commented Mar 7, 2025•
edited
Loading

joostjager commented Feb 21, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading

codecovBot commented Mar 7, 2025•
edited
Loading

joostjager commented Feb 21, 2025•
edited
Loading

joostjagerFeb 27, 2025•
edited
Loading