Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Expose HTLCStats in ChannelDetails - #2349

Closed
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details
Closed

Expose HTLCStats in ChannelDetails#2349
wvanlint wants to merge 1 commit into
lightningdevkit:mainfrom
wvanlint:htlc_stats_in_channel_details

Conversation

@wvanlint

@wvanlintwvanlint commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Exposes details around in-flight HTLCs through HTLCStats in ChannelDetails. This enables measuring saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch 2 times, most recently from 2d30dad to 374cec2CompareJune 9, 2023 23:54
@codecov-commenter

codecov-commenter commented Jun 10, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01⚠️

Comparison is base (d401848) 90.48% compared to head (039c38d) 90.47%.

❗ Current head 039c38d differs from pull request most recent head 0f5cd35. Consider uploading reports for the commit 0f5cd35 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2349 +/- ##
==========================================
- Coverage 90.48% 90.47% -0.01% 
==========================================
Files 104 104 Lines 53920 53930 +10 Branches 53920 53930 +10 ==========================================
+ Hits 48790 48795 +5 - Misses 5130 5135 +5 
Impacted FilesCoverage Δ
lightning/src/routing/router.rs93.28% <ø> (+0.07%)⬆️
lightning/src/ln/channel.rs89.68% <100.00%> (ø)
lightning/src/ln/channelmanager.rs86.78% <100.00%> (+0.03%)⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally makes sense to me, thanks for having a go at this!

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated

impl_writeable_tlv_based!(HTLCStats, {
(0, pending_htlcs, required),
(1, pending_htlcs_value_msat, required),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When introducing mandatory fields for a new structure they usually should be even-numbered as this makes older nodes which for whatever reason encounter the unknown field stop and error rather than ignore and continue deserialization.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't realize this follows BOLT 1 serialization, done.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
/// The total dust exposure for the local node.
pub on_holder_tx_dust_exposure_msat: u64,
/// The total value of pending, outgoing HTLC's being added that are awaiting remote revocation.
/// See `ChannelState::AwaitingRemoteRevoke`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChannelState is not public, so we shouldn't refer to it in the public API. If it has some relevant docs/information, we need to replicate them here in brief.

@wvanlintwvanlintJun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added additional explanation instead of the reference.

Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channel.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
(35, self.inbound_htlc_maximum_msat, option),
(37, user_channel_id_high_opt, option),
(39, self.feerate_sat_per_1000_weight, option),
(40, self.incoming_htlc_stats, option),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should both be odd: older nodes should just ignore them if they would ever encounter a ChannelDetails that was serialized with version 0.0.116 or above.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

/// Statistics on pending incoming HTLCs.
///
/// This field is only `None` for `ChannelDetails` objects serialized prior to LDK 0.0.116.
pub incoming_htlc_stats: Option<HTLCStats>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the channel module is not public, HTLCStats is currently not exposed. We need to expose the object itself if it is now exposed through ChannelDetails.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved HTLCStats to the channelmanager module for that purpose.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 374cec2 to 039c38dCompareJune 12, 2023 18:57
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else. I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

@wvanlint
wvanlintforce-pushed the htlc_stats_in_channel_details branch from 039c38d to 0f5cd35CompareJune 12, 2023 21:24
@wvanlint

wvanlint commented Jun 12, 2023

Copy link
Copy Markdown
ContributorAuthor

This needs a lot more motivation, I'm not sure I understand the goal here - I'm not sure whether this is intended to work towards #2087 or #2264 or something else.

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat, so I think this corresponds closest to #2087.

I'm also afraid these are a bit hard to discipher for a user, there's a lot of states an HTLC can be in, and the HTLCStats struct doesn't do a great job of indicating how much of each state an HTLC can be in.

Does this refer to Inbound/OutboundHTLCState? Should there be more granular aggregation in HTLCStats? For our purposes, the coarse granularity of HTLCStats might be sufficient.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

The main goal is to enable measuring the saturation of a channel with regards to max_accepted_htlcs and max_htlc_value_in_flight_msat

Ah! Okay, that's much more straightforward, so let's not expose all the nuance - lets try to break down the value into buckets such that it always totals to the current channel value eg: our_unencumbered_value, their_unencumbered_value, our_reserve, their_reserve, our_outbound_htlcs, and their_outbound_htlcs, (both as amounts and counts). There will always be some overhead for fees that we could try to write out but its a pita.

I think such an API will be much easier to use than the current HTLCStats, mostly because it would avoid the user having any idea what the "holding cell" is and the dust exposure being broken out.

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Comment threadlightning/src/ln/channelmanager.rs
@wvanlint

Copy link
Copy Markdown
ContributorAuthor

That fails to communicate the dust exposure (ala #2264) but accomplishes what you want. Bonus points for solving both in one go - instead of returning the htlc amount/count, return the actual list of HTLCs with extra info (is it dust? its payment hash, CLTV expiry, etc).

Will handle both use cases by returning the list of pending HTLC's in #2442.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@wvanlint@codecov-commenter@TheBlueMatt@tnull@tlulu