Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Generate latest local commitment transactions via monitor avoiding Channel's copy by TheBlueMatt · Pull Request #551 · lightningdevkit/rust-lightning · GitHub
Skip to content

Generate latest local commitment transactions via monitor avoiding Channel's copy - #551

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon
Mar 20, 2020
Merged

Generate latest local commitment transactions via monitor avoiding Channel's copy#551
TheBlueMatt merged 5 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-03-no-chan-mon

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

One more step towards removing the Channel-internal channel_monitor, having the ChannelMonitor that is external to the Channel generate/broadcast the latest local commitment transaction on force-close via an update instead of doing it in Channel/Manager. This should make #540 much simpler.

@TheBlueMattTheBlueMatt mentioned this pull request Mar 19, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some function descriptions to update.

&HTLCUpdateAwaitingACK::ClaimHTLC { htlc_id, .. } => {
if htlc_id_arg == htlc_id {
// Make sure we don't leave latest_monitor_update_id incremented here:
self.latest_monitor_update_id -= 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c95f91b

Hitting this case would indicate a programming error right ? If so I think function description doesn't match anymore what we really do given we don't return IgnoreError anymore here. You should add a debug_assert too?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I fixed the underlying duplicate events, added the debug_assertions, and added a comment noting that it is possible to hit them in some reorg cases.

Comment threadlightning/src/ln/channelmanager.rs Outdated
log_trace!(self, "Broadcast onchain {}", log_tx!(tx));
self.tx_broadcaster.broadcast_transaction(&tx);
if let Some(funding_txo) = funding_txo_option {
// XXX: Add comment for why this is OK.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

"ChannelMonitor tracks the whole channel state, if requested to do so, it will broadcast local commitment transaction and any associated HTLC transactions for which we have a valid witness"

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. thanks.

@@ -3477,9 +3471,9 @@ impl<'a, ChanSigner: ChannelKeys + Readable, M: Deref, T: Deref, K: Deref, F: De
channel.get_revoked_remote_commitment_transaction_number() != monitor.get_min_seen_secret() ||
channel.get_cur_remote_commitment_transaction_number() != monitor.get_cur_remote_commitment_number() ||
channel.get_latest_monitor_update_id() != monitor.get_latest_update_id() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Hmmm I know that's already current behavior but here that means if channel deserialization doesn't match monitor one, we force-close, isn't this an issue in case of monitor being older than channel ? In that case, we're fallen-behind and should wait?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The monitor should never be behind the channel unless the many-monitor was mis-implemented. The docs are (hopefully) pretty clear in this regard - monitor updates must happen in-sync, channel[manager] updates happen in the background, and only afterwards.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be defensive here and assert in case of many-monitor being misimplemented we catch issue at ChannelManager deserialization?

if should_broadcast {
self.broadcast_latest_local_commitment_txn(broadcaster);
} else {
log_error!(self, "You have a toxic local commitment transaction avaible in channel monitor, read comment in ChannelMonitor::get_latest_local_commitment_txn to be informed of manual action to take");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c6470ad

Update get_latest_local_commitment_txn documentation, it's not called anymore by ChannelManager.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It still is, just indirectly (via broadcast_latest_local_commitment_txn). I'll let you update the comment when you refactor this further.

Comment threadlightning/src/ln/channelmonitor.rs
Comment threadlightning/src/ln/functional_test_utils.rs
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

I believe all our comments have been addressed. Thanks @ariard!

@ariard

Copy link
Copy Markdown

ACK 42eccfa including new commit bb8ebd4 (nit: get_update_fulfill_htlc still mentions IgnoreError?)

If we call get_update_fulfill_htlc (in this case via
ChannelManager::claim_funds_internal ->
Channel::get_update_fulfill_htlc_and_commit) and it finds that we
already have a holding-cell pending HTLC claim, it will return no
monitor update but leave latest_monitor_update_id incremented.
If we later go and add a new monitor update we'll panic as the
updates appear to have been applied out-of-order.
This avoids calling get_update_fulfill_htlc_and_commit twice for
the same HTLC if we have to rescan a block.
This makes it easier to swap out how we fetch the latest local
commitment txn in testing (which we use to check or broadcast old
states).
Eventually, we want to remove the Channel's copy of its own
ChannelMonitor, reducing memory footprint and complexity of
ChannelManager greatly.
This removes the last uses of said ChannelMonitor for latest
local commitment transactions (though it is still used for
would_broadcast_at_height(), which is the last remaining use).
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is I fixed the comment to not refer to IgnoreError, so will merge after travis passes.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@ariard