Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Few more ChannelMonitor Cleanups - #597

Merged
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups
Apr 23, 2020
Merged

Few more ChannelMonitor Cleanups#597
TheBlueMatt merged 7 commits into
lightningdevkit:masterfrom
TheBlueMatt:2020-04-more-chanmon-cleanups

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Based on #594, this cleans up ChannelMonitor a bit more. Still playing with this, so its a draft, most especially not sure if I'm gonna want to do "De-Option<> current_local_signed_commitment_tx in ChannelMonitor" yet, depends on how things shake out at the end.

@TheBlueMatt
TheBlueMatt marked this pull request as ready for review April 20, 2020 05:05
@TheBlueMattTheBlueMatt mentioned this pull request Apr 20, 2020
@valentinewallacevalentinewallace added this to the 0.0.11 milestone Apr 20, 2020

@ariardariard left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly good, just some clarifications. You can take to cleanup a bit more ariard@3bb1afe

DecodeError::UnknownVersion => f.write_str("Unknown realm byte in Onion packet"),
DecodeError::UnknownRequiredFeature => f.write_str("Unknown required feature preventing decode"),
DecodeError::InvalidValue => f.write_str("Nonsense bytes didn't map to the type they were interpreted as"),
DecodeError::ShortRead => f.write_str("Packet extended beyond the provided bytes"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f5b0663

Error description doesn't make sense with field comment "Buffer too short" IMO

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in 594, needs to be in a followup PR.

Comment threadlightning/src/ln/channelmonitor.rs Outdated
/// tells you the last block hash which was block_connect()ed. You MUST rescan any blocks along
/// the "reorg path" (ie not just starting at the same height but starting at the highest
/// common block that appears on your best chain as well as on the chain which contains the
/// last block hash returned) upon deserializing the object!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

01db259

I think this comment may be wrong and lead to bug. You must not only rescan from common ancestor but even before disconnect any block from reorg'ed branch. Not doing so may lead to false onchain_events_waiting_threshold_conf confirmations and therefore returning earlier than expected. Or not rebroadcasting a transaction for a reor'ed out outpoint ?

Should we be more severe in block_connected method and assert than any connection is parent of previous last_block_hash or last_block_hash for in-place rescan ?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, right, its intended to mean that but the parenthetical isn't clear. I'll change it but note that this isn't new documentation. Yes, we should probably be much stricter in block_connected+disconnected methods in a few ways, that included.

@@ -1843,6 +1843,9 @@ impl<ChanSigner: ChannelKeys> Channel<ChanSigner> {
return Err((None, ChannelError::Close("Got wrong number of HTLC signatures from remote")));
}

// TODO: Merge these two, sadly they are currently both required to be passed separately to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3aed7d

I'm not sure about merging both, they aren't targeted to the same layer, HTLCOutputInCommitment is addressed to OnchainTxHandler and HTLCSource for ChannelManager even if we use for both index to take decisions.
First one is to generate correct redeemScript and commitment transaction, second one is to link HTLCs through a payment path.

Even in ChannelMonitor, IIRC we don't use both at same location.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but just because they're intended for different uses doesn't mean we should be passing two separate vectors over with redundant data. We can pass one Vec and let ChannelMonitor figure out how to put what where.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay but that means ChannelMonitor doing more repacking stuff on sensible code paths, which has been error-prone in the past (something we can improve with well-documented named structures). There is also few intersection between them beyond index

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but its even easier to get confused passing things twice than it is to get confused passing things once and having to think about a well-documented value :).

feerate_per_kw: initial_local_commitment_tx.feerate_per_kw,
htlc_outputs: Vec::new(), // There are never any HTLCs in the initial commitment transactions
};
// Returning a monitor error before updating tracking points means in case of using

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

06fc27d

Was this my comment or yours ? Because if watchtower doesn't reject update, it means they have the latest local valid commitment, not us. But if we reject update, because of a block triggering broadcast, our tip local commitment is a valid one too. It's just the prev one?

I think we are robust against this scenario right now or what failure do you consider ? If we reject update (quorum or single-failure only) revocation secret shouldn't have been released offchain?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not mine. It is in provide_latest_local_commitment_tx_info on master with a blame of e46e183.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your point was that if we update one monitor but another refuses the update, then the first monitor may broadcast a new local state, which the second monitor won't have stored anywhere. I believe this is still the case after this PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right my comment is confusing as I read it, what I mean here is by latest local valid is a in fact the prev one from the other watchtower viewpoint. Anyway we aren't robust against this, and more I'm thinking there is also dirty reorg-case where one monitor lockdown itself, broadcast commitment, this one get reorg'ed out... Other monitors have always been on the main chain, what happens ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #604

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, lets discuss it on the issue. Its def confusing.

@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from 06fc27d to d0b6b72CompareApril 21, 2020 20:37
Not only was watchtower mode never implemented, but the bits that
we had were removed some time ago. It doesn't seem likely we'll
move forward with a "watchtower-mode" ChannelMonitor, instead
we'll likely have some other, separate struct for this.
@ariard

Copy link
Copy Markdown

Code Review ACK d0b6b72

3d640da looped over a new HashMap
new_claims, clone()ing entries out of it right before droppng the
whole thing. This is an obvious candidate for drain(..).
The ChanKeys is created with knowledge of the Channel's value and
funding redeemscript up-front, so we should not be providing it
when making signing requests.
1107ab0 introduced some additional
metadata, including per-HTLC data in LocalCommitmentTransaction. To
keep diff reasonable it did so in ChannelMonitor after the
LocalCommitmentTransaction had been constructed and passed over the
wall, but there's little reason to do so - we should just be
constructing them with the data from the start, filled in by Channel.
This cleans up some internal interfaces a bit, slightly reduces
some data duplication and moves us one step forward to exposing
the guts of LocalCommitmentTransaction publicly in a sensible way.
Previously, we created the initial ChannelMonitor on outbound
channels when we generated the funding_created message. This was
somewhat unnecessary as, at that time, we hadn't yet received
clearance to broadcast our initial funding transaction, and thus
there should never be any use for a ChannelMonitor. It also
complicated ChannelMonitor a bit as, at this point, we didn't have
an initial local commitment transaction.
By moving the creation of the initial ChannelMonitor to when we
receive our counterparty's funding_signed, we can ensure that any
ChannelMonitor will always have both a latest remote commitment tx
and a latest local commitment tx for broadcast.
This also fixes a strange API where we would close a channel
unceremoniously on peer-disconnection if we hadn't yet received the
funding_signed, but we'd already have a ChannelMonitor for that
channel. While it isn't strictly a bug (some potential DoS issues
aside), it is strange that these two definitions of a channel being
open were not in sync.
Since we now are always initialised with an initial local commitment
transaction available now, we might as well take advantage of it and
stop using an Option<> where we don't need to.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-04-more-chanmon-cleanups branch from d0b6b72 to 80055d4CompareApril 23, 2020 17:35
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Only change is drop the "NEW: " from the new commit, so gonna merge as-is once travis passes.

@TheBlueMatt
TheBlueMatt merged commit 5b24d3e into lightningdevkit:masterApr 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@TheBlueMatt@ariard@valentinewallace