Skip to content

Make payments not duplicatively fail/succeed on reload/reconnect - #918

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged

Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2021-05-dup-claims

Conversation

@TheBlueMatt

@TheBlueMattTheBlueMatt commented May 9, 2021

Copy link
Copy Markdown
Collaborator

We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:

a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.

b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.

If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.

In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.

Thix fixes#209.

@codecov

codecovBot commented May 9, 2021•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.
The diff coverage is 95.86%.

Impacted file tree graph

@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.

valentinewallace
valentinewallace reviewed May 10, 2021
View reviewed changes

@valentinewallacevalentinewallace left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question

Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
valentinewallace
valentinewallace reviewed May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt mentioned this pull request May 12, 2021
Merged
@TheBlueMatt
TheBlueMattforce-pushed the 2021-05-dup-claims branch 2 times, most recently from b45341e to da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt

TheBlueMatt commented May 13, 2021

Copy link
Copy Markdown
CollaboratorAuthor

Squashed with no diff from 853838c1 to da33f5c3

@TheBlueMattTheBlueMatt added this to the 0.0.15 milestone May 15, 2021
@TheBlueMattTheBlueMatt mentioned this pull request May 18, 2021
Closed
jkczyz
jkczyz approved these changes May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs Outdated
Comment threadlightning/src/ln/channelmanager.rs Outdated
@TheBlueMatt
TheBlueMattforce-pushed the 2021-05-dup-claims branch 2 times, most recently from dbaf591 to 74dc019CompareMay 20, 2021 15:24
@TheBlueMatt

TheBlueMatt commented May 20, 2021

Copy link
Copy Markdown
CollaboratorAuthor

Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.

@jkczyz

jkczyz commented May 20, 2021

Copy link
Copy Markdown
Contributor

Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.

LGTM

@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt

TheBlueMatt commented May 20, 2021

Copy link
Copy Markdown
CollaboratorAuthor

Squashed and added one additional commit to address the fuzz failures this introduced.

jkczyz
jkczyz reviewed May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
[id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
[id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],

@jkczyzjkczyzMay 20, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?

@TheBlueMattTheBlueMattMay 20, 2021

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.

@jkczyz

jkczyz commented May 20, 2021

Copy link
Copy Markdown
Contributor

ACK 864375e

Tested locally.

@TheBlueMatt
TheBlueMatt merged commit b6de281 into lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative

3 participants

@TheBlueMatt@jkczyz@valentinewallace
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Make payments not duplicatively fail/succeed on reload/reconnect by TheBlueMatt · Pull Request #918 · lightningdevkit/rust-lightning · GitHub
Skip to content
Make payments not duplicatively fail/succeed on reload/reconnect - #918
Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
May 20, 2021
Merged
Make payments not duplicatively fail/succeed on reload/reconnect#918
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom 
TheBlueMatt:2021-05-dup-claims
Conversation
@TheBlueMatt
@TheBlueMattTheBlueMatt
commented
May 9, 2021

Copy link


Copy Markdown

Collaborator
We currently generate duplicative PaymentFailed/PaymentSent events

in two cases:
a) If we receive a update_fulfill_htlc message, followed by a

disconnect, then a resend of the same update_fulfill_htlc

message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed

through it are simply dropped when the Channel is. From there,

the ChannelManager relies on the ChannelMonitor having a copy of

the relevant fail-/claim-back data and processes the HTLC

fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and

then we serialize the ChannelManager, but do not re-serialize

the relevant ChannelMonitor, we may end up getting a duplicative

event.
In order to provide the expected consistency, we add explicit

tracking of pending outbound payments using their unique

session_priv field which is generated when the payment is sent.

Then, before generating PaymentFailed/PaymentSent events, we check

that the session_priv for the payment is still pending.
Thix fixes#209.
@codecov
codecovBot
commented
May 9, 2021•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #918 (864375e) into main (5d74cae) will decrease coverage by 0.19%.

The diff coverage is 95.86%.
Impacted file tree graph
@@ Coverage Diff @@## main #918 +/- ##
==========================================
- Coverage 90.45% 90.26% -0.20% 
==========================================
Files 59 59 Lines 29785 30569 +784 ==========================================
+ Hits 26941 27592 +651 - Misses 2844 2977 +133 
Impacted FilesCoverage Δ
lightning/src/util/events.rs17.27% <ø> (ø)
lightning/src/ln/channelmanager.rs83.46% <88.88%> (+0.05%)⬆️
lightning/src/ln/functional_tests.rs96.85% <100.00%> (+0.07%)⬆️
lightning/src/ln/msgs.rs88.24% <0.00%> (-0.05%)⬇️
lightning/src/ln/peer_handler.rs46.95% <0.00%> (+2.78%)⬆️
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data

Powered by Codecov. Last update 5d74cae...864375e. Read the comment docs.
valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
@valentinewallacevalentinewallace
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice usability improvement. Still reviewing but I think I just have one docs question
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

valentinewallace
valentinewallace
reviewed
May 10, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
@TheBlueMattTheBlueMatt
mentioned this pull request
May 12, 2021
 Merged

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
b45341e to
da33f5cCompareMay 13, 2021 22:29
@TheBlueMatt
TheBlueMatt
commented
May 13, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed with no diff from 853838c1 to da33f5c3
@TheBlueMattTheBlueMatt
added this to the 0.0.15 milestone
May 15, 2021
@TheBlueMattTheBlueMatt
mentioned this pull request
May 18, 2021

Closed

jkczyz
jkczyz
approved these changes
May 20, 2021
View reviewed changes
Comment threadlightning/src/ln/channelmanager.rs
Outdated

Comment threadlightning/src/ln/channelmanager.rs
Outdated

@TheBlueMatt
TheBlueMattforce-pushed
the
2021-05-dup-claims
branch
2 times, most recently
from
dbaf591 to
74dc019CompareMay 20, 2021 15:24
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
Rebased on latest upstream and added trivial cleanups to address Jeff's comments above. Would like a once-over from one of the existing reviewers.
LGTM
@TheBlueMatt
Make payments not duplicatively fail/succeed on reload/reconnect
3aa4aa8
We currently generate duplicative PaymentFailed/PaymentSent events
in two cases:
a) If we receive a update_fulfill_htlc message, followed by a
disconnect, then a resend of the same update_fulfill_htlc
message, we will generate a PaymentSent event for each message.
b) When a Channel is closed, any outbound HTLCs which were relayed
through it are simply dropped when the Channel is. From there,
the ChannelManager relies on the ChannelMonitor having a copy of
the relevant fail-/claim-back data and processes the HTLC
fail/claim when the ChannelMonitor tells it to.
If, due to an on-chain event, an HTLC is failed/claimed, and
then we serialize the ChannelManager, but do not re-serialize
the relevant ChannelMonitor, we may end up getting a duplicative
event.
In order to provide the expected consistency, we add explicit
tracking of pending outbound payments using their unique
session_priv field which is generated when the payment is sent.
Then, before generating PaymentFailed/PaymentSent events, we check
that the session_priv for the payment is still pending.
Thix fixeslightningdevkit#209.
@TheBlueMatt
Test pending payments when duplicatively resolved on chain
77412dd
@TheBlueMatt
Test pending payments when resolved duplicatively via reconnect
ef4f4f4
@TheBlueMatt
[fuzz] Make get_secure_random_bytes in chanmon_consistency unique
864375e
@TheBlueMatt
TheBlueMatt
commented
May 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Squashed and added one additional commit to address the fuzz failures this introduced.
jkczyz
jkczyz
reviewed
May 20, 2021
View reviewed changes
Comment threadfuzz/src/chanmon_consistency.rs
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, self.node_id]).unwrap(),
 SecretKey::from_slice(&[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, self.node_id]).unwrap(),
 [id, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
 [id as u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, self.node_id],
@jkczyzjkczyzMay 20, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it we don't need to use four bytes here because we won't exhaust the one-byte space?
@TheBlueMattTheBlueMattMay 20, 2021

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we only ever create up to two of them per peer.
@jkczyz
jkczyz
commented
May 20, 2021

Copy link


Copy Markdown

Contributor
ACK 864375e
Tested locally.
@TheBlueMatt
TheBlueMatt
merged commit b6de281
into
lightningdevkit:mainMay 20, 2021
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jkczyzjkczyzjkczyz approved these changes
@valentinewallacevalentinewallacevalentinewallace approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone

0.0.98

Development

Successfully merging this pull request may close these issues.

PaymentSent/PaymentFailed events can be duplicative


3 participants

@TheBlueMatt@jkczyz@valentinewallace