Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Tweak and Expand the chanmon_consistency fuzz target by TheBlueMatt · Pull Request #753 · lightningdevkit/rust-lightning · GitHub
Skip to content

Tweak and Expand the chanmon_consistency fuzz target - #753

Merged
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits
Nov 23, 2020
Merged

Tweak and Expand the chanmon_consistency fuzz target#753
TheBlueMatt merged 5 commits into
lightningdevkit:mainfrom
TheBlueMatt:2020-11-chanmon_consistency-bits

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

I spent some time with chanmon_consistency to expand the errors it could catch. Luckily no bugs turned up (yet) from this work.

@codecov

codecovBot commented Nov 18, 2020

Copy link
Copy Markdown

Codecov Report

Merging #753 (401880f) into main (4e82003) will decrease coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## main #753 +/- ##
==========================================
- Coverage 91.45% 91.38% -0.07% 
==========================================
Files 37 37 Lines 22249 22249 ==========================================
- Hits 20347 20333 -14 - Misses 1902 1916 +14 
Impacted FilesCoverage Δ
lightning/src/ln/functional_tests.rs96.92% <0.00%> (-0.25%)⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e82003...6563f7a. Read the comment docs.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Comment on lines +417 to +425
match err.as_str() {
"Peer for first hop currently disconnected/pending monitor update!" => test_return!(),
_ if err.starts_with("Cannot push more than their max accepted HTLCs ") => test_return!(),
_ if err.starts_with("Cannot send value that would put us over the max HTLC value in flight our peer will accept ") => test_return!(),
_ if err.starts_with("Cannot send value that would put our balance under counterparty-announced channel reserve value") => test_return!(),
_ if err.starts_with("Cannot send value that would overdraw remaining funds.") => test_return!(),
_ if err.starts_with("Cannot send value that would not leave enough to pay for fees.") => test_return!(),
_ => panic!(err),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.

Doesn't this still result in the test returning if a send fails?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - we only return from do_test if either we read a byte we don't know how to process or we run out of bytes. Prior to this we would test_return!() if we max out sending over a channel as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aye, think it was changed as the quoted code is using test_return! but now is {}. So should be good.

Comment on lines -705 to +720
0x03 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x04 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_c.update_ret.lock().unwrap() = Ok(()),
0x06 => {
0x04 => *monitor_a.update_ret.lock().unwrap() = Ok(()),
0x05 => *monitor_b.update_ret.lock().unwrap() = Ok(()),
0x06 => *monitor_c.update_ret.lock().unwrap() = Ok(()),

0x08 => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help me understand why values like 0x03 and 0x07 are skipped?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep related message groups having the same binary prefix. Honestly it probably doesn't matter, but fuzzers do do some bit-twiddling-based tweaking and I figured no reason to not have bit-twiddling often have similar effects. I noted it in a new comment.

Comment threadfuzz/src/chanmon_consistency.rs Outdated
Instead of simply always considering a payment-send failure as
acceptable (and aborting fuzzing), we check that a payment send
failure is from a list of errors that we know we can hit, mostly
around maxing out our channel balance.
Critically, we keep going after hitting an error, as there's no
reason channels should get out of sync even if a send fails.
This should make it a bit easier for the fuzzer to hit any given
balance breakdown during run as well as tweaks the command strings
to be more bit-pattern friendly.
In previous versions of related commits, the macros in
chanmon_consistency ended up blowing up rustc a bit resulting in
20+GB memory usage and long compile times. Shorter function bodies
by avoiding macros where possible fix this.
We should never generate Ignore-action HandleError events anymore
This adds a new command string in the chanmon_consistency fuzzer
which tests that, once all pending HTLCs are settled, at least one
side of a channel can still send funds.
While this should have caught the recent(ish) spec bug where
channels could get stuck, I did not attempt to reproduce said bug
with this patch.
@TheBlueMatt
TheBlueMattforce-pushed the 2020-11-chanmon_consistency-bits branch from 6626d08 to 6563f7aCompareNovember 21, 2020 17:12
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Gonna take so that we can talk about/analyze things like #757 against git latest instead of against some random branch.

@TheBlueMatt
TheBlueMatt merged commit 8f10a1d into lightningdevkit:mainNov 23, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@TheBlueMatt@jkczyz