Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Try to overpay the recipient if we fail to find a path at all and limit overpay - #2604

Merged
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit
Sep 29, 2023
Merged

Try to overpay the recipient if we fail to find a path at all and limit overpay #2604
TheBlueMatt merged 4 commits into
lightningdevkit:mainfrom
TheBlueMatt:2023-09-route-overpay-limit

Conversation

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.

Depends on #2575.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Checked that tests succeed when rebased on #2575.

Comment threadlightning/src/routing/router.rs Outdated
Comment threadlightning/src/routing/router.rs
@tnulltnull linked an issue Sep 27, 2023 that may be closed by this pull request
This was referenced Sep 27, 2023
@valentinewallace

Copy link
Copy Markdown
Contributor

Needs a rebase on #2575 to fix CI.

Comment threadlightning/src/routing/router.rs Outdated
channel_saturation_pow_half = 0;
} else if already_collected_value_msat < final_value_msat && path_value_msat != recommended_value_msat && !found_new_path {
log_trace!(logger, "Failed to collect enough value, but running again to collect extra paths with a potentially higher limit.");
path_value_msat = recommended_value_msat;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we only do this if we hit_minimum_limit? It doesn't hurt to go around again, but isn't hit_min_limit the only way for there to be uncollected paths remaining?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm, fair, yea, I can add that restriction.

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch 3 times, most recently from 50633ed to ae9df4aCompareSeptember 28, 2023 18:23
Comment threadlightning/src/routing/router.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to squash fixups (when rebasing).

@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from ae9df4a to bbdf4b9CompareSeptember 28, 2023 18:54
@TheBlueMatt

TheBlueMatt commented Sep 28, 2023

Copy link
Copy Markdown
CollaboratorAuthor

Squashed without change - it shouldn't need rebase as its based directly on #2575, as long as that gets merged without any change in the commits git[hub] is smart enough to ignore it.

tnull
tnull previously approved these changes Sep 28, 2023

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

codecov-commenter commented Sep 28, 2023

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (082a19b) 89.00% compared to head (fa48df6) 89.04%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@ Coverage Diff @@## main #2604 +/- ##
==========================================
+ Coverage 89.00% 89.04% +0.03% 
==========================================
Files 112 112 Lines 86273 86337 +64 Branches 86273 86337 +64 ==========================================
+ Hits 76791 76876 +85 + Misses 7248 7230 -18 + Partials 2234 2231 -3 
FilesCoverage Δ
lightning/src/routing/router.rs93.93% <98.68%> (+0.07%)⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:30

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

The merge-base changed after approval.

Lollll that's new. Sorry, have to re-ack.

tnull
tnull previously approved these changes Sep 28, 2023
@TheBlueMatt
TheBlueMatt dismissed tnull’s stale reviewSeptember 28, 2023 20:35

The merge-base changed after approval.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Ugh wtf. Okay I'm gonna have to change ack policy wtf.

While this doesn't matter much in practice, if we go around again
when route-finding to try to meet an htlc_minimum_msat, we use the
`recommended_value_msat` which can work if we meet the
`htlc_minimum_msat` on a channel exactly, so using >= rather than >
can capture cases with 1msat more.
Previously we'd only try to overpay if we managed to find a path
to the recipient which was sufficient. However, if we fail to find
any path to the recipient at all we should still retry overpaying
the recipient. Ultimately we should be silling to pay whatever
reasonable performance penalty if the alternative is not finding a
path at all, which we do here.
If the user told us to limit their total fee exposure, we should
do so including any potential overpayment to the recipient, which
is ultimately a part of the "fee" as far as the user is concerned.
This may be useful in debugging routing failures in the future.
@TheBlueMatt
TheBlueMattforce-pushed the 2023-09-route-overpay-limit branch from bbdf4b9 to fa48df6CompareSeptember 28, 2023 20:39
@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

Rebased without change, I think now maybe it wont drop reviews any time there's a merge.

@TheBlueMatt

Copy link
Copy Markdown
CollaboratorAuthor

As usual, there is no setting anywhere that seems to line up with what just happened, so I guess we just have to require it be based on an upstream merge commit 🤷

@TheBlueMatt
TheBlueMatt merged commit db41b87 into lightningdevkit:mainSep 29, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#2417 followups

4 participants

@TheBlueMatt@valentinewallace@codecov-commenter@tnull