Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Move LockableScore requirement away from Router trait - #1694

Merged
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router
Sep 16, 2022
Merged

Move LockableScore requirement away from Router trait#1694
jkczyz merged 1 commit into
lightningdevkit:mainfrom
jurvis:jurvis/2022-08-move-scorer-from-router

Conversation

@jurvis

@jurvisjurvis commented Sep 5, 2022

Copy link
Copy Markdown
Contributor

Purpose

A scorer is not necessary for users who may be using a different server from their payer node for routing. This PR will allow them to implement a Router without it.

Implementation

  • Move Score type parameterization from InvoicePayer and Router to
    DefaultRouter
  • Adds a new field, scorer, to DefaultRouter
  • Move AccountsForInflightHtlcs to DefaultRouter, which we
    will use to wrap the new scorer field, so scoring only happens in
    DefaultRouter explicitly.
  • Add scoring related functions to Router trait that we used to call
    directly from InvoicePayer.
  • Instead of parameterizing scorer in find_route, we replace it with
    inflight_map so InvoicePayer can pass on information about inflight
    HTLCs to the router.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

I haven't updated the test suite yet, but I wanted to seek concept ACKs on my approach. I'm not 100% sure we want to parameterize inflight_htlcs_map in find_route, in particular.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

@jkczyz

Copy link
Copy Markdown
Contributor

Generally concept ack. Its kinda annoying to make DefaultRouter also an EventHandler but only optionally, though - it means we have to pass both the router and the InvoicePayer to the BackgroundProcessor to pipe events through. As an alternative, we could make the Router trait also require EventHandler, which means we could then call through to the Router's event handling directly in InvoicePayer. Its a bit awkward but at least less code for the user, dunno how @jkczyz would feel about that, though.

My thinking had been we would expand the Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler. Someone may have a RemoteRouter implementation that should still be notified and handle accordingly (e.g., by sending an RPC to the remote server to update the remote scorer).

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Router trait with methods for handling success / failure rather than having DefaultRouter implement EventHandler

ah, got it. makes sense!

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 431d157 to 2032f46CompareSeptember 10, 2022 19:32
@jurvis

Copy link
Copy Markdown
ContributorAuthor

Used a different approach in 2032f46 -- instead of implementing EventHandler in DefaultRouter, have payer call payment_path_successful, payment_path_failed, probe_success, and probe_failure as newly defined function in the Router trait.

@jurvis

Copy link
Copy Markdown
ContributorAuthor

Docs need updating -- I'll look into that in a future commit

@codecov-commenter

codecov-commenter commented Sep 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 90.92% // Head: 92.54% // Increases project coverage by +1.62% 🎉

Coverage data is based on head (df9ea09) compared to base (f99301d).
Patch coverage: 74.21% of modified lines in pull request are covered.

❗ Current head df9ea09 differs from pull request most recent head 82b46ed. Consider uploading reports for the commit 82b46ed to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #1694 +/- ##
==========================================
+ Coverage 90.92% 92.54% +1.62% 
==========================================
Files 85 86 +1 Lines 46268 59531 +13263 Branches 46268 59531 +13263 ==========================================
+ Hits 42067 55091 +13024 - Misses 4201 4440 +239 
Impacted FilesCoverage Δ
lightning-invoice/src/utils.rs95.47% <44.82%> (-1.30%)⬇️
lightning-invoice/src/payment.rs92.34% <80.46%> (+1.47%)⬆️
lightning-background-processor/src/lib.rs96.89% <100.00%> (+1.69%)⬆️
lightning/src/util/events.rs36.73% <0.00%> (-3.04%)⬇️
lightning-block-sync/src/init.rs93.03% <0.00%> (-0.54%)⬇️
lightning/src/ln/chanmon_update_fail_tests.rs97.47% <0.00%> (-0.25%)⬇️
lightning/src/chain/onchaintx.rs94.71% <0.00%> (-0.23%)⬇️
lightning/src/ln/shutdown_tests.rs96.36% <0.00%> (-0.16%)⬇️
lightning/src/ln/payment_tests.rs98.80% <0.00%> (-0.09%)⬇️
lightning/src/ln/script.rs92.13% <0.00%> (-0.06%)⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 7bd046a to 90132ceCompareSeptember 11, 2022 22:01
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM. Would like to make progress quickly here as we may end up needing to backport this for the 0.0.111 bindings.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
// Since the path is reversed, the last element in our iteration is the first
// hop.
let mut locked_scorer = self.scorer.lock();
let scorer = AccountForInFlightHtlcs::new(locked_scorer.deref_mut(), inflight_htlc_map.clone());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange that we are testing AccountForInFlightHtlcs this way rather through DefaultRouter. Then again, seems doing the latter would require using the actual find_route code, which wouldn't lend to a simple test setup. Fine with keeping this as is, but wanted to raise this in case others have any better ideas.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment on lines +579 to +593
fn payment_path_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.payment_path_failed(path, short_channel_id)
}

fn payment_path_successful(&mut self, path: &[&RouteHop]) {
self.scorer.payment_path_successful(path)
}

fn probe_failed(&mut self, path: &[&RouteHop], short_channel_id: u64) {
self.scorer.probe_failed(path, short_channel_id)
}

fn probe_successful(&mut self, path: &[&RouteHop]) {
self.scorer.probe_successful(path)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are never actually called since AccountForInFlightHtlcs is only used for channel_penalty_msat, so they can be implemented as unreachable!(). I wonder if we should split Score into two different traits or if we are fine with duplicating part of the interface in Router. I guess either way, any Score implementation used by DefaultRouter would also need to implement this half of the trait. Not sure what would be prefered. @tnull@TheBlueMatt Any thoughts?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7d95196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yea, I think this is fine for now, we're probably gonna end up revisinting all of it in #1668 anyway, so...

@jurvis
jurvis marked this pull request as ready for review September 13, 2022 03:12

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM.

Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/utils.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs Outdated
jkczyz
jkczyz previously approved these changes Sep 14, 2022

@jkczyzjkczyz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will need a squash once other reviewers are ready.

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, basically.

Comment threadlightning-invoice/src/payment.rs Outdated
Comment threadlightning-invoice/src/payment.rs

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment threadlightning-invoice/src/payment.rs Outdated
@jkczyz

Copy link
Copy Markdown
Contributor

Feel free to squash.

@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from df9ea09 to 82b46edCompareSeptember 15, 2022 21:58
@jurvis

Copy link
Copy Markdown
ContributorAuthor

squashed to 82b46ed without changes

jkczyz
jkczyz previously approved these changes Sep 15, 2022

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit.

//! # use std::ops::Deref;
//! #
//! # #[cfg(not(feature = "std"))]
//! # use core2::io;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you could then also remove the dependency, since it's not used anymore:

diff --git lightning-invoice/Cargo.toml lightning-invoice/Cargo.toml
index cd67bcd3..852f62f4 100644
--- lightning-invoice/Cargo.toml
+++ lightning-invoice/Cargo.toml
@@ -15,7 +15,7 @@ rustdoc-args = ["--cfg", "docsrs"]
[features]
default = ["std"]
-no-std = ["hashbrown", "lightning/no-std", "core2/alloc"]
+no-std = ["hashbrown", "lightning/no-std"]
std = ["bitcoin_hashes/std", "num-traits/std", "lightning/std", "bech32/std"]
[dependencies]
@@ -25,7 +25,6 @@ secp256k1 = { version = "0.24.0", default-features = false, features = ["recover
num-traits = { version = "0.2.8", default-features = false }
bitcoin_hashes = { version = "0.11", default-features = false }
hashbrown = { version = "0.11", optional = true }
-core2 = { version = "0.3.0", default-features = false, optional = true }
serde = { version = "1.0.118", optional = true }
[dev-dependencies]

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 addressed and squashed into c353c3e

We do this to enable users to create routers that do not need a scorer.
This can be useful if they are running a node the delegates pathfinding.
* Move `Score` type parameterization from `InvoicePayer` and `Router` to
`DefaultRouter`
* Adds a new field, `scorer`, to `DefaultRouter`
* Move `AccountsForInFlightHtlcs` to `DefaultRouter`, which we
will use to wrap the new `scorer` field, so scoring only happens in
`DefaultRouter` explicitly.
* Add scoring related functions to `Router` trait that we used to call
directly from `InvoicePayer`.
* Instead of parameterizing `scorer` in `find_route`, we replace it with
inflight_map so `InvoicePayer` can pass on information about inflight
HTLCs to the router.
* Introduced a new tuple struct, InFlightHtlcs, that wraps functionality
for querying used liquidity.
@jurvis
jurvisforce-pushed the jurvis/2022-08-move-scorer-from-router branch from 82b46ed to c353c3eCompareSeptember 16, 2022 15:39
@jkczyz
jkczyz merged commit ca76d06 into lightningdevkit:mainSep 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@jurvis@TheBlueMatt@jkczyz@codecov-commenter@tnull