Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lightning/src/onion_message/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,11 +23,13 @@
mod blinded_route;
mod messenger;
mod packet;
mod router;
mod utils;
#[cfg(test)]
mod functional_tests;

// Re-export structs so they can be imported with just the `onion_message::` module prefix.
pub use self::blinded_route::{BlindedRoute, BlindedHop};
pub use self::messenger::{Destination, OnionMessenger, SendError, SimpleArcOnionMessenger, SimpleRefOnionMessenger};
pub use self::router::find_path;
pub(crate) use self::packet::Packet;
248 changes: 248 additions & 0 deletions lightning/src/onion_message/router.rs
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
use bitcoin::secp256k1::PublicKey;

use ln::msgs::{ErrorAction, LightningError};
use routing::gossip::{NetworkGraph, NodeId};
use util::logger::{Level, Logger};

use alloc::collections::BinaryHeap;
use core::hash::Hash;
use core::ops::Deref;
use prelude::*;

/// Find a path for sending an onion message.
pub fn find_path<L: Deref, GL: Deref>(
Comment thread
valentinewallace marked this conversation as resolved.
our_node_pubkey: &PublicKey, receiver_pubkey: &PublicKey, network_graph: &NetworkGraph<GL>, first_hops: Option<&[&PublicKey]>, logger: L
) -> Result<Vec<PublicKey>, LightningError> where L::Target: Logger, GL::Target: Logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we wouldn't reuse all the information we're learning and storing in our ProbabilisticScorer. I can see how we're limited with the current penalty being based on the link-level and here we might be interested by node-level reliability in the path construction. That said, we can also assume that a reliable channel == a reliable onion message communication channel and go with it. I don't know if the spec says anything here or what are the thinking of other implementations ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an option in the spec to return a separate error for "peer offline" from "no capacity", but I'm not sure how common that is. Once we land the historical scoring PR, we could use the "time in the zero-available-capacity bucket" as a score here.

{
let graph_lock = network_graph.read_only();
let network_channels = graph_lock.channels();
let network_nodes = graph_lock.nodes();
let our_node_id = NodeId::from_pubkey(our_node_pubkey);

let mut for_each_successor = |node_id, callback: &mut FnMut(&NodeId, u64)| {
// TODO: in this method, check if OM forwarding feature bit is supported
if node_id == our_node_id && first_hops.is_some() {
if let Some(first_hops) = first_hops {
for hop in first_hops {
callback(&NodeId::from_pubkey(hop), 1);
}
}
} else if let Some(node_info) = network_nodes.get(&node_id) {
for scid in &node_info.channels {
if let Some(chan_info) = network_channels.get(&scid) {
let successor_node_id = if chan_info.node_one == node_id {
Comment thread
valentinewallace marked this conversation as resolved.
&chan_info.node_two
} else {
debug_assert!(chan_info.node_two == node_id);
&chan_info.node_one
};
callback(successor_node_id, 1); // Use a fixed cost for each hop until scoring is added
}
}
}
};

let mut invalid_final_hop_pk = None;
let mut convert_final_hop = |node_id: &NodeId| {
match PublicKey::from_slice(node_id.as_slice()) {
Ok(pk) => Ok(pk),
Err(e) => {
invalid_final_hop_pk = Some(*node_id);
Err(())
},
}
};

let receiver_node_id = NodeId::from_pubkey(receiver_pubkey);
match dijkstra(our_node_id, &mut for_each_successor, |node_id| node_id == &receiver_node_id, &mut convert_final_hop) {
Ok(p) => Ok(p),
Err(Error::PathNotFound) => Err(LightningError {
err: "Failed to find a path to the given destination".to_owned(),
action: ErrorAction::IgnoreError,
}),
Err(Error::FinalHopConversion) => {
debug_assert!(invalid_final_hop_pk.is_some());
Err(LightningError {
err: format!("Public key {:?} is invalid", invalid_final_hop_pk),
action: ErrorAction::IgnoreAndLog(Level::Trace)
})
}
}
}

#[derive(Debug, PartialEq)]
/// Errored running `dijkstra`.
enum Error {
/// No path exists to the destination.
PathNotFound,
/// Converting the processing hop type to the final hop type failed, see `dijkstra`'s
/// `convert_final_hop` parameter.
FinalHopConversion,
}

// Heavily adapted from https://github.com/samueltardieu/pathfinding/blob/master/src/directed/dijkstra.rs
// TODO: how2credit the repo (is that necessary?)?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmmm, good question. In general, the MIT and Apache licenses both require attribution, including of downstream projects. However, the Apache license only requires it if there is a file called "NOTICE" or any "copyright, patent, trademark, and attribution notices", which then must be provided downstream, and the MIT license only requires that "the above copyright notice be included", but the original repo doesn't actually include the MIT license anywhere, nor does it include any relevant notices as far as I can see, so there is no relevant "above copyright notice" to include, aside from the first paragraph of the MIT license, which we of course include as LICENSE-MIT. Thus, I think we can reasonably argue that a simple comment above this code indicating that it is adapted from (link) which is code Copyright Samuel Tardieu should suffice, which complies with the Apache "You must cause any modified files to carry prominent noticesstating that You changed the files" requirement.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1724 ended up not adapting this crate's implementation anymore (shoutout to Wikpedia)

/// Run Dijkstra's from `start` until `found_target` indicates that we've found the destination.
/// `successor_callback` must invoke the callback that it is provided on each of a given node's
/// next-hop peers. `convert_final_hop` may be used to convert an intermediate processing hop type
/// (`N`) to a final path hop type (`H`).
fn dijkstra<N, H, FN, FS, FC>(start: N, successor_callback: &mut FN, found_target: FS,
Comment thread
valentinewallace marked this conversation as resolved.
convert_final_hop: &mut FC) -> Result<Vec<H>, Error>
where N: Eq + Hash + Copy + Ord,
FN: FnMut(N, &mut FnMut(&N, u64)),
FS: Fn(&N) -> bool,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut to_see = BinaryHeap::new();
to_see.push((start, 0));
Comment thread
valentinewallace marked this conversation as resolved.
let mut parents: HashMap<N, (N, u64)> = HashMap::new();
parents.insert(start, (start, 0));

let mut target_reached = None;
while let Some((node, cost)) = to_see.pop() {
let &(_, c) = parents.get(&node).unwrap();
if found_target(&node) {
Comment thread
valentinewallace marked this conversation as resolved.
target_reached = Some(node);
break;
}
// We may have inserted a node several times into the binary heap if we found a better way to
// access it. Ensure that we are currently dealing with the best path and discard the others.
if cost > c {
Comment thread
valentinewallace marked this conversation as resolved.
continue;
}
successor_callback(node, &mut |successor, move_cost| {
let new_cost = cost + move_cost;
Comment thread
valentinewallace marked this conversation as resolved.
match parents.entry(*successor) {
hash_map::Entry::Vacant(e) => {
e.insert((node, new_cost));
to_see.push((*successor, new_cost));
}
hash_map::Entry::Occupied(mut e) => {
if e.get().1 > new_cost {
e.insert((node, new_cost));
to_see.push((e.get().0, new_cost));
}
}
}
});
}

match target_reached {
Some(t) => reverse_path(parents, t, convert_final_hop).map_err(|()| Error::FinalHopConversion),
None => Err(Error::PathNotFound)
}
}

// Errors if `convert_path_hop` fails.
fn reverse_path<N, H, FC>(parents: HashMap<N, (N, u64)>, start: N, convert_path_hop: &mut FC) -> Result<Vec<H>, ()>
where N: Eq + Hash + Copy + Ord,
FC: FnMut(&N) -> Result<H, ()>,
{
let mut path = vec![convert_path_hop(&start)?];
let mut curr = start;
Comment thread
valentinewallace marked this conversation as resolved.
loop {
if let Some((parent_node_id, _)) = parents.get(&curr) {
if parent_node_id != &curr {
path.push(convert_path_hop(parent_node_id)?);
curr = *parent_node_id;
} else { break; }
} else { break; }
}
path.reverse();
path.remove(0);
Ok(path)
}

#[cfg(test)]
mod tests {
use routing::test_utils;
use super::dijkstra;
use super::Error;

use sync::Arc;

fn expected(target: u8) -> Result<Vec<u8>, Error> {
match target {
0 => Ok(vec![0]),
1 => Ok(vec![]),
2 => Ok(vec![6, 2]),
3 => Ok(vec![0, 3]),
4 => Ok(vec![6, 4]),
5 => Ok(vec![6, 5]),
6 => Ok(vec![6]),
7 => Ok(vec![0, 3, 7]),
8 => Err(Error::PathNotFound),
_ => panic!("no such node"),
}
}

#[test]
fn dijkstra_ok() {
let successors_lookup : Vec<Vec<(u8, usize)>> = vec![
vec![(1, 7), (2, 7), (3, 6)],
vec![(0, 8), (6, 7)],
vec![(5, 7)],
vec![(7, 7)],
vec![(4, 2)],
vec![(1, 1)],
vec![(2, 5), (4, 5), (5, 2)],
vec![(5, 8)],
vec![],
];
let mut successors = |node, callback: &mut FnMut(&u8, u64)| {
for successor in &successors_lookup[node as usize] {
callback(&successor.0, 1);
}
};
for target in 0..9 {
assert_eq!(
dijkstra(1, &mut successors, |&node| node == target, &mut |&node| Ok(node)),
expected(target)
);
}
}

#[test]
fn one_hop() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[0], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 1);
assert!(path[0] == node_pks[0]);
}

#[test]
fn two_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let path = super::find_path(&our_id, &node_pks[2], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 2);
// See test_utils::build_graph ASCII graph, the first hop can be any of these
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
assert_eq!(path[1], node_pks[2]);
}

#[test]
fn three_hops() {
let (secp_ctx, network_graph, _, _, logger) = test_utils::build_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);

let mut path = super::find_path(&our_id, &node_pks[5], &network_graph, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 3);
assert!(path[0] == node_pks[1] || path[0] == node_pks[7] || path[0] == node_pks[0]);
path.remove(0);
assert_eq!(path, vec![node_pks[2], node_pks[5]]);
}

#[test]
fn long_path() {
let (secp_ctx, network, _, _, logger) = test_utils::build_line_graph();
let (_, our_id, _, node_pks) = test_utils::get_nodes(&secp_ctx);
let network_graph = network.read_only();

let path = super::find_path(&our_id, &node_pks[18], &network, None, Arc::clone(&logger)).unwrap();
assert_eq!(path.len(), 19);
}
}
2 changes: 2 additions & 0 deletions lightning/src/routing/mod.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,3 +12,5 @@
pub mod gossip;
pub mod router;
pub mod scoring;
#[cfg(test)]
pub(crate) mod test_utils;
Loading