Add a simple send-funds benchmark in channelmanager

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.

[TheBlueMatt]

I saw https://twitter.com/bottlepay/status/1376918214249218049 and was curious, so I benchmarked sends.

We're doing pretty good - on my local machine somewhere in the neighborhood of 5-6ms for two full back-and-forth sends, ignoring all IO latency.

[codecov]

Codecov Report

Merging #860 (1424b28) into main (6fcac8b) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 1424b28 differs from pull request most recent head 8a9f0b8. Consider uploading reports for the commit 8a9f0b8 to get more accurate results

@@ Coverage Diff @@## main #860 +/- ##
==========================================
+ Coverage 90.60% 90.62% +0.02% 
==========================================
Files 51 51 Lines 27193 27195 +2 ==========================================
+ Hits 24637 24646 +9 + Misses 2556 2549 -7 

Impacted Files	Coverage Δ
lightning-persister/src/lib.rs	95.61% <ø> (ø)
lightning/src/ln/functional_test_utils.rs	95.21% <ø> (ø)
lightning/src/ln/channelmanager.rs	83.23% <100.00%> (+0.01%)	⬆️
lightning/src/util/test_utils.rs	82.95% <100.00%> (ø)
lightning/src/ln/functional_tests.rs	97.02% <0.00%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fcac8b...8a9f0b8. Read the comment docs.

[TheBlueMatt]

Our naive persister...not so much, its currently showing around 100ms for a full back-and-forth send, at least after some time as the ChannelMonitor has been allowed to grow to some size.

[TheBlueMatt]

Note that if you swap to flushing the ChannelMonitorUpdates instead (which we should do eventually, but not today), you get a much more reasonable 69ms (+/-35).

[TheBlueMatt]

If you go even further and cut out the duplicate fsync and file rename for ChannelMonitorUpdate writes we get down to 19ms-per-two-sends, which is much more reasonable. Branch is at https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist if anyone wants to go implement the reading logic for it so we can take it https://git.bitcoin.ninja/index.cgi?p=rust-lightning;a=shortlog;h=refs/heads/2021-03-optimize-chanmon-persist

[arik-so]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

[valentinewallace]

Approve mod CI warning :)

[valentinewallace]

I'd be curious to compare this to an Arc'd version (i.e. using arcs instead of refs)

[TheBlueMatt]

I'd be astounded if it made a difference - we aren't actually cloning the Arcs anywhere that I know of here and our runtime is dominated by cryptographic operations. Non-Arc objects are nice mostly because we'd like to eventually support non-threaded applications, so we don't need trait implementations to implement Sync+Send.

[TheBlueMatt]

Only difference from previous ACKs is fixed the compile warning:

$ git diff-tree -U2 79b3e5d6dfabf1e4fd3fb79589b6afa7d4a33d0d 8a9f0b8cedaf63a921456a27dea9ecba88f3ebcd
diff --git a/lightning/src/ln/functional_test_utils.rs b/lightning/src/ln/functional_test_utils.rs
index 8021a4ace..3c641dd74 100644
--- a/lightning/src/ln/functional_test_utils.rs
+++ b/lightning/src/ln/functional_test_utils.rs
@@ -867,4 +867,5 @@ macro_rules! expect_pending_htlcs_forwardable {
}
+#[cfg(any(test, feature = "unstable"))]
macro_rules! expect_payment_received {
($node: expr, $expected_payment_hash: expr, $expected_recv_value: expr) => {
$

[TheBlueMatt]

Do you think it might make sense to also make a slightly more complicated benchmark with one routing node in between?

Hmm, there should be almost no difference there, forwarding vs initial send should be completely dwarfed by cryptographic operations. If you don't mind, I'd prefer to just take this as-is and we can follow up with more benchmarking if we think its merited later.