Add database persistence benchmarks by benthecarman · Pull Request #915 · lightningdevkit/ldk-node

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman · 2026-05-31T23:06:09Z

Closes#908

Benchmark realistic payment and pending-payment persistence workloads
across filesystem, SQLite, and optional PostgreSQL stores. Use the async
KV-store APIs so the measured paths match the database interfaces used by
async persistence.

Also adds support for different db backends to the current payment benchmark.

Benchmark Results

Results on my computer with a Ryzen 7 5800XT 8-Core, 16-Thread CPU

Operations

Bench	filesystem	sqlite	postgres
`channel_open`	4.99 ms	5.38 ms	14.74 ms
`forwarding`	5.13 s	5.16 s	5.70 s
`channel_open_close`	5.07 s	5.20 s	5.08 s
`payments`	5.69 s	6.26 s	11.73 s

Startup

Scenario	filesystem	sqlite	postgres
`channels_1_payments_2`	10.37 ms	12.04 ms	57.57 ms
`channels_10_payments_2`	12.02 ms	16.68 ms	56.77 ms
`channels_100_payments_2`	18.37 ms	50.61 ms	62.03 ms
`channels_100_payments_1000`	17.77 ms	70.26 ms	61.86 ms

Database Hot Path

Bench	filesystem	sqlite	postgres
`channel_open_like`	0.26 ms	0.46 ms	7.54 ms
`forwarding_25_like`	5.00 ms	10.56 ms	124.89 ms

ldk-reviews-bot · 2026-05-31T23:06:12Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

tnull · 2026-06-03T12:32:12Z

Feel free to rebase on top of #919 and let me know if you see any differences in the benchmarks.

benthecarman · 2026-06-03T16:29:02Z

codex summary:

Benchmark Results

Payment Benchmarks

Backend	Previous	Current	Change
filesystem	6.2475 s	6.2138 s	slightly faster
sqlite	6.7378 s	6.6388 s	faster
postgres	17.669 s	17.489 s	slightly faster; Criterion reported no significant change

Lower is better. Delta compares the async-branch rerun against the previous saved Criterion results.

Bench	Backend	Previous	Async rerun	Delta
single write new	filesystem	1.844 ms	1.847 ms	+0.2%
single write new	sqlite	2.673 ms	2.658 ms	-0.6%
single write new	postgres	636.0 us	630.1 us	-0.9%
single write existing	filesystem	1.860 ms	1.881 ms	+1.1%
single write existing	sqlite	22.1 us	22.5 us	+2.0%
single write existing	postgres	638.1 us	624.9 us	-2.1%
single read	filesystem	14.7 us	15.1 us	+2.2%
single read	sqlite	17.0 us	17.3 us	+1.7%
single read	postgres	145.0 us	146.2 us	+0.8%
single remove	filesystem	908.8 us	920.0 us	+1.2%
single remove	sqlite	2.842 ms	2.728 ms	-4.0%
single remove	postgres	1.445 ms	1.470 ms	+1.8%
warm insert 100	filesystem	185.626 ms	185.919 ms	+0.2%
warm insert 100	sqlite	276.593 ms	269.530 ms	-2.6%
warm insert 100	postgres	63.417 ms	62.791 ms	-1.0%
concurrent distinct 100	filesystem	10.964 ms	10.909 ms	-0.5%
concurrent distinct 100	sqlite	272.638 ms	261.112 ms	-4.2%
concurrent distinct 100	postgres	9.177 ms	9.230 ms	+0.6%
concurrent same-key 100	filesystem	11.495 ms	13.050 ms	+13.5%
concurrent same-key 100	sqlite	249.663 ms	191.029 ms	-23.5%
concurrent same-key 100	postgres	58.739 ms	56.400 ms	-4.0%
cold insert 100	filesystem	185.829 ms	183.843 ms	-1.1%
cold insert 100	sqlite	266.497 ms	307.448 ms	+15.4%
cold insert 100	postgres	74.440 ms	72.031 ms	-3.2%
cold update 100	filesystem	188.561 ms	186.808 ms	-0.9%
cold update 100	sqlite	247.062 ms	246.978 ms	-0.0%
cold update 100	postgres	78.208 ms	74.185 ms	-5.1%
reload 100	filesystem	2.995 ms	3.002 ms	+0.2%
reload 100	sqlite	3.532 ms	3.499 ms	-0.9%
reload 100	postgres	3.237 ms	3.152 ms	-2.7%
first page from 10k	filesystem	25.290 ms	25.090 ms	-0.8%
first page from 10k	sqlite	29.8 us	24.8 us	-17.0%
first page from 10k	postgres	184.5 us	182.3 us	-1.2%
second page from 10k	filesystem	50.665 ms	50.422 ms	-0.5%
second page from 10k	sqlite	66.5 us	50.8 us	-23.7%
second page from 10k	postgres	376.7 us	372.7 us	-1.1%
lifecycle insert-update-read	filesystem	6.786 ms	3.736 ms	-44.9%
lifecycle insert-update-read	sqlite	9.866 ms	5.216 ms	-47.1%
lifecycle insert-update-read	postgres	2.533 ms	1.617 ms	-36.2%
pending insert 100	filesystem	273.515 ms	183.380 ms	-33.0%
pending insert 100	sqlite	333.859 ms	262.138 ms	-21.5%
pending insert 100	postgres	75.244 ms	75.999 ms	+1.0%
pending update 100	filesystem	197.544 ms	187.656 ms	-5.0%
pending update 100	sqlite	257.055 ms	264.137 ms	+2.8%
pending update 100	postgres	75.756 ms	63.715 ms	-15.9%

Takeaways

The async branch materially improves the lifecycle workload across all stores: filesystem improved by 44.9%, sqlite by 47.1%, and Postgres by 36.2%.
Postgres pending-payment updates improved meaningfully, dropping from 75.756 ms to 63.715 ms, a 15.9% reduction.
Most single-operation Postgres results are roughly unchanged, with small improvements on writes and small regressions/noise on reads/removes.
Postgres cold payment-store writes improved modestly: inserts improved by 3.2% and updates by 5.1%.
SQLite saw large improvements in paginated reads and same-key concurrent writes, but cold insert regressed by 15.4%, so that path is worth a closer look.
Payment benchmarks are still not trustworthy for comparison because the workload does not complete cleanly and times out waiting for HTLC slots.

joostjager · 2026-06-04T08:52:46Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

tnull · 2026-06-04T08:56:05Z

It would have been nice if async persistence would show a clear performance win here. The store-level smoke tests are useful, but the closest thing to real node load seems to be the payment benchmark, and that does not show a meaningful improvement, putting aside that it is marked as not reliable yet.

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

benthecarman · 2026-06-04T16:12:19Z

Hmm, note that 'async persistence' means pre and post #919 AFAIU? I.e., it does not refer to LDK's async persistence itself, which we switched to pre-#919.

yeah that is pre/post 919

benthecarman · 2026-06-04T16:27:28Z

Does this tell us much about high-load node performance in practice? If not, what would be the next useful benchmark?

Added a few more with start up time, forwarding payments, and channel opens. But working on making these more robust

benthecarman · 2026-06-09T15:25:56Z

Since #919 is merged marking this ready for review.

Hopefully we get CI re-enabled because I am not sure how much these changes effect how long the bench CI job takes. The whole suite takes awhile so I tried trimming it down for CI. That subset takes a few minutes on my computer but unsure with the CI computers

ldk-reviews-bot · 2026-06-11T15:26:09Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-13T15:27:10Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-15T15:28:01Z

🔔 3rd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-17T15:28:39Z

🔔 4th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-20T00:00:34Z

🔔 5th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2026-06-22T00:01:45Z

🔔 6th Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

I think when we add this we should also document somewhere how to run benchmarks as guidance for contributors.

Besides that, current PR is a quite sloppy. Please clean up before we can move forward here (and split out MigratebleKVStore impls, for instance).

tnull · 2026-06-23T11:41:28Z


 	fn test_connection_string() -> String {
- std::env::var("TEST_POSTGRES_URL")
+ dotenvy::dotenv().ok();


What is dotenvy? This commit wont compile.

tnull · 2026-06-23T11:42:23Z

 proptest = "1.0.0"
 regex = "1.5.6"
 criterion = { version = "0.7.0", features = ["async_tokio"] }
+dotenvy = "0.15"


I don't think we should take a dependency for this?

was simple way to read .env files, made doing test loops faster instead of always doing TEST_POSTGRES_URL=blah cargo test but can remove if not wanted. Figured was fine because just dev-dep

yeah, we're generally more lax about adding dev-dependencies, but this seems very unnecessary as reading env variables can be done via std, and reading files should be a ~5 line helper?

tnull · 2026-06-23T11:43:25Z

Can all benchmark mark-related files rather live in the benches sub-folder? Not sure we want to take on this huge file that's only boilerplate/utilities in the main src dir..

tnull · 2026-06-23T11:46:59Z

+
 	// Set up two nodes. Because this is slow, we reuse the same nodes for each sample.
 	let (bitcoind, electrsd) = setup_bitcoind_and_electrsd();
-	let chain_source = random_chain_source(&bitcoind, &electrsd);


Please don't introduce that much churn for code you just added in the last commit.

tnull · 2026-06-23T11:48:11Z

 	panic!("Timed out waiting for forwarding path readiness");
 }

+async fn open_channel(


This is just a duplicate of the test util? Why copy this over?

Needed a separate one because the test utils one waits for the tx to be in the mempool. Didn't want to that to leak into the benchmark. Refactored to use a shared util

joostjager · 2026-06-30T12:19:55Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

joostjager · 2026-07-01T14:20:31Z

Small note: I’m experimenting with this PR locally and extending the benchmarks to measure single-channel payment performance with many open but unused channels, up to 5000. Since deferred writing makes channel updates scale with the total number of channels, I’m curious to see how much impact that has.

https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/issues/4768

AI-Assisted-By: OpenAI Codex

Benchmark realistic payment and pending-payment persistence workloads across filesystem, SQLite, and optional PostgreSQL stores. Use the async KV-store APIs so the measured paths match the database interfaces used by async persistence. AI-Assisted: Codex

Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman mentioned this pull request May 31, 2026
Add Postgres database #863
Merged

benthecarman force-pushed the db-bench branch 3 times, most recently from 5f75dd9 to bc97c3aCompare June 2, 2026 22:19

benthecarman force-pushed the db-bench branch from bc97c3a to fbc6fb7Compare June 3, 2026 16:41

benthecarman force-pushed the db-bench branch from fbc6fb7 to c721923Compare June 4, 2026 16:25

benthecarman force-pushed the db-bench branch 3 times, most recently from fedeaf4 to c8f415fCompare June 9, 2026 15:11

benthecarman requested a review from tnull June 9, 2026 15:25

benthecarman marked this pull request as ready for review June 9, 2026 15:25

tnull reviewed Jun 23, 2026
View reviewed changes

Comment threadsrc/io/postgres_store/mod.rs Outdated
Comment threadbenches/operations.rs Outdated
Comment threadsrc/bench.rs Outdated

benthecarman mentioned this pull request Jun 23, 2026
Implement MigratableKVStore for all KV Stores #945
Merged

benthecarman force-pushed the db-bench branch from c8f415f to 2196187Compare July 1, 2026 00:48

benthecarman force-pushed the db-bench branch from 2196187 to df1baf8Compare July 1, 2026 01:36

benthecarman force-pushed the db-bench branch from df1baf8 to db9098eCompare July 16, 2026 17:58

Centralize PostgreSQL test URL env var
670630d
AI-Assisted-By: OpenAI Codex

benthecarman force-pushed the db-bench branch from db9098e to a20d295Compare July 20, 2026 17:57

benthecarman requested a review from tnull July 20, 2026 17:58

tnull reviewed Jul 21, 2026
View reviewed changes

Comment threadbenches/operations.rs
Comment threadbenches/operations.rs Outdated
Comment threadbenches/common/db_store.rs Outdated

benthecarman added 5 commits July 22, 2026 14:25

Benchmark payments across stores
36d53b5
Run the existing payments benchmark once per configured store backend so filesystem, SQLite, and optional PostgreSQL results are reported under the same payment flow. AI-assisted-by: OpenAI Codex

Add forwarding operations benchmark
9348e7b
Add an operations bench target with a forwarding benchmark that compares sqlite, filesystem, and postgres stores over a settled multi-hop payment. AI-assisted-by: OpenAI Codex

Add channel open operations benchmark
e379273
Add a channel-open benchmark that measures the open_channel call while leaving chain confirmation cleanup outside the timed section. AI-assisted-by: OpenAI Codex

Add seeded startup operations benchmark
1299728
Add a startup benchmark that restarts a node whose store already contains channel and payment data, so startup cost reflects persisted node state. AI-assisted-by: OpenAI Codex

benthecarman force-pushed the db-bench branch from a20d295 to 1299728Compare July 22, 2026 19:25

benthecarman requested a review from tnull July 22, 2026 19:25