Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Cbf add broadcast - #30

Closed
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast
Closed

Cbf add broadcast#30
randomlogin wants to merge 12 commits into
febyeji:cbf-chain-source-cleanupfrom
randomlogin:cbf-add-broadcast

Conversation

@randomlogin

Copy link
Copy Markdown
Collaborator

This PR lives ontop of #29

randomloginand others added 10 commits June 3, 2026 01:06
Add stub methods/functions, add basic build and start of the CBF chain
source as well as basic struct containing the fields which undoubtedtly
are needed.
Previously tests assumed that the chain source of the lightning node and
is node which mines. This is not the case with CBF chain source which
needs to wait until after mining a new block a new tips propagates to
it.
`wait_for_block` is made to return a new height and a new function
`wait_for_node_tip` is added which waits until the given height is
processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait
ans add register_script method as well as implementation of registered
scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need
to take any action (e.g. apply block). These actions are sent to a new
abstraction — `BlockApplicator` which holds `ChainListener`
(which in turn has wallets and can apply blocks / filtered blocks). This
`BlockApplicator` has to have a receiver of a channel (and
`process_kyoto_events` has to have a sender to this channel. Thus we
cannot create them in `new`, because we would own them at
`CbfChainSource`, so they are created in `start`.
Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`:
- ConnectFull,
- ConnectFiltered,
- Disconnect,
- Synced
Now `process_kyoto_events` reacts to an event from kyoto and sends a
`ChainOp` to listener (`BlockApplicator`).
Note that on a filter with no match we still need to apply header and we
need double check that header in the canonical chain is the relevant
one and has not been reorged. Right now it is left as a todo, because of
an upstream PR.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source.
1) esplora
2) electrum.
For electrum we altered the existing code from the electrum chain
source to make it reusable for the CBF node We change runtime argument
to `start()` to be a field in `CbfChainSource` struct, because
electrum fee source explictely needs it.
3) CBF native fee source.
It downloads blocks and calculates fees based on previous blocks. We
maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For
confirmation target we use estimation vie percentil of previous
blocks' fees. Also if we download a block on a matched block filter,
we insert it in the cache to avoid re-downloading.
As the block download might be slow, we don't fail the fee estimation,
but rather use fallback fees. This is especially relevant to the very
first call of `update_fee_rate_estimates` during the start.
We add helper pure functions in a new file `src/util.rs`.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@randomlogin
randomlogin requested a review from febyejiJune 9, 2026 13:56
Comment threadsrc/chain/cbf.rs
},
};

match Package::from_vec(package.clone()) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spotted this from codex PR review , and it may be worth checking:

This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.

But it is what all other chain sources do:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}

bitcoind:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}

and esplora:

	pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...

So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can add a PR to ldk-node about this. Let's keep this way for now!

randomloginand others added 2 commits June 11, 2026 23:41
Onchain wallet's scripts are pulled each time we receive `IndexedFilter`
event. That way we rely on a single source of truth (wallet) instead of
having two overlapping script sets.
Now we pass a reference to the `onchain_wallet` to make possible get all
revealed scripts from it.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
@febyeji

Copy link
Copy Markdown
Owner

Applied this contribution to cbf-chain-source-cleanup as commit 9d36b17 (cbf: implement package broadcasting), preserving Alexander Shevtsov as the commit author. Closing this PR instead of merging per permission to incorporate the work with attribution.

AI-assisted-by: OpenAI Codex

@febyejifebyeji closed this Jun 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@randomlogin@febyeji