Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Pad all sent TCP frames to the same size via ChunkedMessageQueue - #4446

Closed
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky
Closed

Pad all sent TCP frames to the same size via ChunkedMessageQueue#4446
tnull wants to merge 3 commits into
lightningdevkit:mainfrom
tnull:2026-02-chunky

Conversation

@tnull

@tnulltnull commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

In this PR we implement padding all sent TCP frames to the same size by ensuring all writes happen in chunks of length CHUNK_SIZE and any remainders are filled by adequately sized Ping messages.

This approach operating on the network-layer was discussed in the spec call and seems to be preferred by everybody. Hence it supersedes the approach taken in #4248.

TODO

  • Measurement/evaluation of introduced overhead

@ldk-reviews-bot

ldk-reviews-bot commented Feb 25, 2026

Copy link
Copy Markdown

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@tnulltnull self-assigned this Feb 25, 2026
@tnulltnull moved this to Goal: Merge in Weekly GoalsFeb 25, 2026
@tnulltnull changed the title Add ChunkedMessageQueue for message paddingPad all sent TCP frames to the same size via ChunkedMessageQueueFeb 25, 2026
@tnull
tnullforce-pushed the 2026-02-chunky branch 2 times, most recently from 48bb745 to 64e28dbCompareFebruary 25, 2026 15:08
@tnull

tnull commented Feb 25, 2026

Copy link
Copy Markdown
ContributorAuthor

Validated via Wireshark and LDK Node tests:

2026-02-25-161017_hyprshot

@codecov

codecovBot commented Feb 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.11307% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.98%. Comparing base (eb3980d) to head (d8de722).
⚠️ Report is 39 commits behind head on main.

Files with missing linesPatch %Lines
lightning/src/ln/peer_handler.rs95.63%9 Missing and 2 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #4446 +/- ##
==========================================
+ Coverage 85.88% 85.98% +0.10% 
==========================================
Files 159 159 Lines 104302 104850 +548 Branches 104302 104850 +548 ==========================================
+ Hits 89578 90157 +579 + Misses 12222 12188 -34 - Partials 2502 2505 +3 
FlagCoverage Δ
tests85.98% <96.11%> (+0.10%)⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add `BorrowedVecWriter`, `encrypt_with_header_0s_at`, `encrypt_message_into`,
and `encrypt_buffer_into` to `PeerChannelEncryptor` to support encrypting
messages directly into a shared buffer at arbitrary offsets.
Define `ChunkedMessageQueue` struct in `peer_handler` with a contiguous byte
buffer that will replace the current `pending_outbound_buffer` /
`gossip_broadcast_buffer` dual-queue system. The queue pads all outgoing
traffic into fixed-size `CHUNK_SIZE` chunks using Ping messages as filler,
so an observer sees only constant-size writes on the wire. Ping padding
uses `ponglen = 65533` to suppress Pong responses per BOLT-1.
Co-Authored-By: HAL 9000
Replace `pending_outbound_buffer`,
`pending_outbound_buffer_first_msg_offset`, and
`gossip_broadcast_buffer` fields in `Peer` with a single `message_queue:
ChunkedMessageQueue`.
Co-Authored-By: HAL 9000
Add five new tests:
- `test_chunked_message_queue_ping_padding`: verifies chunk alignment for
various message sizes
- `test_chunked_message_queue_small_remainder_overflow`: verifies the
two-Ping edge case when remainder < `MIN_ENCRYPTED_PING_SIZE`
- `test_chunked_message_queue_chunk_alignment`: verifies alignment after
encrypting multiple real messages
- `test_chunked_message_queue_buffer_compaction`: verifies `maybe_compact`
correctly drains sent bytes
- `test_chunked_message_queue_pending_msg_bytes_tracking`: verifies that
`pending_msg_bytes` tracks real message bytes and is unaffected by padding
Also extract a `get_test_encryptor` helper to reduce boilerplate across
the new tests.
Co-Authored-By: HAL 9000
Signed-off-by: Elias Rohrer <dev@tnull.de>
@ldk-reviews-bot

Copy link
Copy Markdown

🔔 1st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 2nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 3rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 4th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 5th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 6th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 7th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 8th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 9th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 10th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 11th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 12th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 27th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 28th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 29th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 30th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 31st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 32nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 33rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 34th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 35th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 36th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 37th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 38th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 39th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 40th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@tnull

tnull commented Jun 2, 2026

Copy link
Copy Markdown
ContributorAuthor

Relatedly, good that we bumped this to 0.4, should make the rollout more reasonable:

  • CLN #8893 implements the same general idea: pad with no-reply pings and write fixed-size encrypted chunks. CLN also sets TCP_NODELAY, added pcap/tshark-style tests, and later added compatibility gating plus a message-padding config option. Current CLN is conservative because older peers had bugs around no-reply pings.

  • LND had a BOLT 1 bug where ping(num_pong_bytes >= 65532) could disconnect the peer instead of being ignored. This was reported in lightningnetwork/lnd#10671 and fixed in lightningnetwork/lnd#10674. The fix is in the v0.21.0 line / release candidates. Older LND versions should be treated as unsafe for no-reply ping padding.

  • Eclair had the same class of issue. It was fixed by ACINQ/eclair#3278 and ACINQ/eclair#3284, released in Eclair v0.14.0. ACINQ’s lightning-kmp/Phoenix-side code also has the counterpart fix in ACINQ/
    lightning-kmp#855
    .

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 41st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 42nd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 43rd Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 44th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 45th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 46th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 47th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 48th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 49th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 50th Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

🔔 51st Reminder

Hey @TheBlueMatt! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

@ldk-reviews-bot

Copy link
Copy Markdown

Hi @tnull,

Thanks for your contributions to rust-lightning!

After too many struggles with bugs, outages, contributor bans, and, finally, a multi-week CI ban, the rust-lightning project is moving off of GitHub for day-to-day development.

You can still file issues and access the git tree here, but PRs will now take place exclusively at https://git.rust-bitcoin.org/. As such, this PR has been migrated to https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/4446

If you log in using GitHub (or otherwise link your GitHub account from https://git.rust-bitcoin.org/user/settings/security), ownership of your PRs, issues, and comments will automatically transfer. To push updates to this PR, you'll need to use git push git@gitea-ssh.bitcoin.ninja:lightningdevkit/rust-lightning YOUR_LOCAL_COMMIT_OR_BRANCH:tnull/2026-02-chunky. This may require a permissions change - if it doesn't work initially just leave a comment and we'll get you access.

@github-project-automationgithub-project-automationBot moved this from Goal: Merge to Done in Weekly GoalsJul 3, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@tnull@ldk-reviews-bot