Generate local signatures with additional randomness by wpaulino · Pull Request #2205 · lightningdevkit/rust-lightning

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino · 2023-04-19T23:37:42Z

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

TheBlueMatt · 2023-04-20T02:29:55Z

-	/// because we are about to broadcast a holder transaction.
-	pub fn sign<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {
+	/// Signs the counterparty's commitment transaction.
+	pub fn sign_counterparty_commitment<T: secp256k1::Signing>(&self, funding_key: &SecretKey, funding_redeemscript: &Script, channel_value_satoshis: u64, secp_ctx: &Secp256k1<T>) -> Signature {


I mean, this is fine, but is there a reason to break this out and treat them separately?

We have a few test assertions that rely on the same commitment_signed being resent after a channel_reestablish. This is also the case for closing_signed signatures. We could remove those assertions I guess, since the protocol does allow you to retransmit a different signature after a reconnection.

Ah, okay, yea, that's fine then, I think, I just wanted to check that there's some reason.

Sorry, fine as in we can remove the assertions or just leave them as is?

Let's leave it as-is.

codecov-commenter · 2023-04-20T03:34:05Z

Codecov Report

❌ Patch coverage is 92.06349% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.48%. Comparing base (8d50c91) to head (86531e5).
⚠️ Report is 6122 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/chain/keysinterface.rs	92.68%	3 Missing ⚠️
lightning/src/util/test_utils.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2205 +/- ##
==========================================
+ Coverage 91.38% 91.48% +0.10% 
==========================================
Files 102 104 +2 Lines 50832 51176 +344 Branches 50832 51176 +344 ==========================================
+ Hits 46451 46820 +369 + Misses 4381 4356 -25

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Previously, our local signatures would always be deterministic, whether we'd grind for low R value signatures or not. For peers supporting SegWit, Bitcoin Core will generally use a transaction's witness-txid, as opposed to its txid, to advertise transactions. Therefore, to ensure a transaction has the best chance to propagate across node mempools in the network, each of its broadcast attempts should have a unique/distinct witness-txid, which we can achieve by introducing random nonce data when generating local signatures, such that they are no longer deterministic.

Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes" API Updates =========== * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107). * Attempting to claim an un-expired payment on a channel which has closed no longer fails. The expiry time of payments is exposed via `PaymentClaimable::claim_deadline` (lightningdevkit#2148). * `payment_metadata` is now supported in `Invoice` deserialization, sending, and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127). * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142). * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989). * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161). * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138). * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085). * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098). * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136). * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079). * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094). * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023). * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206). Backwards Compatibility ======================= * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139). * `Event::PaymentPathFailed::retry` was removed and will always be `None` for payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063). * `Route`s and `PaymentParameters` with blinded path information will not be readable on prior versions of LDK. Such objects are not currently constructed by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146). * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is in general unsupported, and included here only for completeness. Bug Fixes ========= * Fixed a case where `process_events_async` may `poll` a `Future` which has already completed (lightningdevkit#2081). * Fixed deserialization of `u16` arrays. This bug may have previously corrupted the historical buckets in a `ProbabilisticScorer`. Users relying on the historical buckets may wish to wipe their scorer on upgrade to remove corrupt data rather than waiting on it to decay (lightningdevkit#2191). * The `process_events_async` task is now `Send` and can thus be polled on a multi-threaded runtime (lightningdevkit#2199). * Fixed a missing macro export causing `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091). * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187) * Fix an issue where the `background-processor` would not wake when a `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090). * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145). * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather than `find_route`, as was intended and described in the API (lightningdevkit#2092). * Ensure `process_events_async` always exits if any sleep future returns true, not just if all sleep futures repeatedly return true (lightningdevkit#2145). * `channel_update` messages no longer set the disable bit unless the peer has been disconnected for some time. This should resolve cases where channels are disabled for extended periods of time (lightningdevkit#2198). * We no longer remove CLN nodes from the network graph for violating the BOLT spec in some cases after failing to pay through them (lightningdevkit#2220). * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172). * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114) * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158). Security ======== * Transaction re-broadcasting is now substantially more aggressive, including a new regular rebroadcast feature called on a timer from the `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`. This should substantially increase transaction confirmation reliability without relying on downstream `TransactionBroadcaster` implementations for rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208). * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a privacy vulnerability which allows an intermediate node on the path to discover the final destination for a payment (lightningdevkit#2062). In total, this release features 110 files changed, 11928 insertions, 6368 deletions in 215 commits from 21 authors, in alphabetical order: * Advait * Alan Cohen * Alec Chen * Allan Douglas R. de Oliveira * Arik Sosman * Elias Rohrer * Evan Feenstra * Jeffrey Czyz * John Cantrell * Lucas Soriano del Pino * Marc Tyndel * Matt Corallo * Paul Miller * Steven * Steven Williamson * Steven Zhao * Tony Giorgio * Valentine Wallace * Wilmer Paulino * benthecarman * munjesi

wpaulino force-pushed the sign-ecdsa-with-noncedata branch 2 times, most recently from b8d85c9 to 099d875Compare April 20, 2023 02:23

TheBlueMatt reviewed Apr 20, 2023
View reviewed changes

TheBlueMatt assigned TheBlueMatt and arik-so Apr 20, 2023

wpaulino added this to the 0.0.115 milestone Apr 20, 2023

wpaulino added 4 commits April 20, 2023 12:14

Implement EntropySource for InMemorySigner
2c5bd1c
This allows the `InMemorySigner` to produce its own randomness, which we plan to use when generating signatures in future work. We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to implement it manually.

Sign gossip messages without grinding low R signatures
16edbd4
Gossip messages always use signatures in their compact form, so grinding for low R signatures is unnecessary.

Use new feature to gate test vectors behind
86531e5
To match the local signatures found in test vectors, we must make sure we don't use any additional randomess when generating signatures, as we'll arrive at a different signature otherwise.

wpaulino force-pushed the sign-ecdsa-with-noncedata branch from 099d875 to 86531e5Compare April 20, 2023 19:14

arik-so approved these changes Apr 20, 2023
View reviewed changes

TheBlueMatt approved these changes Apr 20, 2023
View reviewed changes

TheBlueMatt merged commit 9d5adfc into lightningdevkit:mainApr 20, 2023

wpaulino deleted the sign-ecdsa-with-noncedata branch April 20, 2023 22:18