implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

implement fromstr trait to netaddress - #2134

Merged
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress
Sep 7, 2023
Merged

implement fromstr trait to netaddress#2134
tnull merged 4 commits into
lightningdevkit:mainfrom
jbesraa:add_fromstr_to_netaddress

Conversation

@jbesraa

@jbesraajbesraa commented Mar 28, 2023

Copy link
Copy Markdown
Contributor

resolves#2056

@tnull
tnull self-requested a review March 28, 2023 11:24
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt@tnull any recommendation on how to decode the onionv3 address? I could potentially use https://crates.io/crates/base32 but im not sure about adding a new package just for that?

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Hmm, yea, that's a good question - I took a brief glance at the base32 code, it looks fine, though could use one or two trivial changes (and a fuzzer, it has an obvious panic in the decode!). Luckily its the same license so we'd have to add a notice that it includes code which is "Copyright (c) 2015 The base32 Developers" but that's nbd. We should pull it in by coping their lib.rs, though, not taking the dependency, there's no need for a dep here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

made some changes(main functionality is implemented and tests are passing) to the code and updated the top comment accordingly

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/Cargo.toml Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from 362a9b4 to d807a53CompareApril 8, 2023 17:02
Comment threadlightning/src/util/base32.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 25250ca to 3e46d4cCompareApril 23, 2023 16:21
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

i can see the fuzzer failing locally as well, but not sure what to make from the error message
Screenshot from 2023-04-24 16-10-24

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from b6f5ae6 to 471c5d7CompareMay 1, 2023 09:55
use crate::prelude::*;

/// Alphabet used for encoding and decoding.
#[derive(Copy, Clone)]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a) we can drop the crockford encoding, we don't use it. (b) we should move the zbase32 stuff we already have to here - its the same code anyway, but just a different alphabet.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 2 times, most recently from a092195 to cf72e89CompareMay 11, 2023 08:16
@TheBlueMatt

TheBlueMatt commented May 11, 2023

Copy link
Copy Markdown
Collaborator

Let me know when you're ready for another look here, looks like you're in the midst of merging the two base32s.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from cf72e89 to ea1b296CompareMay 12, 2023 12:09
@codecov-commenter

codecov-commenter commented May 12, 2023

Copy link
Copy Markdown

Codecov Report

Patch coverage: 93.17% and project coverage change: +0.01% 🎉

Comparison is base (e9d9711) 90.58% compared to head (529a09f) 90.60%.

❗ Current head 529a09f differs from pull request most recent head 9a8b7e3. Consider uploading reports for the commit 9a8b7e3 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@ Coverage Diff @@## main #2134 +/- ##
==========================================
+ Coverage 90.58% 90.60% +0.01% 
==========================================
Files 110 110 Lines 57526 57555 +29 Branches 57526 57555 +29 ==========================================
+ Hits 52112 52149 +37 + Misses 5414 5406 -8 
Files ChangedCoverage Δ
lightning/src/ln/msgs.rs86.20% <89.77%> (+0.38%)⬆️
lightning/src/util/message_signing.rs92.30% <90.47%> (ø)
lightning/src/util/base32.rs96.63% <96.63%> (ø)

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@TheBlueMatt can u please give this another round of review? also, any tips on how to debug the fuzzer locally would be much appreciated

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
Comment threadlightning/src/ln/msgs.rs Outdated
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Re: fuzzer reproduction, the fuzz/README.md file has a section on ## A fuzz test failed on Travis, what do I do?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 359e099 to d74c87fCompareJune 28, 2023 22:04
@jbesraa

jbesraa commented Jun 28, 2023

Copy link
Copy Markdown
ContributorAuthor

apologies for the slow execution, life got in the middle (:
did some improvements to from_str, will handle the base32 code and the fuzzer in the next days hopefully!

...and ill cleanup commits

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, really just one nit left!

Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/util/base32.rs Outdated
Comment threadlightning/src/ln/msgs.rs
@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 56b168a to dbded6eCompareAugust 26, 2023 11:44
TheBlueMatt
TheBlueMatt previously approved these changes Aug 27, 2023

@TheBlueMattTheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits, but nothing blocking.

Comment threadlightning/src/util/base32.rs Outdated
];

const RFC4648_TEST_VECTORS: &[(&[u8], &[u8])] = &[
(b"", b""),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data type for the second element in the tuple should be a str, which you're currently doing in the tests with std::str::from_utf8...unwrap.

Comment threadlightning/src/util/base32.rs Outdated

for (_, encoded) in RFC4648_TEST_VECTORS {
let decoded = &Alphabet::RFC4648 { padding: true }.decode(std::str::from_utf8(encoded).unwrap()).unwrap();
assert_eq!(&Alphabet::RFC4648 { padding: true }.encode(decoded).as_bytes(), encoded);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is redundant - we already test that if we decode we get the expected value and if we encode we get the expected value, there's no need for an additional round-trip test.

channel_id: [2; 32],
scriptpubkey:
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }
if script_type == 1 { Address::p2pkh(&::bitcoin::PublicKey{compressed: true, inner: pubkey_1}, Network::Testnet).script_pubkey() }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the alignment here and below?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this was a space, now it looks like this:
Screenshot from 2023-08-27 21-37-37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it was a space to align the opening { nicely no matter the users' configured tabstop.

Comment threadlightning/src/ln/msgs.rs Outdated
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
NetAddressParseError::SocketAddrParse => write!(f, "Socket address (IPv4/IPv6) parsing error"),
NetAddressParseError::InvalidInput => write!(f, "{}", "Invalid input format. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the , "{}" part.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Oh, if you push again, please fix the commit title on the last commit - commit description text must include a blank line between the title and the first non-title line (otherwise git treats it as a really long title, which breaks lots of things).

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Also, the description in the first commit doesn't tell me anything " restructure encode function restructure decode function" doesn't tell me what the commit is doing or why its doing that, though the title itself is pretty clear. Commit descriptions shouldn't include the history of that commit, by the time we merge it we don't care too much about the history of the PR's review.

@tnulltnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good from my side, but have yet to go over the base32 logic stuff one last time.

Comment threadlightning/src/util/base32.rs
Comment threadlightning/src/util/message_signing.rs
Comment threadlightning/src/ln/msgs.rs
@tnull

tnull commented Aug 29, 2023

Copy link
Copy Markdown
Contributor

Also needs a minor rebase now it seems.

@TheBlueMatt

Copy link
Copy Markdown
Collaborator

Any update here, would really love to land this for 0.0.117 but its gonna get over the line in the next week or two.

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch 4 times, most recently from c9750ce to 03e4ac8CompareSeptember 4, 2023 15:29
@jbesraa

Copy link
Copy Markdown
ContributorAuthor

@tnull split the indentation to a separate commit and added short docs for the numbers

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, mod some nits/minor comments.

Comment threadlightning/src/util/mod.rs Outdated
#[cfg(any(test, feature = "_test_utils"))]
pub mod test_channel_signer;

pub mod enforcing_trait_impls;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was removed on main and we shouldn't re-add the module here.

}
}
}
} No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't remove the trailing newline.

Comment threadlightning/src/util/base32.rs Outdated
impl Alphabet {
/// Encode bytes into a base32 string.
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string, which is equal to the original data length * 8 / 5, rounded up. The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might want to wrap the comment lines also.

Comment threadlightning/src/util/base32.rs Outdated
if *padding {
let len = ret.len();
let num_extra = len - output_length;
for i in 1..num_extra + 1 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just the same as the following, which is a lot less confusing?:

for i in output_length..len {
ret[i] = b'=';}

Comment threadlightning/src/util/base32.rs Outdated
// If the string has more characters than are required to alphabet_encode the number of bytes
// decodable, treat the string as invalid.
match data.len() % 8 { 1|3|6 => return Err(()), _ => {} }
let mut ret = Self::decode_data(data, alphabet)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Doesn't need to be mut.

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 4) / 5 * 8;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(5) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this additional let binding? Why not just use the mut buf below?

Comment threadlightning/src/util/base32.rs Outdated
let cap = (data.len() + 7) / 8 * 5;
let mut ret = Vec::with_capacity(cap);
for chunk in data.chunks(8) {
let buf = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why do we need this additional let binding?

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 03e4ac8 to 53db4caCompareSeptember 5, 2023 13:23
@jbesraa

ghost commented Sep 5, 2023

Copy link
Copy Markdown
ContributorAuthor

Fixed Up:

  • Wrapped comments into multiple lines
  • Removed unnecessary mut and let bindings
  • Improved readability of if
  • Reverted EOL removal

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 53db4ca to 5f354faCompareSeptember 5, 2023 14:16
Comment threadlightning/src/util/base32.rs Outdated
pub fn encode(&self, data: &[u8]) -> String {
// This line calculates the length of the output string,
// which is equal to the original data length * 8 / 5, rounded up.
// The + 4 at the end is to account for the extra padding characters that may be added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +4 is the "rounded up" part - rounding up with integer division is generally done by doing (numerator + divisor - 1) / divisor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

1pubfn encode(&self,data:&[u8]) -> String{
| 45// output_length is calculated as follows:
| 1// / 5 divides the data length by the number of bits per chunk (5),
| 2// * 8 multiplies the result by the number of characters per chunk (8).
| 3// + 4 rounds up to the nearest character.4let output_length = (data.len()*8 + 4) / 5;
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment threadlightning/src/util/base32.rs Outdated

/// Encode a byte slice into a base32 string.
fn encode_data(data: &[u8], alphabet: &'static [u8]) -> Vec<u8> {
// The + 4 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +5 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

102fn encode_data(data:&[u8],alphabet:&'static[u8]) -> Vec<u8>{
| 1// cap is calculated as follows:
| 2// / 5 divides the data length by the number of bits per chunk (5),
| 3// * 8 multiplies the result by the number of characters per chunk (8).
| 4// + 4 rounds up to the nearest character.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated


fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the +7 is to round up.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about this?

129fn decode_data(data:&[u8],alphabet:[i8;43]) -> Result<Vec<u8>,()>{
| 1// cap is calculated as follows:
| 2// / 8 divides the data length by the number of characters per chunk (8),
| 3// * 5 multiplies the result by the number of bits per chunk (5),
| 4// + 7 rounds up to the nearest byte.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Comment threadlightning/src/util/base32.rs Outdated

fn decode_data(data: &[u8], alphabet: [i8; 43]) -> Result<Vec<u8>, ()> {
// The + 7 at the end is to account for the padding characters that may be needed,
// The / 8 divides the Base64 string length by the number of characters per chunk (8),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here, and a few places, you have excess whitespace at the end of your line. A local git show should highlight these depending on your terminal settings.

Comment threadlightning/src/ln/msgs.rs Outdated
Comment on lines +947 to +949
match {
addr
} {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match{
addr
}{
match addr {

@jbesraa
jbesraaforce-pushed the add_fromstr_to_netaddress branch from 5f354fa to 4f45cdcCompareSeptember 6, 2023 16:28
@tnull
tnull merged commit b5e9594 into lightningdevkit:mainSep 7, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement FromStr for NetAddress

4 participants

@jbesraa@TheBlueMatt@codecov-commenter@tnull