Add configurable logging system by enigbe · Pull Request #407 · lightningdevkit/ldk-node

enigbe · 2024-11-20T16:01:23Z

Overview

This PR introduces a flexible logging system for LDK Node by implementing a LogWriter interface that supports writing logs to different destinations.

What this PR does

Introduces a LogWriter interface, allowing Writer variants to handle log output destinations. The supported Writer variants can now:
- Write logs to the filesystem,
- Forward to a log implementer,
- Relay logs to a custom logger.
Exposes LogWriter to bindings.
Test logging to different destinations with:
- In-memory log logger,
- In-memory LogWriter logger.

Related Issue(s)

Expose logger interface #309
Cleaner implementation of closed draft feat: add customizable logging system #401

tnull

Cool, thank you for looking into this! This already looks pretty good, but I have some comments after the first round of review.

tnull · 2024-11-27T13:37:30Z

 }

 /// Simple in-memory mock `log` logger for tests.
+#[derive(Debug)]


Why is this Debug?

NodeBuilder is Debug, and it fields an optional LogWriterConfig. The config object has to be Debug and custom log writer variant needs to implement it for this reason.

tnull · 2024-12-09T08:14:56Z

@enigbe Is there any update on this? Please let me know if you're hitting any blockers.

This seems to need a minor rebase by now.

enigbe · 2024-12-10T03:50:34Z

No blockers on this. I'll be pushing updates later today.

tnull · 2024-12-18T09:32:43Z

@enigbe Please let me know if/when this is ready for the next round of review!

enigbe

@tnull I believe this is ready for another review. I have addressed the majority of the concerns you raised in the first pass.

Regarding your concerns about testing the logging to custom loggers, I agree that the necessary refactor would be extensive and could detract from the purpose of this PR. As such, I plan to address these test-related changes in a follow-up PR.

Additionally, I encountered some challenges testing a custom logger in Kotlin and would greatly appreciate your guidance or suggestions.

tnull

Did another round of review, already looks pretty good.

I took a quick look at the Kotlin failures, but also couldn't immediately spot what's up, let me know if you want me to have a closer look though.

Btw, you could consider rebasing on #426 (or on main after it lands) which generally fixes pre-existing CI failures.

tnull · 2025-01-06T13:39:38Z

+
+impl LogWriter for Writer {
+	fn log(&self, record: LogRecord) {
+ let log = format!(


Hmm, so format! allocates a string on the heap. As we try to avoid allocations (to reduce heap fragmentation where possible), can we a) only create this string when we're sure we need it to log (i.e, after passing the log level filtering), and b) can we avoid it altogether for the CustomWriter case, and possibly even the LogFacadeWriter case? In the latter case we should be able to give the arguments to the respective macros directly, no?

tnull · 2025-01-17T12:35:46Z

@enigbe Sorry, this needs a rebase now that #426 landed (which however also should fix the pre-existing CI failures, finally)., Let me know if this is ready for another round of review!

tnull · 2025-01-21T13:30:27Z

Feel free to drop this commit and rebase cleanly on main.

tnull

Cool, basically LGTM, just a few last-minute nits/and cleanups.

Let's address them quickly and land this soon.

tnull · 2025-01-27T11:41:11Z

-	/// Sets the log file path if the log file needs to live separate from the storage directory path.
-	pub fn set_log_file_path(&mut self, log_dir_path: String) -> &mut Self {
- self.config.log_file_path = Some(log_dir_path);
+	/// Configures the [`Node`] instance to write logs to the filesystem with an optional


nit: Let's end the first paragraph after filesystem and give explaining the two arguments a short sentence of their own, something like:
Suggested change
/// Configures the [`Node`] instance to write logs to the filesystem with an optional
/// Configures the [`Node`] instance to write logs to the filesystem.
///
/// If set to `None`, `log_file_path` will default to [`DEFAULT_LOG_FILE_PATH`].
/// If set to `None`, `log_level` will default to [`DEFAULT_LOG_LEVEL`].

tnull · 2025-01-27T11:42:07Z

+ self
+	}
+
+	/// Configures the [`Node`] instance to write logs to the provided custom log writer.


nit: Let's link LogWriter here:
Suggested change
/// Configures the [`Node`] instance to write logs to the provided custom log writer.
/// Configures the [`Node`] instance to write logs to the provided custom [`LogWriter`].

tnull · 2025-01-27T11:42:19Z

-	/// Sets the log file path if logs need to live separate from the storage directory path.
-	pub fn set_log_file_path(&self, log_file_path: String) {
- self.inner.write().unwrap().set_log_file_path(log_file_path);
+	/// Configures the [`Node`] instance to write logs to the filesystem with an optional


Adjust as above.

tnull · 2025-01-27T11:42:31Z

+ self.inner.write().unwrap().set_log_facade_logger(log_level);
+	}
+
+	/// Configures the [`Node`] instance to write logs to the provided custom log writer.


Adjust as above.

tnull · 2025-01-27T11:54:28Z


 pub use io::utils::generate_entropy_mnemonic;

+pub use config::FilesystemLoggerConfig;


As mentioned above, no need to export this anymore.

tnull · 2025-01-27T11:55:31Z

 use std::path::Path;
+use std::sync::Arc;
+
+/// A unit of logging output with Metadata to enable filtering module_path,


nit: Lowercase metadata and tick all the variable names:
Suggested change
/// A unit of logging output with Metadata to enable filtering module_path,
/// A unit of logging output with metadata to enable filtering `module_path`,

tnull · 2025-01-27T11:55:58Z

+	pub line: u32,
+}
+
+#[cfg(feature = "uniffi")]


This needs docs (copied from above).

tnull · 2025-01-27T11:57:55Z

-	level: Level,
+/// Defines the behavior required for writing log records.
+///
+/// This version is used when the `uniffi` feature is enabled.


Let's still add the second paragraph from above here before going into the Uniffi-specific differences.

tnull · 2025-01-27T12:00:31Z

 	setup_builder!(builder, config);
 	builder.set_chain_source_esplora(esplora_url.clone(), Some(sync_config));

+	let log_file_path = format!("{}/{}", config.storage_dir_path, "ldk_node.log");


Let's drop this? Or any reason why this test would need a special case? If we want to test the interface works, we could consider adding a test for that which then however should confirm that only the correct types of messages are logged. But let's maybe add that as a follow-up.

* Add flexible log writer interface for multiple destinations * Implement filesystem writing capability via FilesystemLogger * Prefix LDK-based objects with 'Ldk' for consistency * Add configuration options for log file path and log level

- modify tests to forward logs to mock in-memory `log` logger - correct "Forwards" spelling error

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger. * Fix setting log's global logger twice. * Revert the renaming of LogLevel to LdkLevel.

This commit addresses a series of fixes, refactors, and documentation changes meant to optimize the new logging system to be better with: - Improved memory usage by eliminating unnecessary data allocations and improving log record handling. - Enhanced logging system with support for foreign implementations (as tested with Kotlin). - Improved documentation across board. - Cleaner refactors and simpler code that improve clarity. - Streamlined logging configuration. - Pre-existing bug fixes.

tnull

LGTM 🎉

Thank you very much!

amackillop · 2025-01-30T16:29:07Z

Sweet will try this out

enigbe mentioned this pull request Nov 20, 2024
feat: add customizable logging system #401
Closed
7 tasks

G8XSU requested a review from tnull November 21, 2024 18:37

tnull reviewed Nov 27, 2024
View reviewed changes

enigbe force-pushed the 2024-11-configurable-logging-system branch from 1d57cab to d8eb0e1Compare December 11, 2024 12:18

G8XSU mentioned this pull request Dec 12, 2024
Ldk-Server Milestone 3 lightningdevkit/ldk-server#11
Open
9 tasks

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 7fa2928 to 283fe85Compare December 16, 2024 00:32

enigbe force-pushed the 2024-11-configurable-logging-system branch 4 times, most recently from 0660dfa to 6766c60Compare December 19, 2024 21:54

enigbe commented Dec 20, 2024
View reviewed changes

tnull reviewed Jan 6, 2025
View reviewed changes

tnull reviewed Jan 13, 2025
View reviewed changes

Comment threadsrc/logger.rs Outdated

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 322f97f to 3b71223Compare January 21, 2025 10:01

tnull reviewed Jan 21, 2025
View reviewed changes

Comment threadsrc/payment/unified_qr.rs
tnullJan 21, 2025
Copy link
Copy Markdown
Collaborator
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feel free to drop this commit and rebase cleanly on main.

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from f06097b to c43fe47Compare January 27, 2025 10:40

tnull reviewed Jan 27, 2025
View reviewed changes

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 241bfcc to 96522d9Compare January 29, 2025 19:56

tnull mentioned this pull request Jan 30, 2025
Minor follow-ups to #407 #450
Merged

enigbe added 2 commits January 30, 2025 11:42

feat(logger): add & impl LogWriter for Writer
6d8c65e

feat(facade): forward logs to log facade
fe0f547
- modify tests to forward logs to mock in-memory `log` logger - correct "Forwards" spelling error

enigbe added 2 commits January 30, 2025 13:04

enigbe force-pushed the 2024-11-configurable-logging-system branch from 96522d9 to e509cf8Compare January 30, 2025 12:24

tnull approved these changes Jan 30, 2025
View reviewed changes

tnull merged commit 83159d0 into lightningdevkit:mainJan 30, 2025

tnull mentioned this pull request Jan 30, 2025
Expose logger interface #309
Closed

tnull mentioned this pull request Jan 31, 2025
Fix log-facade logging and clarify max level #454
Merged

This was referenced Feb 10, 2025
Test logging interface enigbe/ldk-node#4
Merged
Test logging interface #463
Merged

rustaceanrob mentioned this pull request Feb 14, 2025
Logging requires heap allocations 2140-dev/kyoto#298
Closed

benthecarman mentioned this pull request Sep 19, 2025
Expose logger interface lightningdevkit/orange-sdk#21
Closed

-/// Configures the [`Node`] instance to write logs to the filesystem with an optional
+/// Configures the [`Node`] instance to write logs to the filesystem.
+///
+/// If set to `None`, `log_file_path` will default to [`DEFAULT_LOG_FILE_PATH`].
+/// If set to `None`, `log_level` will default to [`DEFAULT_LOG_LEVEL`].

	/// Configures the [`Node`] instance to write logs to the provided custom log writer.
	/// Configures the [`Node`] instance to write logs to the provided custom [`LogWriter`].


		pub use io::utils::generate_entropy_mnemonic;

		pub use config::FilesystemLoggerConfig;

	/// A unit of logging output with Metadata to enable filtering module_path,
	/// A unit of logging output with metadata to enable filtering `module_path`,

Uh oh!

Conversation

enigbe commented Nov 20, 2024

Overview

What this PR does

Related Issue(s)

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

enigbeDec 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tnull commented Dec 9, 2024

Uh oh!

enigbe commented Dec 10, 2024

Uh oh!

tnull commented Dec 18, 2024

Uh oh!

enigbe left a comment

Choose a reason for hiding this comment

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tnull commented Jan 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tnull left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amackillop commented Jan 30, 2025

enigbeDec 17, 2024•
edited
Loading

tnull left a comment •
edited
Loading