Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328

Merged
44 commits merged into
masterfrom
cecton-logging-singleton
Oct 21, 2020
Merged

Add node name to the log lines#7328
44 commits merged into
masterfrom
cecton-logging-singleton

Conversation

@cecton

@cectoncecton commented Oct 15, 2020•
edited
Loading

Copy link
Copy Markdown
Contributor

Related to paritytech/cumulus#149

polkadot companion: paritytech/polkadot#1825

@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr

bkchr commented Oct 15, 2020

Copy link
Copy Markdown
Member

Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.

@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton

cecton commented Oct 15, 2020

Copy link
Copy Markdown
ContributorAuthor

Yes it's more related to paritytech/cumulus#154 actually.

This change already prefixes the logs with the node name. Example here on substrate:

2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0

On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).

The way this is done is by using a span from tracing which includes the node name.

I still need to figure out for separating the telemetry.

@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr

bkchr commented Oct 15, 2020

Copy link
Copy Markdown
Member

This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.

And why are still some lines without any prefix in your example output?

Regarding telemetry, as I said, we should pass this object around like prometheus.

@cecton

cecton commented Oct 15, 2020

Copy link
Copy Markdown
ContributorAuthor

This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.

Yes, I don't see any blocker for that. No worry.

And why are still some lines without any prefix in your example output?

Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().

Some I'm not sure I'm investigating.

Regarding telemetry, as I said, we should pass this object around like prometheus.

Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.

@bkchr

bkchr commented Oct 15, 2020

Copy link
Copy Markdown
Member

Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.

We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.

But this is clearly now part of another pr!

@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton marked this pull request as ready for review October 16, 2020 06:34
@github-actionsgithub-actionsBot added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr reviewed Oct 21, 2020
View reviewed changes

@bkchrbkchr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.

Comment threadclient/basic-authorship/src/basic_authorship.rs Outdated
Comment threadclient/basic-authorship/src/basic_authorship.rs Outdated
Comment threadclient/cli/proc-macro/src/lib.rs Outdated
use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.

@bkchrbkchrOct 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent

Comment threadclient/cli/proc-macro/src/lib.rs Outdated
///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.

@bkchrbkchrOct 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.

Or similar?

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment threadclient/cli/proc-macro/src/lib.rs Outdated
///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]

@bkchrbkchrOct 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment threadclient/cli/proc-macro/src/lib.rs Outdated
(quote! {
#(#attrs)*
#vis #sig {
let span = #crate_name::tracing::info_span!("substrate-node", name = #name);

@bkchrbkchrOct 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!

@cectoncectonOct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment threadclient/cli/src/logging.rs Outdated
cectonand others added 6 commits October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton

cecton commented Oct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

BOT MERGE

@ghost

ghost commented Oct 21, 2020

Copy link
Copy Markdown

Checks failed; merge aborted.

@cecton

cecton commented Oct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

geezus

@cecton

cecton commented Oct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

bot merge --force

@cecton

cecton commented Oct 21, 2020

Copy link
Copy Markdown
ContributorAuthor

bot merge force

@ghost

ghost commented Oct 21, 2020

Copy link
Copy Markdown

Trying merge.

@ghost
ghost merged commit a467358 into masterOct 21, 2020
@ghost
ghost deleted the cecton-logging-singleton branch October 21, 2020 15:13
@tomakatomaka mentioned this pull request Nov 3, 2020
Merged
@dvdplmdvdplm mentioned this pull request Nov 4, 2020
Merged
5 tasks
This pull request was closed.
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf
2 more reviewers
@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Add node name to the log lines by cecton · Pull Request #7328 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Add node name to the log lines - #7328
Merged
44 commits merged into
masterfrom 
cecton-logging-singleton
Oct 21, 2020
Merged
Add node name to the log lines#7328
44 commits merged into
masterfrom 
cecton-logging-singleton
Conversation
@cecton
@cectoncecton
commented
Oct 15, 2020•

edited

Loading

Copy link


Copy Markdown

Contributor
Related to paritytech/cumulus#149
polkadot companion: paritytech/polkadot#1825
@cecton
Initial commit
88c1f42
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
95762f1
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
685c122
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
fea1d0c
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
f0d5964
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
a55ccf0
Forked at: d67fc4c
Parent branch: origin/master
@github-actionsgithub-actionsBot
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
Oct 15, 2020
@cecton
CLEANUP
ad7dbd5
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add notes to original source code
c8e0841
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Can you please explain on what your idea here is? I don't see the connection to the ticket and what we spoke about.
@cecton
CLEANUP
dc2f401
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
Yes it's more related to paritytech/cumulus#154 actually.
This change already prefixes the logs with the node name. Example here on substrate:
2020-10-15 13:32:49 Substrate Node 2020-10-15 13:32:49 ✌️ version 2.0.0-a55ccf09b-x86_64-linux-gnu 2020-10-15 13:32:49 ❤️ by Parity Technologies <admin@parity.io>, 2017-2020 2020-10-15 13:32:49 📋 Chain specification: Flaming Fir 2020-10-15 13:32:49 🏷 Node name: wry-wren-1666 2020-10-15 13:32:49 👤 Role: FULL 2020-10-15 13:32:49 💾 Database: RocksDb at /tmp/substrateHXzOtQ/chains/flamingfir8/db 2020-10-15 13:32:49 ⛓ Native runtime: node-259 (substrate-node-1.tx1.au10) 2020-10-15 13:32:50 🔨 Initializing Genesis block/state (state: 0x89b8…d3e7, header-hash: 0xe40f…ed39) 2020-10-15 13:32:50 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2020-10-15 13:32:50 ⏱ Loaded block-time = 3000 milliseconds from genesis on first-launch 2020-10-15 13:32:50 👶 Creating empty BABE epoch changes on what appears to be first startup. 2020-10-15 13:32:50 🏷 Local node identity is: 12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 📦 Highest known block at #0 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: 〽️ Prometheus server started at 127.0.0.1:9615 2020-10-15 13:32:50 [wry-wren-1666] substrate-node: Listening for new connections on 127.0.0.1:9944. 2020-10-15 13:32:51 🔍 Discovered new external address for our node: /ip4/91.181.156.13/tcp/30333/p2p/12D3KooWSQXLEGL2Ho2Qq8djLTXysTDhoTjFAUYvvEjbfW1kSd4F 2020-10-15 13:32:55 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 2.7kiB/s ⬆ 2.4kiB/s 2020-10-15 13:33:00 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 67 B/s ⬆ 58 B/s 2020-10-15 13:33:05 [wry-wren-1666] substrate-node: 💤 Idle (0 peers), best: #0 (0xe40f…ed39), finalized #0 (0xe40f…ed39), ⬇ 0 ⬆ 0 
On cumulus we will get the node name of the parachain and the relaychain depending on which node is the log. We could also customize this instead of using the node name (it's not done but it's doable easily if you want).
The way this is done is by using a span from tracing which includes the node name.
I still need to figure out for separating the telemetry.
@cecton
CLEANUP
34106dd
Forked at: d67fc4c
Parent branch: origin/master
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
And why are still some lines without any prefix in your example output?
Regarding telemetry, as I said, we should pass this object around like prometheus.
@cecton
cecton
commented
Oct 15, 2020

Copy link


Copy Markdown

ContributorAuthor
This should be an optional feature and we actually want to have [Parachain] or [Relaychain] being printed as we already do it in some places.
Yes, I don't see any blocker for that. No worry.
And why are still some lines without any prefix in your example output?
Some are definitely outside the span (the span is created in client/service/src/builder.rs fn spawn_tasks().
Some I'm not sure I'm investigating.
Regarding telemetry, as I said, we should pass this object around like prometheus.
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
@bkchr
bkchr
commented
Oct 15, 2020

Copy link


Copy Markdown

Member
Tbh I'm still not 100% to understand how telemetry works. I'm still investigating.
We just call a macro that aggregates the data and sends it as json to the server. But that is not important at all. Important is that we replace the singleton with an instance that is passed everywhere and we pass this instance to these macros.
But this is clearly now part of another pr!
@cecton
WIP
76f2d43
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
0c278a8
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
8257480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
47f7d3f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
WIP
efb9b82
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Some doc
346eaf1
@cecton
Test with trybuild
9055ec2
@cecton
Revert "Test with trybuild" (issue with trybuild atm)
d46d87b
This reverts commit 9055ec2.
dtolnay/trybuild#53
@cecton
cecton
marked this pull request as ready for review
October 16, 2020 06:34
@github-actionsgithub-actionsBot
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
Oct 16, 2020
@cecton
CLEANUP
a554480
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Improve error message
58a7284
@cecton
CLEANUP
de4678f
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Fix async issue
c101a09
@cecton
CLEANUP
35ae152
Forked at: d67fc4c
Parent branch: origin/master
@cecton
CLEANUP
fb4c7bb
Forked at: d67fc4c
Parent branch: origin/master
@cecton
Add test
2238794
@cecton
fix doc test
c4e0039
@cecton
Merge commitc2469eb(no conflict)
1a72fc5
bkchr
bkchr
reviewed
Oct 21, 2020
View reviewed changes
@bkchrbkchr
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some last changes and than we are ready.
Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/basic-authorship/src/basic_authorship.rs
Outdated

Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

use quote::quote;
use syn::{Error, Expr, Ident, ItemFn};

/// Macro that inserts a tracing span with the node name at the beginning of the function.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says something different than the body.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the doc. Should be more consistent
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// # Implementation notes
///
/// If there are multiple spans with a node name, only the latest will be shown.
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

/// If there are multiple spans with a node name, only the latest will be shown.
/// If there are multiple spans with a log prefix, only the latest will be shown.
Or similar?
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

///
/// ```ignore
/// Builds a new service for a light client.
/// #[sc_cli::substrate_cli_node_name("light")]
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/proc-macro/src/lib.rs
Outdated

	(quote! {
 #(#attrs)*
 #vis #sig {
 let span = #crate_name::tracing::info_span!("substrate-node", name = #name);
@bkchrbkchrOct 21, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this span name not be a constant in sc-cli? And it should be renamed to substrate-log-prefix or similar.
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hardly because I can't import the const from sc-cli (sc-cli-proc-macro is a dependency of sc-cli). And I'm not too fan to put it in the the proc macro crate... but if you want think it's better I will do it
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, you're right. I see. nevermind!
@cectoncectonOct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
Comment threadclient/cli/src/logging.rs
Outdated

cectonand others
added 6 commits
October 21, 2020 15:00
@cecton@bkchr
Update client/cli/src/logging.rs
2752062
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
208bbe2
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton@bkchr
Update client/basic-authorship/src/basic_authorship.rs
243dd6c
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
@cecton
Apply suggestions
929bd1e
@cecton
Suggestions
f12a3f7
@cecton
Clarify doc
b9d7723
@cecton
WIP
f141b73
Forked at: d67fc4c
Parent branch: origin/master
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
BOT MERGE
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Checks failed; merge aborted.
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
geezus
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge --force
@cecton
cecton
commented
Oct 21, 2020

Copy link


Copy Markdown

ContributorAuthor
bot merge force
@ghost
ghost
commented
Oct 21, 2020

Copy link


Copy Markdown

Trying merge.
@ghost

ghost

merged commit a467358
into
masterOct 21, 2020
@ghost

ghost

deleted the

cecton-logging-singleton

branch
October 21, 2020 15:13
@tomakatomaka
mentioned this pull request
Nov 3, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Nov 4, 2020
 Merged

5 tasks

This pull request was closed.

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@andresilvaandresilvaAwaiting requested review from andresilva
@sorpaassorpaasAwaiting requested review from sorpaas
@dvdplmdvdplmAwaiting requested review from dvdplm
@NikVolfNikVolfAwaiting requested review from NikVolf

2 more reviewers

@bkchrbkchrbkchr approved these changes
@mattrutherfordmattrutherfordmattrutherford approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

A0-please_reviewPull request needs code review.B0-silentChanges should not be mentioned in any release notesC1-lowPR touches the given topic and has a low impact on builders.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@cecton@bkchr@dvdplm@mattrutherford