Add some timing info to rustdoc

[jyn514]

There are various improvements, but the main one is to time each pass
that rustdoc performs (rustdoc::passes).

Before, these were the top five timings for cargo doc on the cargo
repository:

+---------------------------------+-----------+-----------------+----------+------------+
| Item | Self time | % of total time | Time | Item count |
+---------------------------------+-----------+-----------------+----------+------------+
| <unknown> | 854.70ms | 20.888 | 2.47s | 744823 |
+---------------------------------+-----------+-----------------+----------+------------+
| expand_crate | 795.29ms | 19.436 | 848.00ms | 1 |
+---------------------------------+-----------+-----------------+----------+------------+
| metadata_decode_entry | 256.73ms | 6.274 | 279.49ms | 518344 |
+---------------------------------+-----------+-----------------+----------+------------+
| resolve_crate | 240.56ms | 5.879 | 242.86ms | 1 |
+---------------------------------+-----------+-----------------+----------+------------+
| hir_lowering | 146.79ms | 3.587 | 146.79ms | 1 |
+---------------------------------+-----------+-----------------+----------+------------+

Now the timings are:

+---------------------------------+-----------+-----------------+----------+------------+
| Item | Self time | % of total time | Time | Item count |
+---------------------------------+-----------+-----------------+----------+------------+
| <unknown> | 1.40s | 22.662 | 3.73s | 771430 |
+---------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls | 1.34s | 21.672 | 2.87s | 1 |
+---------------------------------+-----------+-----------------+----------+------------+
| expand_crate | 1.21s | 19.577 | 1.28s | 1 |
+---------------------------------+-----------+-----------------+----------+------------+
| build extern trait impl | 704.66ms | 11.427 | 1.07s | 21893 |
+---------------------------------+-----------+-----------------+----------+------------+
| metadata_decode_entry | 354.84ms | 5.754 | 391.81ms | 544919 |
+---------------------------------+-----------+-----------------+----------+------------+

The goal is to help me debug regressions like #74518 (comment) (currently I have no idea what could have gone wrong).

r? @eddyb or @Mark-Simulacrum

[Mark-Simulacrum]

Do we have some idea where the unknown is coming from? The time spent in it seems to have ... gone up?

[jyn514]

@Mark-Simulacrum the overall time went up as well, I think because of one of the regressions in https://github.com/rust-lang/rustc-perf/blob/master/triage/2020-07-21.md. I would look at the percentages instead.

[jyn514]

I'm not sure where the <unknown> is coming from - I tried to wrap all of rustdoc::core in a timer but that didn't work because the main function doesn't have access to a compile session.

[bors]

☔ The latest upstream changes (presumably #75008) made this pull request unmergeable. Please resolve the merge conflicts.

[eddyb]

@jyn514 You don't need to wrap the entirety of rustdoc, just the outermost thing that has access to a timer/profiler.

r? @Mark-Simulacrum or @wesleywiser

[eddyb]

Note: not a bad change, but would not have been marked as <unknown> before (it would show up as part of typeck).

[eddyb]

At the very least, it looks like the entirety of global_ctxt.enter(|tcx| { could be wrapped in a named region (if not more).

[jyn514]

Addressed review comments. I separated the enormous closure into a separate function, so best to view with whitespace changes disabled. Building rustdoc now to see if that got rid of most of the <unknown>.

[jyn514]

Timing the global context didn't seem to help much.

+-------------------------------------------------+-----------+-----------------+----------+------------+
| Item | Self time | % of total time | Time | Item count |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls | 1.46s | 23.619 | 3.26s | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| expand_crate | 1.20s | 19.538 | 1.25s | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| <unknown> | 1.09s | 17.746 | 3.17s | 753829 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| run_global_ctxt | 3.34ms | 0.054 | 4.24s | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+

[jyn514]

I did find this comment though:

rust/src/librustc_codegen_ssa/base.rs

Lines 782 to 792 in 62cf767

	fnfinalize_tcx(tcx:TyCtxt<'_>){
	tcx.sess.time("assert_dep_graph", || ::rustc_incremental::assert_dep_graph(tcx));
	tcx.sess.time("serialize_dep_graph", || ::rustc_incremental::save_dep_graph(tcx));
	// We assume that no queries are run past here. If there are new queries
	// after this point, they'll show up as "<unknown>" in self-profiling data.
	{
	let _prof_timer = tcx.prof.generic_activity("self_profile_alloc_query_strings");
	tcx.alloc_self_profile_query_strings();
	}
	}

[jyn514]

Aha, I think I found the issue: all of the html rendering isn't timed.

rust/src/librustdoc/lib.rs

Lines 525 to 533 in 1d69e3b

	let diag = core::new_handler(error_format,None,&debugging_options);
	match output_format {
	None | Some(config::OutputFormat::Html) => {
	run_renderer::<html::render::Context>(krate, renderopts, renderinfo,&diag, edition)
	}
	Some(config::OutputFormat::Json) => {
	run_renderer::<json::JsonRenderer>(krate, renderopts, renderinfo,&diag, edition)
	}
	}

[jyn514]

Well, that helped a little.

$ summarize summarize time-html/unknown-crate-3300.events +-------------------------------------------------+-----------+-----------------+----------+------------+
| Item | Self time | % of total time | Time | Item count |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls | 1.46s | 21.568 | 3.29s | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| expand_crate | 1.22s | 18.103 | 1.27s | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| <unknown> | 1.12s | 16.504 | 3.35s | 753829 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| build extern trait impl | 822.75ms | 12.162 | 1.34s | 21968 |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| render html | 439.39ms | 6.495 | 439.39ms | 1 |
+-------------------------------------------------+-----------+-----------------+----------+------------+

[jyn514]

It's coming from item_types_checking.

rustdoc.zip

[jyn514]

Anyone know why this might be getting the timing name reset?

 tcx.sess.time("item_types_checking", || {for&module in tcx.hir().krate().modules.keys(){
tcx.sess.time("check_mod_item_types", || {
tcx.ensure().check_mod_item_types(tcx.hir().local_def_id(module));});}});

check_mod_item_types works fine and shows almost exactly 0 time spent in it, instead there's half a second of <unknown>.
https://github.com/rust-lang/rust/pull/74590/files?diff=unified&w=1#diff-0de644786a2f1fd88c7d7f44bc3fa4bbR497

[Mark-Simulacrum]

I would personally like us to track down the problem with unknown timing before merging this in, though I don't have great ideas for doing that. Maybe @wesleywiser has suggestions?

[bors]

☔ The latest upstream changes (presumably #74489) made this pull request unmergeable. Please resolve the merge conflicts.

[jyn514]

@Mark-Simulacrum I don't expect to have time to debug this soon ... are you okay with merging the current changes and opening an issue for the <unknown>? I'd rather have some timing than having to guess.

[wesleywiser]

I've only skimmed the PR but I think we should not block merging this with resolving the unknown issue. We've seen that before in other profiles so it's probably another instance of the existing issue. I hope to have some time this week to look into it.

[Mark-Simulacrum]

Okay, knowing that we've seen this before is indeed helpful and that convinces me that it should be fine to move ahead here.

@bors r+

[bors]

📌 Commit 6e3e74d has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit 6e3e74d with merge 0301700...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: Mark-Simulacrum
Pushing 0301700 to master...

[camelid]

You may already know this, but this seems to be the rustc code that produces the <unknown> name:

rust/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs

Line 795 in 192c7db

let file_name = file_name.as_deref().unwrap_or("<unknown>");

Not sure why it's happening though.

[jyn514]

That's under debuginfo, so I wouldn't expect it to be related.

I did notice a while back that rustdoc doesn't include the crate name in the .events file - I think that might be because doctree passes None for the crate name? I need to test it.
https://github.com/rust-lang/rust/blob/master/src/librustdoc/visit_ast.rs#L71

[jyn514]

I did notice a while back that rustdoc doesn't include the crate name in the .events file - I think that might be because doctree passes None for the crate name? I need to test it.

This was #79586.