[perf] don't use temporary files for archive builder

[belovdv]

This is a draft - for perf testing only.

r? @petrochenkov

[petrochenkov]

@bors try @rust-timer queue

[bors]

⌛ Trying commit eb0cba5437deeb68fc44c60b6df85fbfc219ad7e with merge 2a3475da1922519835b37aa7d28bfc574e55f09d...

[bors]

☀️ Try build successful - checks-actions
Build commit: 2a3475da1922519835b37aa7d28bfc574e55f09d (2a3475da1922519835b37aa7d28bfc574e55f09d)

[rust-timer]

Finished benchmarking commit (2a3475da1922519835b37aa7d28bfc574e55f09d): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.8%	[0.5%, 3.7%]	48
Regressions ❌ (secondary)	2.5%	[0.6%, 5.2%]	84
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.1%]	1
Improvements ✅ (secondary)	-2.7%	[-2.7%, -2.7%]	1
All ❌✅ (primary)	1.8%	[-0.1%, 3.7%]	49

Cycles

This benchmark run did not return any relevant results for this metric.

[bjorn3]

There doesn't seem to be much task clock or wall time change in either direction, but it shows a clear regression in max-rss. I think having a way to stream archive files directly to the disk while building them would work better. This is not something that LLVM supports though and it is non-trivial for every variant of the ar archive format due to the presence of the symbol table (and for the gnu variant a member name table for long names) that needs to be at the start of the file. For lld the symbol table is optional since recently as it was found to not provide a perf benefit for lld, but most other linkers still mandate it's presence.

[petrochenkov]

@bjorn3

but it shows a clear regression in max-rss

I don't understand why it happens.
LLVM's NewArchiveMember::getFile reads this file back into memory or mmaps it if it's large enough.
In the second case all the pages will also be retreived because they need to be read.
Not sure where the memory economy comes from.

[petrochenkov]

If this PR goes towards merging, then this is better added as a new variant to enum Addition instead of a separate vector.

[petrochenkov]

r? @bjorn3

[bjorn3]

Not sure where the memory economy comes from.

Mmap only counts towards max-rss when being paged in afaik. In addition the os is free to page it out even when no swap memory exists as it can be loaded from the original file again. This last point is important as without we did likely hit an OOM condition a lot more on systems with little memory.

[klensy]

This was run on CI with SSD\NVMe drives (probably), maybe running it on HDD will show the diff.

[belovdv]

Changes could be more noticeable after #104091, so I'll work on that first.
@rustbot author

[bors]

☔ The latest upstream changes (presumably #97485) made this pull request unmergeable. Please resolve the merge conflicts.

[Dylan-DPC]

@belovdv any updates on this?

[belovdv]

I couldn't find clear benefits from this change (please, correct me if it's wrong).