Add support for allocators in Rc & Arc

[Cyborus04]

Adds the ability for std::rc:Rc, std::rc::Weak, std::sync::Arc, and std::sync::Weak to live in custom allocators

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @m-ou-se (or someone else) soon.

Please see the contribution instructions for more information.

[Cyborus04]

Oh, finally

This was a mess, I'm so sorry.

[jyn514]

@Cyborus04 can you squash the commits? 35 is a bit much haha

[Cyborus04]

Oh dang, was it really that many? Haha wow, sorry
5 seems better

[bors]

☔ The latest upstream changes (presumably #89144) made this pull request unmergeable. Please resolve the merge conflicts.

[Cyborus04]

I think I did that right? The guide is for the Git CLI but I used Github Desktop so I'm not entirely sure

[bors]

☔ The latest upstream changes (presumably #89386) made this pull request unmergeable. Please resolve the merge conflicts.

[Cyborus04]

I'm so confused, I didn't think I touched those files.
(src/tools/cargo & src/tools/rust-analyzer)

[bors]

⌛ Testing commit 215cf36 with merge e6d0ee82dfeb886635f13bca987932406364c1db...

[bors]

💔 Test failed - checks-actions

[tgross35]

2023-07-01T01:52:19.7038601Z failures:
2023-07-01T01:52:19.7038789Z 2023-07-01T01:52:19.7039019Z ---- [debuginfo-cdb] tests\debuginfo\rc_arc.rs stdout ----
2023-07-01T01:52:19.7039304Z 2023-07-01T01:52:19.7047446Z error: line not found in debugger output: rc,d : 111 [Type: alloc::rc::Rc<i32>]
2023-07-01T01:52:19.7047792Z status: exit code: 0
...
2023-07-01T01:52:19.7105360Z ---- [debuginfo-cdb] tests\debuginfo\thread.rs stdout ----
2023-07-01T01:52:19.7105522Z 2023-07-01T01:52:19.7105764Z error: line not found in debugger output: [...] inner [...][Type: core::pin::Pin<alloc::sync::Arc<std::thread::Inner> >]
2023-07-01T01:52:19.7106051Z status: exit code: 0

tests\debuginfo\rc_arc.rs and tests\debuginfo\thread.rs seem to be the two failures. Are these tests blessable? (I can't even directly run them locally) If not, I think the below is all that's needed to fix it:

--- tests/debuginfo/rc_arc.rs+++ tests/debuginfo/rc_arc.rs@@ -30 +30 @@- // cdb-check:rc,d : 111 [Type: alloc::rc::Rc<i32>]+ // cdb-check:rc,d : 111 [Type: alloc::rc::Rc<i32,alloc::alloc::Global>]@@ -35 +35 @@- // cdb-check:weak_rc,d : 111 [Type: alloc::rc::Weak<i32>]+ // cdb-check:weak_rc,d : 111 [Type: alloc::rc::Weak<i32,alloc::alloc::Global>]@@ -40 +40 @@- // cdb-check:arc,d : 222 [Type: alloc::sync::Arc<i32>]+ // cdb-check:arc,d : 222 [Type: alloc::sync::Arc<i32,alloc::alloc::Global>]@@ -45 +45 @@- // cdb-check:weak_arc,d : 222 [Type: alloc::sync::Weak<i32>]+ // cdb-check:weak_arc,d : 222 [Type: alloc::sync::Weak<i32,alloc::alloc::Global>]@@ -50 +50 @@- // cdb-check:dyn_rc,d [Type: alloc::rc::Rc<dyn$<core::fmt::Debug> >]+ // cdb-check:dyn_rc,d [Type: alloc::rc::Rc<dyn$<core::fmt::Debug,alloc::alloc::Global> >]@@ -55 +55 @@- // cdb-check:dyn_rc_weak,d [Type: alloc::rc::Weak<dyn$<core::fmt::Debug> >]+ // cdb-check:dyn_rc_weak,d [Type: alloc::rc::Weak<dyn$<core::fmt::Debug,alloc::alloc::Global> >]@@ -60 +60 @@- // cdb-check:slice_rc,d : { len=3 } [Type: alloc::rc::Rc<slice2$<u32> >]+ // cdb-check:slice_rc,d : { len=3 } [Type: alloc::rc::Rc<slice2$<u32,alloc::alloc::Global> >]@@ -69 +69 @@- // cdb-check:slice_rc_weak,d : { len=3 } [Type: alloc::rc::Weak<slice2$<u32> >]+ // cdb-check:slice_rc_weak,d : { len=3 } [Type: alloc::rc::Weak<slice2$<u32,alloc::alloc::Global> >]@@ -78 +78 @@- // cdb-check:dyn_arc,d [Type: alloc::sync::Arc<dyn$<core::fmt::Debug> >]+ // cdb-check:dyn_arc,d [Type: alloc::sync::Arc<dyn$<core::fmt::Debug,alloc::alloc::Global> >]@@ -83 +83 @@- // cdb-check:dyn_arc_weak,d [Type: alloc::sync::Weak<dyn$<core::fmt::Debug> >]+ // cdb-check:dyn_arc_weak,d [Type: alloc::sync::Weak<dyn$<core::fmt::Debug,alloc::alloc::Global> >]@@ -88 +88 @@- // cdb-check:slice_arc,d : { len=3 } [Type: alloc::sync::Arc<slice2$<u32> >]+ // cdb-check:slice_arc,d : { len=3 } [Type: alloc::sync::Arc<slice2$<u32,alloc::alloc::Global> >]@@ -97 +97 @@- // cdb-check:slice_arc_weak,d : { len=3 } [Type: alloc::sync::Weak<slice2$<u32> >]+ // cdb-check:slice_arc_weak,d : { len=3 } [Type: alloc::sync::Weak<slice2$<u32,alloc::alloc::Global> >]--- tests/debuginfo/thread.rs+++ tests/debuginfo/thread.rs@@ -17 +17 @@- // cdb-check:[...] inner [...][Type: core::pin::Pin<alloc::sync::Arc<std::thread::Inner> >]+ // cdb-check:[...] inner [...][Type: core::pin::Pin<alloc::sync::Arc<std::thread::Inner,alloc::alloc::Global> >]

[thomcc]

@bors r=Amanieu

[bors]

📌 Commit 34f6d24 has been approved by Amanieu

It is now in the queue for this repository.

[bors]

⌛ Testing commit 34f6d24 with merge 25696703bd19f4ea7885e7c86acfc70a558b423d...

[bors]

💔 Test failed - checks-actions

[tgross35]

This test that failed has the output:

2023-07-03T20:29:37.2300246Z ---- [debuginfo-cdb] tests\debuginfo\rc_arc.rs stdout ----
2023-07-03T20:29:37.2300549Z 2023-07-03T20:29:37.2308293Z error: line not found in debugger output: slice_rc,d : { len=3 } [Type: alloc::rc::Rc<slice2$<u32>,alloc::alloc::Global>]
2023-07-03T20:29:37.2308643Z status: exit code: 0
...
2023-07-03T20:29:37.2318224Z --- stdout -------------------------------
...
2023-07-03T20:29:37.2355587Z slice_rc,d [Type: alloc::rc::Rc<slice2$<u32>,alloc::alloc::Global>]
2023-07-03T20:29:37.2355883Z [<Raw View>] [Type: alloc::rc::Rc<slice2$<u32>,alloc::alloc::Global>]
2023-07-03T20:29:37.2356162Z 0:000> dx slice_rc_weak,d
2023-07-03T20:29:37.2356409Z slice_rc_weak,d [Type: alloc::rc::Weak<slice2$<u32>,alloc::alloc::Global>]
2023-07-03T20:29:37.2356761Z Build completed unsuccessfully in 0:37:29
...
2023-07-03T20:29:37.2366562Z ------------------------------------------
2023-07-03T20:29:37.2366755Z stderr: none

It doesn't seem like this can be right... Isn't that exact text right there? I think we need a hand understanding what is going on, if anyone is more familiar with these tests

[Cyborus04]

Pointed out in Zulip that the { len=3 } is missing in the debugger, but if that's the error, why didn't Arc have an issue too?

[tgross35]

It looks like these strings are generated here:

rust/src/etc/natvis/liballoc.natvis

Lines 61 to 187 in 0d50ab7

	<!--
	The display string for Rc, Arc, etc is optional because the expression cannot be evaluated
	if the pointee is unsized (i.e. if `ptr.pointer` is a fat pointer).
	There are also two versions for the reference count fields, one for sized and one for
	dyn pointees.
	Rc<[T]> and Arc<[T]> are handled separately altogether so we can actually show
	the slice values.
	-->
	<!-- alloc::rc::Rc<T> -->
	<TypeName="alloc::rc::Rc&lt;*&gt;">
	<DisplayStringOptional="true">{ptr.pointer->value}</DisplayString>
	<Expand>
	<!-- thin -->
	<ExpandedItemOptional="true">ptr.pointer->value</ExpandedItem>
	<ItemName="[Reference count]"Optional="true">ptr.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer->weak</Item>
	<!-- dyn -->
	<ItemName="[Reference count]"Optional="true">ptr.pointer.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer.pointer->weak</Item>
	</Expand>
	</Type>
	<!-- alloc::rc::Rc<[T]> -->
	<TypeName="alloc::rc::Rc&lt;slice2$&lt;*&gt;&gt;">
	<DisplayString>{{ len={ptr.pointer.length} }}</DisplayString>
	<Expand>
	<ItemName="[Length]"ExcludeView="simple">ptr.pointer.length</Item>
	<ItemName="[Reference count]">ptr.pointer.data_ptr->strong</Item>
	<ItemName="[Weak reference count]">ptr.pointer.data_ptr->weak</Item>
	<ArrayItems>
	<Size>ptr.pointer.length</Size>
	<!-- We add +2 to the data_ptr in order to skip the ref count fields in the RcBox -->
	<ValuePointer>($T1*)(((size_t*)ptr.pointer.data_ptr) + 2)</ValuePointer>
	</ArrayItems>
	</Expand>
	</Type>
	<!-- alloc::rc::Weak<T> -->
	<TypeName="alloc::rc::Weak&lt;*&gt;">
	<DisplayStringOptional="true">{ptr.pointer->value}</DisplayString>
	<Expand>
	<!-- thin -->
	<ExpandedItemOptional="true">ptr.pointer->value</ExpandedItem>
	<ItemName="[Reference count]"Optional="true">ptr.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer->weak</Item>
	<!-- dyn -->
	<ItemName="[Reference count]"Optional="true">ptr.pointer.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer.pointer->weak</Item>
	</Expand>
	</Type>
	<!-- alloc::rc::Weak<[T]> -->
	<TypeName="alloc::rc::Weak&lt;slice2$&lt;*&gt;&gt;">
	<DisplayString>{{ len={ptr.pointer.length} }}</DisplayString>
	<Expand>
	<ItemName="[Length]"ExcludeView="simple">ptr.pointer.length</Item>
	<ItemName="[Reference count]">ptr.pointer.data_ptr->strong</Item>
	<ItemName="[Weak reference count]">ptr.pointer.data_ptr->weak</Item>
	<ArrayItems>
	<Size>ptr.pointer.length</Size>
	<ValuePointer>($T1*)(((size_t*)ptr.pointer.data_ptr) + 2)</ValuePointer>
	</ArrayItems>
	</Expand>
	</Type>
	<!-- alloc::sync::Arc<T> -->
	<TypeName="alloc::sync::Arc&lt;*&gt;">
	<DisplayStringOptional="true">{ptr.pointer->data}</DisplayString>
	<Expand>
	<!-- thin -->
	<ExpandedItemOptional="true">ptr.pointer->data</ExpandedItem>
	<ItemName="[Reference count]"Optional="true">ptr.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer->weak</Item>
	<!-- dyn -->
	<ItemName="[Reference count]"Optional="true">ptr.pointer.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer.pointer->weak</Item>
	</Expand>
	</Type>
	<!-- alloc::sync::Arc<[T]> -->
	<TypeName="alloc::sync::Arc&lt;slice2$&lt;*&gt;&gt;">
	<DisplayString>{{ len={ptr.pointer.length} }}</DisplayString>
	<Expand>
	<ItemName="[Length]"ExcludeView="simple">ptr.pointer.length</Item>
	<ItemName="[Reference count]">ptr.pointer.data_ptr->strong</Item>
	<ItemName="[Weak reference count]">ptr.pointer.data_ptr->weak</Item>
	<ArrayItems>
	<Size>ptr.pointer.length</Size>
	<ValuePointer>($T1*)(((size_t*)ptr.pointer.data_ptr) + 2)</ValuePointer>
	</ArrayItems>
	</Expand>
	</Type>
	<!-- alloc::sync::Weak<T> -->
	<TypeName="alloc::sync::Weak&lt;*&gt;">
	<DisplayStringOptional="true">{ptr.pointer->data}</DisplayString>
	<Expand>
	<!-- thin -->
	<ExpandedItemOptional="true">ptr.pointer->data</ExpandedItem>
	<ItemName="[Reference count]"Optional="true">ptr.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer->weak</Item>
	<!-- dyn -->
	<ItemName="[Reference count]"Optional="true">ptr.pointer.pointer->strong</Item>
	<ItemName="[Weak reference count]"Optional="true">ptr.pointer.pointer->weak</Item>
	</Expand>
	</Type>
	<!-- alloc::sync::Weak<[T]> -->
	<TypeName="alloc::sync::Weak&lt;slice2$&lt;*&gt;&gt;">
	<DisplayString>{{ len={ptr.pointer.length} }}</DisplayString>
	<Expand>
	<ItemName="[Length]"ExcludeView="simple">ptr.pointer.length</Item>
	<ItemName="[Reference count]">ptr.pointer.data_ptr->strong</Item>
	<ItemName="[Weak reference count]">ptr.pointer.data_ptr->weak</Item>
	<ArrayItems>
	<Size>ptr.pointer.length</Size>
	<ValuePointer>($T1*)(((size_t*)ptr.pointer.data_ptr) + 2)</ValuePointer>
	</ArrayItems>
	</Expand>
	</Type>
	</AutoVisualizer>

but I can't figure out why the length would not be displayed. Unless the ptr.pointer part isn't computing for some reason? I don't think anything there should have changed.

cc @rust-lang/wg-debugging, would you be able to help us figure out what's going on with the mismatched debug output here?

(edit: sorry wg-mir-opt, I have no clue why I accidentally cc'd you instead of wg-debugging)

[wesleywiser]

Hi @Cyborus04 and @tgross35 👋

I took the liberty of pushing a commit into your branch that fixes the failing debuginfo tests. The main thing to note is that some of the natvis visualizers do a partial match like this Rc<[*]> but with this change, they needed to be adjusted to do Rc<[*], *> instead. With that done, the tests pass locally for me so I think this is ready for another:

@bors r=Amanieu

[bors]

📌 Commit 12bed9d has been approved by Amanieu

It is now in the queue for this repository.

[tgross35]

Thank you for taking a look Wesley! Don't think we ever would have figured that out on our own 🙂

[Cyborus04]

yes, tysm!

[bors]

⌛ Testing commit 12bed9d with merge da6b55c...

[bors]

☀️ Test successful - checks-actions
Approved by: Amanieu
Pushing da6b55c to master...

[Cyborus04]

I had a double take when I saw the purple icon in my notifications, I'm so happy!

[rust-timer]

Finished benchmarking commit (da6b55c): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.8%	[0.8%, 0.8%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-1.3%, -1.3%]	1
Improvements ✅ (secondary)	-0.7%	[-0.8%, -0.5%]	4
All ❌✅ (primary)	-0.3%	[-1.3%, 0.8%]	2

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.5%	[-4.5%, -2.5%]	2
Improvements ✅ (secondary)	-2.6%	[-2.6%, -2.6%]	1
All ❌✅ (primary)	-3.5%	[-4.5%, -2.5%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.9%	[1.9%, 1.9%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.9%	[1.9%, 1.9%]	1

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.0%, 0.8%]	42
Regressions ❌ (secondary)	1.5%	[1.5%, 1.5%]	4
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	3
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.2%	[-0.2%, 0.8%]	45

Bootstrap: 657.068s -> 658.015s (0.14%)

[pnkfelix]

Visiting for perf-triage

• primary regression was to image-0.24.1 opt full by 0.79%
• I think this is just noise. From the graph, it seems like image has unpredictably jumped up and down between two plateaus since PR Eliminate ZST allocations in Box and Vec #113113 (a PR discussed up above that changed low level allocation procotol code in Box and Vec, and thus might be expected to have some weird follow-on effects).
• marking as triaged

@rustbot label: +perf-regression-triaged