Small tweaks in ToOwned::clone_into by cuviper · Pull Request #70201 · rust-lang/rust

cuviper · 2020-03-20T23:08:06Z

<[T]>::clone_into is slightly more optimized.
CStr::clone_into is new, letting it reuse its allocation.
OsStr::clone_into now forwards to the underlying slice/Vec.

rust-highfive · 2020-03-20T23:08:10Z

(rust_highfive has picked a reviewer for you, use r? to override)

cuviper · 2020-03-20T23:11:39Z

To justify the slice optimization, here's output from compiler explorer. They are nearly the same, but note the smaller prologue and epilogue in the new split_at version, apparently indicating less register pressure if nothing else.

example::clone_into_orig:pushr15pushr14pushr12pushrbxpushraxmovr14,rdxmovr12,rsimovr15,rdimovrbx, qword ptr [rdx+16]cmprbx,rsijb .LBB1_2mov qword ptr [r14+16],r12movrbx,r12.LBB1_2:testrbx,rbxje .LBB1_4movrdi, qword ptr [r14]leardx,[4*rbx]movrsi,r15call qword ptr [rip+ memcpy@GOTPCREL].LBB1_4:learsi,[r15+4*rbx]subr12,rbxmovrdi,r14movrdx,r12addrsp,8poprbxpopr12popr14popr15jmp alloc::vec::Vec<T>::extend_from_sliceexample::clone_into_split:pushr15pushr14pushrbxmovr14,rdxmovrbx,rsimovrsi,rdimovrdx, qword ptr [rdx+16]cmprdx,rbxjb .LBB2_2mov qword ptr [r14+16],rbxmovrdx,rbx.LBB2_2:lear15,[rsi+4*rdx]subrbx,rdxtestrdx,rdxje .LBB2_4movrdi, qword ptr [r14]shlrdx,2call qword ptr [rip+ memcpy@GOTPCREL].LBB2_4:movrdi,r14movrsi,r15movrdx,rbxpoprbxpopr14popr15jmp alloc::vec::Vec<T>::extend_from_slice

Centril · 2020-03-20T23:51:09Z

@bors try @rust-timer queue

rust-timer · 2020-03-20T23:51:10Z

Awaiting bors try build completion

bors · 2020-03-20T23:51:19Z

⌛ Trying commit 28791f6 with merge c462c8500772945a0f68a265dc1f8decf6b42e99...

bors · 2020-03-21T02:38:45Z

☀️ Try build successful - checks-azure
Build commit: c462c8500772945a0f68a265dc1f8decf6b42e99 (c462c8500772945a0f68a265dc1f8decf6b42e99)

rust-timer · 2020-03-21T02:38:47Z

Queued c462c8500772945a0f68a265dc1f8decf6b42e99 with parent 1057dc9, future comparison URL.

rust-timer · 2020-03-21T04:44:07Z

Finished benchmarking try commit c462c8500772945a0f68a265dc1f8decf6b42e99, comparison URL.

cuviper · 2020-03-21T16:11:46Z

Seems neutral to compiler performance. I think clone_into and clone_from are not used very often.

dtolnay

Nice!

dtolnay · 2020-03-21T17:06:09Z

@bors r+

bors · 2020-03-21T17:06:10Z

📌 Commit 28791f6 has been approved by dtolnay

Small tweaks in ToOwned::clone_into - `<[T]>::clone_into` is slightly more optimized. - `CStr::clone_into` is new, letting it reuse its allocation. - `OsStr::clone_into` now forwards to the underlying slice/`Vec`.

Dylan-DPC-zz · 2020-03-21T21:53:39Z

Failed in rollup #70237 (comment)

@bors r-

dtolnay · 2020-03-21T21:56:44Z

The new test in c_str.rs fails on Windows x86_64-msvc-1:

---- ffi::c_str::tests::test_c_str_clone_into stdout ----thread '<unnamed>' panicked at 'assertion failed: `(left == right)` left: `0x210c5e05800`, right: `0x210c5e05830`', src\libstd\ffi\c_str.rs:1527:9note: run with `RUST_BACKTRACE=1` environment variable to display a backtracefailures: ffi::c_str::tests::test_c_str_clone_intotest result: FAILED. 743 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out

cuviper · 2020-03-22T04:30:18Z

Hmm, maybe that test is assuming too much about the allocator. I could try it with equal lengths, but I'll confirm on Windows before I resubmit.

dtolnay · 2020-03-29T19:01:01Z

It could just be there is #[cfg(windows)] codepath that is missing a clone_into function.

It appears to codegen slightly more efficiently with `split_at` taking two slices at once, rather than slicing across different calls.

It can try to keep its allocation by converting the inner `Box` to `Vec`, using `clone_into` on the bytes, then convert back to `Box`.

cuviper · 2020-04-07T01:35:36Z

OK, I was able to confirm that failure on Windows, and now it passes when the strings are equal length.

dtolnay · 2020-04-07T01:40:21Z

@bors r+

bors · 2020-04-07T01:40:23Z

📌 Commit f854070 has been approved by dtolnay

Small tweaks in ToOwned::clone_into - `<[T]>::clone_into` is slightly more optimized. - `CStr::clone_into` is new, letting it reuse its allocation. - `OsStr::clone_into` now forwards to the underlying slice/`Vec`.

bors · 2020-04-07T11:36:57Z

⌛ Testing commit f854070 with merge 83177ff73c809ab897cbc450cf19aee95d48f94a...

Dylan-DPC-zz · 2020-04-07T13:43:25Z

@bors retry (yield)

@ghost

…ievink Rollup of 5 pull requests Successful merges: - rust-lang#70201 (Small tweaks in ToOwned::clone_into) - rust-lang#70762 (Miri leak check: memory reachable through globals is not leaked) - rust-lang#70846 (Keep codegen units unmerged when building compiler builtins) - rust-lang#70854 (Use assoc int submodules) - rust-lang#70857 (Don't import integer and float modules, use assoc consts 2) Failed merges: r? @ghost

bors · 2020-04-07T16:58:18Z

⌛ Testing commit f854070 with merge 42abbd8...

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang#155707 rust-lang#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

Rollup merge of #158807 - Vastargazing:tests/cstr-clone-into-alloc-error, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in #155707 #70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. #155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…ror, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang/rust#155707 rust-lang/rust#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang/rust#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

@RalfJung

…ror, r=RalfJung Add regression test for CString::clone_into unwind safety Regression test for the panic-safety fix in rust-lang/rust#155707 rust-lang/rust#70201 gave `<CStr as ToOwned>::clone_into` a path that moved the target `CString`'s buffer out (leaving it empty) before growing a `Vec`; if that growth's allocation failed and unwound, the target was left without its nul terminator, which is UB. rust-lang/rust#155707 fixed this but didn't add a test. The failing allocator here is only honored under Miri - in a normal build a `#[global_allocator]` in a library test doesn't intercept the reallocation inside `CString::clone_into` (it lives in libstd, linked `-C prefer-dynamic`). So the test passes as a regular test and does the real check under Miri, with the unwind assertion gated on `cfg!(miri)`; it runs under Miri in CI via the library-tests-under-Miri job. This started as rust-lang/miri#5157, but per @RalfJung a std regression test belongs with the standard library. r? @RalfJung

rust-highfive assigned dtolnay Mar 20, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 20, 2020

dtolnay approved these changes Mar 21, 2020
View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 21, 2020

Dylan-DPC-zz mentioned this pull request Mar 21, 2020
Rollup of 7 pull requests #70237
Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 21, 2020

joelpalmer added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 6, 2020

cuviper added 2 commits April 6, 2020 15:50

Use split_at in slice's ToOwned::clone_into
e8339e8
It appears to codegen slightly more efficiently with `split_at` taking two slices at once, rather than slicing across different calls.

Implement ToOwned::clone_into for CStr
b80fa76
It can try to keep its allocation by converting the inner `Box` to `Vec`, using `clone_into` on the bytes, then convert back to `Box`.

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 7, 2020

Centril mentioned this pull request Apr 7, 2020
Rollup of 6 pull requests #70878
Closed

Dylan-DPC-zz mentioned this pull request Apr 7, 2020
Rollup of 5 pull requests #70884
Merged

bors merged commit 795bc2c into rust-lang:masterApr 7, 2020

cuviper deleted the clone_into branch May 30, 2020 21:55

This was referenced Jul 4, 2026
Add Miri regression test for CString::clone_into unwind safety #158763
Closed
Add regression test for CString::clone_into unwind safety rust-lang/miri#5157
Closed
Add regression test for CString::clone_into unwind safety #158807
Merged

Uh oh!

Uh oh!

Conversation

cuviper commented Mar 20, 2020

Uh oh!

rust-highfive commented Mar 20, 2020

Uh oh!

cuviper commented Mar 20, 2020

Uh oh!

Centril commented Mar 20, 2020

Uh oh!

rust-timer commented Mar 20, 2020

Uh oh!

bors commented Mar 20, 2020

Uh oh!

bors commented Mar 21, 2020

Uh oh!

rust-timer commented Mar 21, 2020

Uh oh!

rust-timer commented Mar 21, 2020

Uh oh!

cuviper commented Mar 21, 2020

Uh oh!

dtolnay left a comment

Choose a reason for hiding this comment

Uh oh!

dtolnay commented Mar 21, 2020

Uh oh!

bors commented Mar 21, 2020

Uh oh!

Dylan-DPC-zz commented Mar 21, 2020

Uh oh!

dtolnay commented Mar 21, 2020

Uh oh!

cuviper commented Mar 22, 2020

Uh oh!

dtolnay commented Mar 29, 2020

Uh oh!

cuviper commented Apr 7, 2020

Uh oh!

dtolnay commented Apr 7, 2020

Uh oh!

bors commented Apr 7, 2020

Uh oh!

bors commented Apr 7, 2020

Uh oh!

Dylan-DPC-zz commented Apr 7, 2020

Uh oh!

bors commented Apr 7, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants