Add functions core::ptr::dangling/_mut<T>() -> *const/*mut T

[bluss]

Add functions to create a dangling aligned non-null raw pointer. The
use case is to fix all the places we use "1 as *mut T", in
particular in the slice iterators.

The aligned pointer has a value like 0x1, 0x4, 0x8 etc. depending on the
alignment of the pointed-to type. It is useful when a non-null pointer
is required like in references, slices, boxes, and other places.

NOTE: This changes the observable behaviour of slice iterators.
Previously, it would always yield 0x1 pointers (as references) for ZST,
including types like [SomeType; 0]. They now use an aligned non-null
pointer.

Expose std::ptr::dangling/_mut as new unstable functions with intent
on later stabilization, because if this low level trick is needed in
collections, the rest of the Rust ecosystem will need them as well.

[rust-highfive]

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

[bluss]

Choices, choices.

1. We could have two versions, one for const and one for mut? I don't know if we should bother with that.
2. Should it be a static method on raw pointers instead of a free function?

[Gankra]

This seem fine. API should exactly match ptr::null(_mut), imo.

[petrochenkov]

empty sounds too benign, IMO, almost like default or null, as if it were something valid and not a hack to workaround non-null/alignment attributes.

[bluss]

@petrochenkov I see that, maybe "dangling" is better? empty follows the existing Unique/Shared method names.

[petrochenkov]

@bluss
invalid_nonnull_aligned? 😄

[bluss]

Is std::ptr::nonnull() too confusing? It could std::ptr::dummy() instead, for the dummiest but simplest name.

[Gankra]

I vote dangling if we can't use empty. It's clearly !!!scary!!! but easy to learn and remember.

[Mark-Simulacrum]

Marking as relnotes due to the potential breakage with ZST iterators.

[bluss]

Rewritten to use std::ptr::dangling/_mut.

[kennytm]

Can this function be const, just like null()?

[bluss]

Ah great idea, align_of is const so it should be no problem.

[bluss]

Renaming Unique::empty() / Shared::empty() to the new "dangling" names should probably happen as well, but it could be another PR?

[bors]

☔ The latest upstream changes (presumably #45285) made this pull request unmergeable. Please resolve the merge conflicts.

[durka]

Please add #[rustc_const_unstable] attributes (we should add this to the unmarked-api lint).

[durka]

Or maybe we can hold off adding those until the functions themselves are stabilized?

[nagisa]

nit: This is not necessarily true. For types with very large alignments, this may point to outside the 0th page (the page that’s guaranteed to be unmapped on most systems) and actually point to something (e.g. manually mapped 1st page). I would reword this to not mention value validity and rather mention that dereferencing the pointer is invalid, or something along the lines.

[bluss]

My first draft was that dereferencing the pointer is invalid. While that's true in the "read memory" sense, in the higher level Rust semantic, dereferencing a pointer to ZST is fine.

[bluss]

😄 I'm having trouble finding the right words that navigate all of this.

[bluss]

I think any integer_constant as *const T pointer is invalid to use by these rules, since it's not based on anything http://llvm.org/docs/LangRef.html#pointer-aliasing-rules

So it must be treated as "truly dangling" or use some highly implementation defined way to access an actual value through that pointer.

[bluss]

In that sense, it seems best to just say it does not point to any value. If it's equal to a bona fide pointer to a value, that's incidental?

[nagisa]

I think any integer_constant as *const T pointer is invalid to use by these rules

This isn’t exactly true, because pretty much any low level code that depends on stuff at well known addresses would be invalid (embedded/low level/OS code relies heavily on that, for example). I believe this rule from the page you’ve linked applies here:

An integer constant other than zero or a pointer value returned from a function not defined within LLVM may be associated with address ranges allocated through mechanisms other than those provided by LLVM. Such ranges shall not overlap with any ranges of addresses allocated by mechanisms provided by LLVM.

This rule does not forbid creation and use of such address ranges, it only forbids overlap with address ranges generated by concepts understood by LLVM (alloca, malloc, ...).

[bluss]

That makes sense. I have pushed a change that has no Safety section, because I have no useful advice apart from what's already in the other doc.

[bluss]

Updated to simplify docs after discussion with @nagisa. We can't say so much more than non-null and aligned about the return value.

[nagisa]

I have reviewed the functional changes, but am uncomfortable taking a decision whether the breakage this potentially causes is acceptable. I feel that it is, but people "higher up" might feel otherwise.

[bluss]

Box of Zst has already made the corresponding breaking change, and this catches up with that in the slice iterator.

[Gankra]

Yeah when I changed Vec I think I shook the ecosystem loose from expecting the old heap::EMPTY (they now check <= alignof).

[shepmaster]

Randomly reassigning to...

r? @alexcrichton

[alexcrichton]

I think I may not be quite up to speed on this area, but mind explaining again the motivation behind these functions? For example is the current iteration today incorrect? Memory unsafe?

[bluss]

Shared/Unique (So Rc, Box, Vec etc) had the corresponding change of pointer value in #41064. @gankro would know most.

My idea is that this is something we're gradually making more correct so that we end up more consistent. For example that even ZST pointer values are aligned. As usual, it's hard to find any memory safety issues with any pointer to ZST.

I think that if Box<[T; 0]> has a pointer that is aligned for [T; 0], then std::slice::Iter<[T; 0]> should emit references (&[T; 0]) that match that. This PR changes the slice iterator in that way.

[alexcrichton]

I feel like internal changes are fine, but I'm just wary to increase the API surface area of the standard library because it "makes us feel good" rather than having a technical driver under the hood?

[scottmcm]

For example is the current iteration today incorrect? Memory unsafe?

I think that, technically, we're currently relying on UB. This code

unsafe{ std::ptr::read(1as*const[i64;0]);}

gets trans'd as

callvoid@llvm.memcpy.p0i8.p0i8.i64(i8*%3, i8*%4, i640, i328, i1false)

specifying an alignment to the llvm intrinsic that's a lie.

Probably the pass ordering means that the 0-byte memcpy is DCE'd before any kind of value propagation can assume the low three bits of the pointer are zero, but I'm pretty sure LLVM would be allowed to replace this code with unreachable since "if the call to this intrinsic has an alignment value that is not 0 or 1, then the caller guarantees that both the source and destination pointers are aligned to that boundary".

[Gankra]

It can be considered a bug that accesses to ZSTs ever make it to llvm.

[nagisa]

Why? LLVM documentation is phrased in a way that does not invalidate loading a ZST type through any pointer. Sure there's some inefficiency in the IR we generate as they are not strictly necessary, but I wouldn't call it a bug.

…

On Nov 15, 2017 00:37, "Alexis Beingessner" ***@***.***> wrote: It can be considered a bug that accesses to ZSTs *ever* make it to llvm. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45527 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AApc0oEDA_72wPikxd7HqfwgZ7xA4kpwks5s2hYggaJpZM4QGVzA> .

[alexcrichton]

@scottmcm thanks for the info! That definitely sounds bad and like possible UB. I agree with @gankro though in that we probably shouldn't be generating that in the first place?

In that sense I'm still a little unclear on the motivation here in the sense that there doesn't seem to be a fundamental technical reason for why we would want such functions.

[bluss]

Moving towards aligned references is something we should do, now that it has begun. The soft thing about that is that it's a gradual process, tightening the screws in various places before we are completely in shape.

The motivation for the functions was the rather generic one: Expose std::ptr::dangling/_mut as new unstable functions with intent on later stabilization, because if this low level trick is needed in collections, the rest of the Rust ecosystem will need them as well. I sounds like it's too much of a bother to do it that way. In this case we can easily abstain since users can reimplement the function instead.

[scottmcm]

I do think it makes sense for the rule to be "reading through a pointer requires that it's aligned", without a "unless the size is zero" exception. And whether the memcpy is right or not, it feels like a ZST load ought to trans as something, if only an assume(p != 0).

Whether there's a need for these methods to be stabilized is a fair question, though. Something similar to dangling() can even be done as [].as_ptr() these days...

[alexcrichton]

@bluss I'm afraid that doesn't answer my question though? I still don't know why we want to move towards aligned raw pointers in the first place?

@scottmcm but this still doesn't feel like a technical argument in favor of more API surface area in the standard library? It would seem natural to me that a ZST does absolutely nothing in trans (no instructions emitted) so in that sense I'm still unsure on the motivation to add new stable surface area to the standard library.

[bluss]

@alexcrichton I see. It's not about raw pointers, it's to be used by all the other ones (Unique, Shared, &, etc).

[alexcrichton]

Ok I think that makes sense to me! So we'll be handing out safe references from those types and those require valid alignment? (in that &T should always be aligned for any T).

Right now though you can't pull a "safe" reference without using unsafe, though?

[bors]

☔ The latest upstream changes (presumably #45039) made this pull request unmergeable. Please resolve the merge conflicts.

[bluss]

(Rebased & bonus squashing into logical parts)

[bluss]

Closing with intent on picking up again after the slice iterator implementation change is worked out. (#46223)