std: Rewrite the sync module

[alexcrichton]

This commit is a reimplementation of std::sync to be based on the
system-provided primitives wherever possible. The previous implementation was
fundamentally built on top of channels, and as part of the runtime reform it has
become clear that this is not the level of abstraction that the standard level
should be providing. This rewrite aims to provide as thin of a shim as possible
on top of the system primitives in order to make them safe.

The overall interface of the std::sync module has in general not changed, but
there are a few important distinctions, highlighted below:

• The condition variable type, Condvar, has been separated out of a Mutex.
  A condition variable is now an entirely separate type. This separation
  benefits users who only use one mutex, and provides a clearer distinction of
  who's responsible for managing condition variables (the application).

• All of Condvar, Mutex, and RWLock are now directly built on top of
  system primitives rather than using a custom implementation. The Once,
  Barrier, and Semaphore types are still built upon these abstractions of
  the system primitives.

• The Condvar, Mutex, and RWLock types all have a new static type and
  constant initializer corresponding to them. These are provided primarily for C
  FFI interoperation, but are often useful to otherwise simply have a global
  lock. The types, however, will leak memory unless destroy() is called on
  them, which is clearly documented.

• The fundamental architecture of this design is to provide two separate layers.
  The first layer is that exposed by sys_common which is a cross-platform
  bare-metal abstraction of the system synchronization primitives. No attempt is
  made at making this layer safe, and it is quite unsafe to use! It is currently
  not exported as part of the API of the standard library, but the stabilization
  of the sys module will ensure that these will be exposed in time. The
  purpose of this layer is to provide the core cross-platform abstractions if
  necessary to implementors.

  The second layer is the layer provided by std::sync which is intended to be
  the thinnest possible layer on top of sys_common which is entirely safe to
  use. There are a few concerns which need to be addressed when making these
  system primitives safe:

  • Once used, the OS primitives can never be moved. This means that they
    essentially need to have a stable address. The static primitives use
    &'static self to enforce this, and the non-static primitives all use a
    Box to provide this guarantee.
  • Poisoning is leveraged to ensure that invalid data is not accessible from
    other tasks after one has panicked.

  In addition to these overall blanket safety limitations, each primitive has a
  few restrictions of its own:

  • Mutexes and rwlocks can only be unlocked from the same thread that they
    were locked by. This is achieved through RAII lock guards which cannot be
    sent across threads.
  • Mutexes and rwlocks can only be unlocked if they were previously locked.
    This is achieved by not exposing an unlocking method.
  • A condition variable can only be waited on with a locked mutex. This is
    achieved by requiring a MutexGuard in the wait() method.
  • A condition variable cannot be used concurrently with more than one mutex.
    This is guaranteed by dynamically binding a condition variable to
    precisely one mutex for its entire lifecycle. This restriction may be able
    to be relaxed in the future (a mutex is unbound when no threads are
    waiting on the condvar), but for now it is sufficient to guarantee safety.

• Condvars support timeouts for their blocking operations. The
  implementation for these operations is provided by the system.

Due to the modification of the Condvar API, removal of the std::sync::mutex
API, and reimplementation, this is a breaking change. Most code should be fairly
easy to port using the examples in the documentation of these primitives.

[breaking-change]

Closes#17094
Closes#18003

[rust-highfive]

Warning

• These commits modify unsafe code. Please review it carefully!

[alexcrichton]

Note that this is duplicating a lot of implementation details of librustrt and libsync as of this red hot minute. Once these two libs are merged into std I'll delete all the duplication and make sure that we've only got one copy of the definitions. This PR is meant to reflect the final state of the stdlib for std::sync and its implementation details.

[sfackler]

Condvars and RWLocks now support timeouts for their blocking operations. The
implementation for these operations is provided by the system.

Woo!

[sfackler]

Might want to qualify this a bit - it's not really possible in general to unblock the thread after exactly the right duration.

[sfackler]

I didn't see any timeout stuff in the RWLock impl, btw

[alexcrichton]

Ah yes sorry, I mis-remembered what was added to rwlocks. RWLocks now support try_read and try_write, not timeouts just yet. I'd like to provide a unix extension trait to provide these functions, but Windows at least does not implement locking with a timeout.

[alexcrichton]

One thing I also just remembered, this removes the ability to use a condition variable with a RWLock for now. We can build something such as C++'s condition_variable_any in the future, but this PR is just aimed at exposing the system primitives for now.

[alexcrichton]

Now that #19255 has merged I've rebased on top and removed all implementations of primitives based on channels. At the same time I have also remove all concurrent queues from the public interface as I don't think we're going to be able to stabilize them before 1.0. Two are kept as implementation details of channels, but the chase-lev deque and mpmc queue have been entirely removed.

[aturon]

Why is this second comparison needed?

[alexcrichton]

This is mostly just copy-pasted from the existing implementation, but I believe the first comparison is to determine whether this was the final thread (the notifier) or a thread which needs to wait. The second comparison is redundant the first iteration of the loop, but all other iterations it's intended to help with spurious wakeups.

[aturon]

"tasks have"

[aturon]

OK, I've finished reading this over and it looks great! (I didn't scrutinize the OS bindings.)

I think there are a lot of questions going forward about the organization of this module, static variants, and additional system bindings, but this looks like a great step forward. I'm happy for it to land for now, and then we can talk through exactly what we want to stabilize a bit later.

r=me after nits and a rebase.

[sfackler]

Needs a rebase

[nodakai]

This rx.recv() is nice because the old version couldn't catch a possible assertion failure in the spawned proc. Can we perhaps define a macro to standardize this pattern and to make sure there's no longer any instances of meaningless assert_eq!?

[alexcrichton]

This may be somewhat difficult to refactor out into a standard pattern, but for now I'm going to focus on landing this to unblock some more runtime removal.

[thestinger]

This isn't correct. You need to use the monotonic clock support. The system time can and will often change.

[thestinger]

http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_getclock.html

[nodakai]

@thestinger Please remember that Linux CLOCK_MONOTONIC is susceptible to machine hibernation during which TSC doesn't count up. We would have to wait for futex(2) to support CLOCK_BOOTTIME for truly correct timeout operation. So IMHO it's acceptable to live with this code meantime (unless we don't forget that we're repeating the same bugs found in Java bug tracker for long...)

[alexcrichton]

I don't personally want to debate these details on this PR as this is blocking more runtime removal and reform. I would like to prioritize pushing through this rewrite as much as possible so the wait_timeout functions are now non-public with comments pointing at this PR to review the implementation before making them public.

[thestinger]

It's painful (but possible) to do timeout support properly on Windows. It should be left out if it's not going to be done correctly because having incorrect implementations is worse than not having the feature.