Add a generic CAS loop to std::sync::Atomic*

[llogiq]

This adds two new methods to both AtomicIsize and AtomicUsize with optimized safe compare-and-set loops, so users will no longer need to write their own, except in very strange circumstances.

update_and_fetch will apply the function and return its result, whereas fetch_and_update will apply the function and return the previous value.

This solves #48384 with x.update_and_fetch(|x| x.max(y)). It also relates to #48655 (which I misuse as tracking issue for now)..

note This might need a crater run because the functions could clash with third party extension traits.

[rust-highfive]

r? @kennytm

(rust_highfive has picked a reviewer for you, use r? to override)

[ranma42]

atömic?

[kennytm]

This is a heavy metal atom 😆

[llogiq]

I swear it was a typo.

[ranma42]

atömic?

[kennytm]

Capitalize the F, and also use the singular form.

Fetches the value, applies a function to it and returns the result.

[kennytm]

Examples should be a section header, followed by a summary what the example does.

/// # Examples
///
/// Compute the maximum of a slice, where each update step is done atomically.

[kennytm]

Is there any chance the user wants to use ordering other than SeqCst/Relaxed? (Similar question for the self.get(Relaxed) above)

[nagisa]

Definitely yes. Current implementation seems to be always right, however it might end up doing more loops than strictly necessary if some stricter ordering was used for get… I think. And SeqCst isn’t always necessary for CEW, but once you go for a weaker ordering there, it is important to be able to specify a stronger one for get and failure cases.

[llogiq]

@nagisa Benchmarks in other languages show that using the most relaxed ordering and allowing for more loops achieves for better throughput. However, as this is not other languages, I'll see if I can come up with some benchmarks for it.

[nagisa]

@llogiq it is difficult to generalise the benchmarks like this. The article you’ve linked to is benchmarking on x86, which will have different performance characteristics compared to an architecture where the memory model is weaker. This is exactly why we provide means to specify the orderings everywhere.

[nagisa]

Furthermore, I couldn’t manage to convince LLVM to generate as good a code with manual CAS loop compared to just calling the atomic_max intrinsic.

Here’s the code for the atomic_max intrinsic for the ARMv7 architecture:

	dmb	ish.LBB1_1:	ldrexd	r4, r5,[r0]	subs	r1, r2, r4	sbcs	r1, r3, r5mov	r1, #0	movwlt	r1, #1cmp	r1, #0mov	r1, r3	moveq	r4, r2	movne	r1, r5mov	r5, r1	strexd	r1, r4, r5,[r0]cmp	r1, #0	bne	.LBB1_1	dmb	ish

and something that gets generated when implementing the same-ish functionality using the CAS loop:

Details

push	{r4, r5, r6, r7,r8,r9,r10,r11, lr}	ldrd	r8,r9,[r0]	ldrexd	r10,r11,[r0]	eor	r1,r11,r9	eor	r7,r10,r8	orr	r5, r7, r1	subs	r7, r2,r8	sbcs	r7, r3,r9mov	r1, #0mov	r7, #0	movwlo	r7, #1cmp	r7, #0mov	r7, r3	movne	r7,r9mov	r6, r2	movne	r6,r8cmp	r5, #0	bne	.LBB2_4	dmb	ish.LBB2_2:	strexd	r5, r6, r7,[r0]cmp	r5, #0	beq	.LBB2_6	ldrexd	r10,r11,[r0]	eor	r5,r10,r8	eor	r4,r11,r9	orrs	r5, r5, r4	beq	.LBB2_2.LBB2_4:	clrexcmp	r1, #0	beq	.LBB2_8.LBB2_5:pop	{r4, r5, r6, r7,r8,r9,r10,r11, pc}.LBB2_6:	dmb	ishmov	r1, #1cmp	r1, #0	beq	.LBB2_8	b	.LBB2_5.LBB2_7:movr10, r6movr11, r7cmp	r1, #0	bne	.LBB2_5.LBB2_8:	ldrexd	r6, r7,[r0]movr9, r3	eor	r1, r6,r10	eor	r5, r7,r11	orr	r1, r1, r5	subs	r5, r2,r10	sbcs	r5, r3,r11mov	r5, #0	movwlo	r5, #1cmp	r5, #0	movne	r9,r11movr8, r2	movne	r8,r10cmp	r1, #0	bne	.LBB2_12	dmb	ish.LBB2_10:	strexd	r1,r8,r9,[r0]cmp	r1, #0	beq	.LBB2_13	ldrexd	r6, r7,[r0]	eor	r1, r6,r10	eor	r4, r7,r11	orrs	r1, r1, r4	beq	.LBB2_10.LBB2_12:mov	r1, #0	clrex	b	.LBB2_7.LBB2_13:	dmb	ishmov	r1, #1	b	.LBB2_7

---

So it seems to me that it would be worthwhile to add a binding for such a function anyway.

[0xAX]

s/prvious/previous

[emilyalbini]

Ping from triage @llogiq! Could you finish this PR so it can be merged?

[llogiq]

I'm a bit unsure how to progress:

• Should I add the Orderings for each operation as arguments? Or unify the load orderings?
• Should I add the atomic_min / atomic_max intrinsics, too?

[nagisa]

I think we should add two ordering arguments -- one for CAS success case and the other for for failure case (also used for initial load). We should expose the min and max methods separately as well IMO.

Also cc @Amanieu, who may have opinions about this stuff.

[kennytm]

At the very least one should be able to choose an ordering other than SeqCst. There are three orderings involved here (load, compare_exchange_weak's success + failure). If one can prove that any one can be always Relaxed or any two ordering must be equivalent, then we don't need to expose that ordering to the API.

[Amanieu]

@kennytm The ordering for the initial load and for the CAS failure are the same: in both cases this is used to load a value for the next loop iteration.

For the API itself, I think something like this would be better:

fntransaction<F:FnMut($int_type) -> Option<$int_type>>(&self,f:F,success:Ordering,failure:Ordering) -> bool;

The closure can return None, which aborts the operation: the value of the atomic is not changed.

[Amanieu]

I'll also leave this as an example of prior work (in C++).

[Amanieu]

@llogiq I'm not convinced that we need both fetch_and_update and update_and_fetch, hence my suggestion above to only have a single method (transaction). In both cases the old/new value can easily be extracted from the closure, so I think that the previous value should be returned for consistency with the other atomic operations.

[llogiq]

@Amanieu in that case I think it'd be prudent to have an example of extracting the new value from the closure. How would you do that? Setting a mutable binding outside? We also should look at the generated assembly for that case.

[Amanieu]

Yes, basically something like this:

letmut new = 0;ATOMIC.fetch_and_update(|x| {new = x + 1; new},Ordering::Relaxed,Ordering::Relaxed);

You need to replace Fn with FnMut in the function definition, there's no reason not to support FnMut here.

(As a side note, I'm still not a big fan of the fetch_and_update name, have you considered transaction like I suggested above?)

[llogiq]

I'll change the name when I remove update_and_fetch. But I'll be offline for a few hours first.

[kennytm]

I strongly oppose to the name transaction, it tells nothing about the intended behavior of the function without looking up the doc. Please keep the name fetch_and_update (or shorten it to fetch_update), it is totally fine.

[llogiq]

I disagree that transaction doesn't tell the intended behavior of the function (though the interface of the C++ method of the same name returns bool for some reason), but to keep with the naming scheme of the Atomic* type methods, fetch_update would be preferrable (fetch_with would be another option, but that's not as telling. Perhaps fetch_modifyor fetch_change?). I'm open for suggestions.

[Amanieu]

After some consideration, I'm OK with the name fetch_update, since it is consistent with the other method names.

I went through a few of the CAS loops in my various crates, and the majority of them are of this form:

loop{if some_condition {returnfalse;}match atomic.compare_exchange_weak(
oldval,
newval,Ordering::Relaxed,Ordering::Relaxed,){Ok(_) => returntrue,Err(x) => oldval = x,}}

I therefore think that we should make the closure return an Option<$int_type> where a None will simply break out of the loop without changing the underlying value (this is very similar to the behavior of compare_exchange).

[llogiq]

So the fndecl would be pub fn fetch_update<F>(&self, f: F) ->$int_type where F: FnMut($int_type) -> Option<$int_type>;, right? Returning the previous value in all cases & breaking the loop if f returns None?

[Amanieu]

@llogiq That's correct.

I'm somewhat torn on the return type of fetch_update itself. Should we unconditionally return the previous value or should we wrap it in a Result like the return value of compare_exchange?

[llogiq]

So you'd return either Ok(old_value) or Err(old_value)?

[Amanieu]

Yes

[llogiq]

Rebased and squashed.

@kennytm r?

[kennytm]

@llogiq The fetch_update method is gone?

[llogiq]

It appears I lost it during the rebase. I'll re-add it.

[llogiq]

Done. Let's see if Travis likes it.

[llogiq]

Besides fixing some typos, I had to remove the random number generator, as it doesn't work with types below 32 bits.

[llogiq]

Now with the fetch_update method back, @kennytm r? (also thanks for bearing with me for so long :slight_smile:)

[kennytm]

It will however only apply the function once to the value stored within.

I don't get what does this mean. Could be clarify this in the code?

[llogiq]

I wanted to say that the function will be applied only once per observed value. I've changed the text accordingly.

[Amanieu]

Since you are using compare_exchange_weak, it could actually be applied more than once even if the value has not changed. For example on ARM, a compare_exchange_weak may spuriously fail due to an interrupt.

[llogiq]

That is true. Ok, I've changed the text once more to say that the function will only have been applied once to the stored value. Otherwise people may fear they need to make the function idempotent.

[kennytm]

I suggest we pick a different feature name to fetch_{max, min} (e.g. atomic_min_max). We may have different stabilization schedule between fetch_{min, max} and fetch_update.

[llogiq]

Done.

[llogiq]

Now we have the split feature gate and a better wording for the comments. Travis also seems to like it. r?

[kennytm]

@bors r+

[bors]

📌 Commit 0f5e419 has been approved by kennytm