Skip to content

Per-call publish ordering: assign( f, std::memory_order ) - #10

Merged
psiha merged 1 commit into
feat/lock-free-empty-checkfrom
feat/per-call-publish-order
Aug 12, 2026
Merged

Per-call publish ordering: assign( f, std::memory_order )#10
psiha merged 1 commit into
feat/lock-free-empty-checkfrom
feat/per-call-publish-order

Conversation

@psiha

@psiha psiha commented Aug 8, 2026

Copy link
Copy Markdown
Owner

⚠️ Stacked on #9 — targets feat/lock-free-empty-check, not master. Review/merge #9 first; this diff is only the second commit.

Traits::concurrent_reads from #9 is a per-type decision, and a costlier one than it looks: it makes every access to the vtable pointer atomic, the invoke path included, which blocks the scalar replacement a non-escaping callable would otherwise get. A type used single-threaded in a thousand places, one of which needs the publish-once pattern, should not have to pay that everywhere.

So the ordering can now also be asked for per call, on any Traits:

f.assign( target, std::memory_order_release );   // arm
f.empty ( std::memory_order_acquire );           // probe

Shape

publish_order — an optional<memory_order> — threads down to the single publishing store. Unset (the default on every pre-existing path) keeps the Traits-driven behaviour, so nothing changes for callers that do not ask, and it constant-folds away.

The ordered accessors are now unconditional: asking for an ordering is itself the opt-in. What the trait still adds is that the type's own internal accesses are ordered too, so the guarantee holds by construction rather than by caller discipline. It stays the recommendation; the per-call form carries std::atomic_ref's own contract — while an ordered access is in flight, every conflicting access to that object must also be ordered.

std::atomic_ref cannot bind to the restrict-qualified member a non-concurrent_reads instantiation carries, so the accessors const_cast the qualifier away. That is sound — __restrict there qualifies the pointee (the vtable), not this pointer's own storage — and it lets everyone who never asks for an ordering keep __restrict.

The ordered assign ARMS an empty callable, once

This is the mechanism's own restriction, not an implementation limit. A release store publishes the writes that precede it. Re-arming an already-armed callable would publish over a target that a concurrent prober may have observed and be invoking right now — and the reassignment destroys it under that reader. No memory ordering closes that window; it needs deferred reclamation (refcounting, RCU grace periods, hazard pointers), which a callable does not and should not carry.

So the only race-free transition is empty → armed, once. Readers poll empty( acquire ) and, on observing engagement, may use the target for as long as it is never re-armed or cleared.

That falls out neatly in the implementation: the ordered assign takes the direct path — whose publication is exactly one store, and whose own precondition is this same emptiness (it skips pre-destruction because there is nothing to destroy). The ordinary route would not do: its general, may-throw case publishes through swap(), i.e. several plain stores, one of which hands a concurrent prober an engaged vtable with nothing ordering the target behind it. Not theoretical — it is what ThreadSanitizer reported when this was first written the obvious way.

…and for the same reason there is no ordered clear()

Disengagement has no preceding writes to publish, so an ordered clear would order nothing while reading as though it made disarming safe. It does not, for the reason above.

The Linux kernel reached the same conclusion for the identical shape: rcu_assign_pointer() degrades to a plain WRITE_ONCE() when publishing NULL, precisely because there is nothing to order (commit "rcu: No ordering for rcu_assign_pointer() of NULL") — and RCU then keeps removal on an entirely separate mechanism, grace periods, rather than pretending a store ordering can retire an object. Disarm here likewise requires the callable to be quiesced by other means, at which point plain clear() is exactly right.

A static_assert pins the deliberate absence of the overload.

Also

Restores the vertical alignment of the two data members, which #9 broke — via a type alias for the now-conditional vtable pointer.

Verification

  • clang-22 and gcc-16, Debug and Release — ctest green.
  • New PerCallPublishOrder tests: ordered/plain empty() agreement on non-opted-in Traits, a sizeof check that the opt-in costs the type nothing, and a publish-once case (4 polling readers × 64 rounds) on plain default_traits.
  • ThreadSanitizer-clean over repeated full-suite runs, and it discriminates: neutering the ordered store brings the reports straight back.

Pre-existing and unrelated: the two gcc-16 -O2 -DNDEBUG failures tracked from #8.

Traits::concurrent_reads is a per-TYPE decision, and a costly one: it makes
every access to the vtable pointer atomic, including the invoke path, which
blocks the scalar replacement a non-escaping callable would otherwise get.
A type used single-threaded in a thousand places, one of which needs the
publish-once pattern, should not have to pay that everywhere.

So the ordering can now also be requested per call, for any Traits:

    f.assign( target, std::memory_order_release );   // arm
    f.empty ( std::memory_order_acquire );           // probe

`publish_order` (an optional<memory_order>) threads down to the single
publishing store; unset - the default on every existing path - keeps the
Traits-driven behaviour, so nothing changes for callers that do not ask.
The ordered accessors themselves are unconditional now: asking for an
ordering IS the opt-in. What the trait still adds is that the type's own
internal accesses are ordered too, so the guarantee holds by construction
instead of by caller discipline - it remains the recommendation, and the
per-call form carries std::atomic_ref's own contract (while an ordered
access is in flight, every conflicting access must also be ordered).

Because the atomic accessors const_cast the member's qualification away
anyway - std::atomic_ref binds to neither cv nor __restrict, and both
concern the pointee rather than the pointer's own storage - the member
itself needs no conditional type and no `mutable`: it goes back to being
declared exactly as it always was, which also restores the vertical
alignment #9 disturbed. Non-opted-in codegen re-verified byte-identical to
master for a default_traits TU at -O3 -DNDEBUG.

The vtable accessors now speak in references. That separates two things the
pointer form conflated: `vtable_slot()` reads what the pointer HOLDS, which
mid-construction is legitimately not a vtable at all (debug_clear's
invalid_ptr) and is what the identity/validity assertions want; while
`load_vtable()` / `store_vtable()` deal in an actual vtable, so they cannot
be handed null.

The ordered assign ARMS an empty callable, once. That is the mechanism's own
restriction, not an implementation limit: a release store publishes the
writes preceding it, so re-arming would publish over a target a prober may
have observed and be invoking, and the reassignment destroys it under that
reader. Closing that needs deferred reclamation, not ordering. It is
therefore routed through the `direct` path, whose publication is exactly one
store and whose precondition is this same emptiness; the ordinary route's
may-throw case publishes through swap(), i.e. several plain stores, one of
which hands a prober an engaged vtable with nothing ordering the target
behind it (ThreadSanitizer confirms).

For the same reason there is no ordered clear(): disengagement has no
preceding writes to publish, so an ordered clear would order nothing while
reading as though it made disarming safe. The Linux kernel reached the same
conclusion for the identical shape - rcu_assign_pointer() degrades to a
plain WRITE_ONCE() when publishing NULL - and keeps removal on grace periods
rather than pretending a store ordering can retire an object.

Verified clang-22 and gcc-16, Debug and Release, ctest green; the
publish-once test on non-opted-in Traits is ThreadSanitizer-clean over
repeated runs and reports races when the ordered store is removed.
@psiha
psiha force-pushed the feat/per-call-publish-order branch from 66d779d to 230b362 Compare August 12, 2026 10:20
@psiha
psiha merged commit 6030146 into feat/lock-free-empty-check Aug 12, 2026
psiha added a commit that referenced this pull request Aug 12, 2026
Land the per-call publish ordering on master (re-target of #10)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant