Per-call publish ordering: assign( f, std::memory_order ) - #10
Merged
Conversation
psiha
force-pushed
the
feat/per-call-publish-order
branch
from
August 12, 2026 10:01
5437c83 to
66d779d
Compare
Traits::concurrent_reads is a per-TYPE decision, and a costly one: it makes
every access to the vtable pointer atomic, including the invoke path, which
blocks the scalar replacement a non-escaping callable would otherwise get.
A type used single-threaded in a thousand places, one of which needs the
publish-once pattern, should not have to pay that everywhere.
So the ordering can now also be requested per call, for any Traits:
f.assign( target, std::memory_order_release ); // arm
f.empty ( std::memory_order_acquire ); // probe
`publish_order` (an optional<memory_order>) threads down to the single
publishing store; unset - the default on every existing path - keeps the
Traits-driven behaviour, so nothing changes for callers that do not ask.
The ordered accessors themselves are unconditional now: asking for an
ordering IS the opt-in. What the trait still adds is that the type's own
internal accesses are ordered too, so the guarantee holds by construction
instead of by caller discipline - it remains the recommendation, and the
per-call form carries std::atomic_ref's own contract (while an ordered
access is in flight, every conflicting access must also be ordered).
Because the atomic accessors const_cast the member's qualification away
anyway - std::atomic_ref binds to neither cv nor __restrict, and both
concern the pointee rather than the pointer's own storage - the member
itself needs no conditional type and no `mutable`: it goes back to being
declared exactly as it always was, which also restores the vertical
alignment #9 disturbed. Non-opted-in codegen re-verified byte-identical to
master for a default_traits TU at -O3 -DNDEBUG.
The vtable accessors now speak in references. That separates two things the
pointer form conflated: `vtable_slot()` reads what the pointer HOLDS, which
mid-construction is legitimately not a vtable at all (debug_clear's
invalid_ptr) and is what the identity/validity assertions want; while
`load_vtable()` / `store_vtable()` deal in an actual vtable, so they cannot
be handed null.
The ordered assign ARMS an empty callable, once. That is the mechanism's own
restriction, not an implementation limit: a release store publishes the
writes preceding it, so re-arming would publish over a target a prober may
have observed and be invoking, and the reassignment destroys it under that
reader. Closing that needs deferred reclamation, not ordering. It is
therefore routed through the `direct` path, whose publication is exactly one
store and whose precondition is this same emptiness; the ordinary route's
may-throw case publishes through swap(), i.e. several plain stores, one of
which hands a prober an engaged vtable with nothing ordering the target
behind it (ThreadSanitizer confirms).
For the same reason there is no ordered clear(): disengagement has no
preceding writes to publish, so an ordered clear would order nothing while
reading as though it made disarming safe. The Linux kernel reached the same
conclusion for the identical shape - rcu_assign_pointer() degrades to a
plain WRITE_ONCE() when publishing NULL - and keeps removal on grace periods
rather than pretending a store ordering can retire an object.
Verified clang-22 and gcc-16, Debug and Release, ctest green; the
publish-once test on non-opted-in Traits is ThreadSanitizer-clean over
repeated runs and reports races when the ordered store is removed.
psiha
force-pushed
the
feat/per-call-publish-order
branch
from
August 12, 2026 10:20
66d779d to
230b362
Compare
psiha
added a commit
that referenced
this pull request
Aug 12, 2026
Land the per-call publish ordering on master (re-target of #10)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Traits::concurrent_readsfrom #9 is a per-type decision, and a costlier one than it looks: it makes every access to the vtable pointer atomic, the invoke path included, which blocks the scalar replacement a non-escaping callable would otherwise get. A type used single-threaded in a thousand places, one of which needs the publish-once pattern, should not have to pay that everywhere.So the ordering can now also be asked for per call, on any
Traits:Shape
publish_order— anoptional<memory_order>— threads down to the single publishing store. Unset (the default on every pre-existing path) keeps theTraits-driven behaviour, so nothing changes for callers that do not ask, and it constant-folds away.The ordered accessors are now unconditional: asking for an ordering is itself the opt-in. What the trait still adds is that the type's own internal accesses are ordered too, so the guarantee holds by construction rather than by caller discipline. It stays the recommendation; the per-call form carries
std::atomic_ref's own contract — while an ordered access is in flight, every conflicting access to that object must also be ordered.std::atomic_refcannot bind to the restrict-qualified member a non-concurrent_readsinstantiation carries, so the accessorsconst_castthe qualifier away. That is sound —__restrictthere qualifies the pointee (the vtable), not this pointer's own storage — and it lets everyone who never asks for an ordering keep__restrict.The ordered assign ARMS an empty callable, once
This is the mechanism's own restriction, not an implementation limit. A release store publishes the writes that precede it. Re-arming an already-armed callable would publish over a target that a concurrent prober may have observed and be invoking right now — and the reassignment destroys it under that reader. No memory ordering closes that window; it needs deferred reclamation (refcounting, RCU grace periods, hazard pointers), which a callable does not and should not carry.
So the only race-free transition is empty → armed, once. Readers poll
empty( acquire )and, on observing engagement, may use the target for as long as it is never re-armed or cleared.That falls out neatly in the implementation: the ordered assign takes the
directpath — whose publication is exactly one store, and whose own precondition is this same emptiness (it skips pre-destruction because there is nothing to destroy). The ordinary route would not do: its general, may-throw case publishes throughswap(), i.e. several plain stores, one of which hands a concurrent prober an engaged vtable with nothing ordering the target behind it. Not theoretical — it is what ThreadSanitizer reported when this was first written the obvious way.…and for the same reason there is no ordered
clear()Disengagement has no preceding writes to publish, so an ordered clear would order nothing while reading as though it made disarming safe. It does not, for the reason above.
The Linux kernel reached the same conclusion for the identical shape:
rcu_assign_pointer()degrades to a plainWRITE_ONCE()when publishingNULL, precisely because there is nothing to order (commit "rcu: No ordering forrcu_assign_pointer()of NULL") — and RCU then keeps removal on an entirely separate mechanism, grace periods, rather than pretending a store ordering can retire an object. Disarm here likewise requires the callable to be quiesced by other means, at which point plainclear()is exactly right.A
static_assertpins the deliberate absence of the overload.Also
Restores the vertical alignment of the two data members, which #9 broke — via a type alias for the now-conditional vtable pointer.
Verification
PerCallPublishOrdertests: ordered/plainempty()agreement on non-opted-inTraits, asizeofcheck that the opt-in costs the type nothing, and a publish-once case (4 polling readers × 64 rounds) on plaindefault_traits.Pre-existing and unrelated: the two gcc-16
-O2 -DNDEBUGfailures tracked from #8.