OSC-SM: Notified RMA Implementation with Set notify and Bounds

[joe-explr]

Implements the MPI-5.1 notified communication chapter (§12.6) for the
osc/sm one-sided component, together with the framework plumbing, C
bindings, window attributes, and error class the feature requires.

Notified RMA lets an origin attach a notification to an RMA operation:
the target's notification counter is incremented only after the data
movement has completed at the target, so the target can poll a counter
instead of participating in a synchronization epoch.

Public MPI interface

• 8 notified data-movement operations, each in both small-count and
  bigcount _c form (16 entry points): MPI_Put_notify,
  MPI_Get_notify, MPI_Accumulate_notify,
  MPI_Get_accumulate_notify, and the request-returning
  MPI_Rput_notify, MPI_Rget_notify, MPI_Raccumulate_notify,
  MPI_Rget_accumulate_notify. Each takes an additional
  int notification_idx immediately before win.
• 4 notification-management procedures: MPI_Win_set_num_notify,
  MPI_Win_get_num_notify, MPI_Win_get_notify_value,
  MPI_Win_reset_notify_value.
• New error classMPI_ERR_RMA_NOTIFICATION (value 83), registered
  in ompi/errhandler/errcode.c and ompi/include/mpif-values.py.
• 3 predefined window attributes — MPI_WIN_NOTIFICATION_NUM_SB,
  MPI_WIN_NOTIFICATION_NUM_UB, MPI_WIN_NOTIFICATION_VALUE_UB
  (keyvals 13–15), appended after the existing MPI_WIN_* values so
  that no established attribute value shifts.

C bindings

12 new .c.in generator templates under ompi/mpi/c/ (one per notified
operation and window procedure), wired into ompi/mpi/c/Makefile.am,
with MPI_ERR_RMA_NOTIFICATION added to
ompi/mpi/bindings/ompi_bindings/consts.py. No generated output is
hand-edited.

osc framework

ompi/mca/osc/osc.h gains the 8 notified operation slots plus 5 module
function pointers: osc_win_get_notify_value,
osc_win_reset_notify_value, osc_win_set_num_notify,
osc_win_get_num_notify, and osc_win_get_notify_bounds. A component
that does not implement notified communication leaves them NULL.

Window setup

The three notification attributes are cached on every window at
creation — querying osc_win_get_notify_bounds when the component
supplies it, and caching zeros when it does not, which is an honest
report that no notification counter can be attached to such a window.

MPI_WIN_NOTIFICATION_VALUE_UB is stored via ompi_attr_set_aint
because the attribute machinery has no MPI_Count slot and every other
predefined attribute is integer- or address-valued. This is safe while
MPI_Aint tracks the pointer width; a comment in ompi/win/win.c
records that a 32-bit revival would need a real MPI_Count slot in
attribute_value_t.

osc/sm implementation

• Notification counters are int64_t reserved inline in the main shared
  segment (16 per rank by default, tunable via the
  osc_sm_num_notify_counters MCA parameter), reached through a
  per-process notify_bases[] pointer array so that each notified
  operation costs a single indexed load regardless of where the counters
  currently live.
• Three distinct per-rank quantities are tracked in node_states[]: the
  attached count, the reserved capacity, and the segment
  offset. The last is an offset rather than a pointer because the
  shared segment is mapped at a different address in every process.
• MPI_Win_set_num_notifygrows on demand into a separately
  allocated overflow segment: allgather the requests so the grow/no-grow
  decision is identical everywhere, bcast the new segment descriptor (an
  empty seg_name signals failure so that no process hangs in a later
  collective), allreduce on attach success before any shared state is
  mutated, raise the attached count only once the space behind it
  exists, barrier, then unlink and drop the old mapping. The allocation
  never shrinks.
• The mpi_assert_max_num_notify info key turns the reservation into a
  hard cap rather than a trigger for reallocation, and is reported back
  through MPI_Win_get_info only when one was actually given.
• Memory ordering is chosen per operation class. Put uses a
  store-store fence; get and the whole accumulate family use a full
  barrier, because there the notification asserts that a read of the
  target window happened — a load-before-store constraint that a
  one-sided fence cannot express. opal_atomic_add() is relaxed and
  contributes no ordering of its own.
• The notification index is validated against the target's attached
  count before any data moves, so an erroneous call cannot leave the
  target window (or, for get, the origin buffer) modified before the
  error is reported.
• Accumulate-family operations notify only on success, so a counter is
  never bumped for an operation that failed. MPI_NO_OP is explicitly
  not treated as failure: the target window was still read into the
  result buffer, which is an access the notification must cover.
• MPI_Win_reset_notify_value uses a single opal_atomic_swap_64 so
  that increments arriving between a read and a separate zeroing cannot
  be lost.

Instrumentation

Four new SPC counters: OMPI_SPC_PUT_NOTIFY, OMPI_SPC_RPUT_NOTIFY,
OMPI_SPC_GET_NOTIFY, OMPI_SPC_RGET_NOTIFY.

Tests

Two new programs under ompi/test/general/, both wired into
make check, with the binaries added to .gitignore.

win_notify.c covers counter management, blocking and request-based
notified operations, notification-index error cases, on-demand counter
growth, the mpi_assert_max_num_notify assertion path, and the three
window attributes. All 8 notified operations and all 4 management
procedures are exercised.

win_notify_unsupported.c verifies that a window whose component does
not implement notified communication returns
MPI_ERR_UNSUPPORTED_OPERATION from every notified entry point rather
than calling through a NULL function pointer.

Open question for reviewers

osc/sm pre-attaches its reserved counters, whereas osc/ucx starts at
zero and requires MPI_Win_set_num_notify before any counter may be
referenced. MPI-5.1 §12.6.1 does not state what the initial attached
count is, so neither behavior is provably wrong — but the divergence
means a program that omits MPI_Win_set_num_notify works on sm and
fails on ucx. Worth settling before this lands.