DsBatchSampler: probe-epoch auto cap + fail-fast OOM ordering

[yxlllc]

Summary

DsBatchSampler re-groups samples into batches on every epoch, so the batch
geometries differ across epochs. A batch appearing in a later epoch can exceed
anything seen in epoch 0 while still respecting max_batch_frames. Since the
CUDA caching allocator grows segment-wise and never shrinks, such a batch can
lock in a permanently larger reserved footprint — or OOM — several epochs into
a run that looked stable.

This PR turns epoch 0 into a memory probe epoch:

1. Fail-fast ordering — on the probe epoch, batches are served in strictly
   decreasing padded-frames order, so the most memory-hungry batch runs at the
   very first steps. If the configuration does not fit, the run fails at the
   start of training instead of several epochs in.

2. Auto-capped batching — the probe epoch's measured maximum padded-frames
   becomes the effective max_batch_frames for all subsequent epochs. Every
   later grouping is therefore bounded by a geometry already proven to fit.
   Because the allocator's reserved high-water mark is path-dependent and only
   grows when a step introduces demand beyond anything seen before, the epoch-0
   peak becomes a hard upper bound for the rest of training — no late-epoch
   reserved-memory growth, and no late-epoch OOM from novel batch shapes.

DDP behavior

No communication is needed: the batch pool and the cap are pure functions of
the dataset sizes and the epoch-seeded RNG (seeded identically on all ranks),
so every rank independently derives the same cap. The probe sort happens before
per-rank assignment, hence each rank's worst batch still lands in its first
steps, and fail-fast holds for DDP.

Compatibility

Gated behind probe_and_cap_max_frames (default False) and enabled for the
training dataloader only. With the flag off, batch streams are bit-identical to
before, including the validation sampler (its non-shuffled paths never touch
the RNG). drop_last and leftover/padding assignment are unchanged and only
ever operate on already-capped batches. Sampler state is not serialized, so
resuming from a checkpoint re-probes deterministically.