Add UTS test specs for LiveObjects path-based API by paddybyers · Pull Request #473 · ably/specification

paddybyers · 2026-05-13T12:51:32Z

Summary

Adds portable UTS test specs for the LiveObjects path-based API (~330 tests across 21 files)
Covers PathObject, Instance, batch operations, LiveCounter/LiveMap CRDT internals, ObjectsPool sync state machine, RealtimeObject lifecycle, value types, subscriptions, and integration scenarios
Specs are language-independent pseudocode at uts/objects/
Corresponding ably-js translations: Add UTS tests for LiveObjects path-based API ably-js#2219

Test plan

ably-js translations of these specs pass (290 tests, 1 pending for unimplemented LiveMap#clear())

🤖 Generated with Claude Code

Complete portable test suite covering the LiveObjects path-based API: 21 files across unit tests (pure + mock WebSocket), integration tests (sandbox), and proxy integration tests. Covers PathObject, Instance, BatchContext, LiveCounter/LiveMap CRDTs, ObjectsPool sync state machine, value types, subscriptions, and GC. Includes table-driven validation tests, bytes/binary data coverage, and REST fixture provisioning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove client-side allocated_port/port_base patterns from all proxy test specs and helper docs. Port is now auto-assigned by the proxy when omitted from create_proxy_session(). Matches uts-proxy v0.2.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Align all ~330 LiveObjects UTS test specs with the squashed spec revision a397e34 (LiveObjects path-based API). Key changes: - Add parent_references.md (20 tests): RTLO3f, RTLO4g/4h, RTLO4f, RTO5c10 - Add public_object_message.md (13 tests): PAOM1-3, PAOOP1-3 - Thread ObjectMessage through all CRDT operations and LiveObjectUpdate - Add RTO25 (access preconditions) and RTO26 (write preconditions) - Update subscription model: subscribe returns Subscription object - Add RTO24 (PathObjectSubscriptionRegister) dispatch tests - Add parentReferences maintenance tests to live_map.md (+8 tests) - Add post-sync parentReferences rebuild tests to objects_pool.md (+3) - Rename "consume"/"consumption" to "evaluate"/"evaluation" in value_types - Remove batch.md (Batch API deferred from current spec revision) - Remove subscribeIterator and LiveObject#unsubscribe tests - Update PLAN.md to reflect new file structure and test counts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

sacOO7

We also need to update Spec File Counts section at uts/README.md.
Looks like the Directory structure section needs an update as well. We could potentially remove it altogether—I’m not sure it serves much purpose anymore. WDYT?

paddybyers · 2026-06-30T07:05:43Z

Could we avoid merge commits like 61f16f8 ?

…uidance Fixes found while evaluating the derived ably-java ObjectsFaultsTest against the sandbox through uts-proxy: - objects_faults.md: restructure the RTO20e test to the RTO20e1 sequence (mutation in flight while SYNCING, then channel enters FAILED -> 92008). The previous steps mutated after FAILED, which per RTO26b fails with 90001 and never reaches publishAndApply - confirmed against objects-features.md, ably-js source and both ably-js/ably-java behaviour. - objects_faults.md: rule match.action values must be strings ("20"), not JSON numbers (uts-proxy rejects numbers with HTTP 400); bound every await with a timeout; extend the action-number table (ACK, ERROR); align the Common Cleanup with multi-client tests. - connection_resume.md: RTN15g rule used invalid action type "refuse" (silently passes through) - corrected to "refuse_connection". - helpers/proxy.md: document the match.action string format, the v0.3.0 action-name resolution limit (names beyond AUTH need numeric strings), and add a wall-clock timeout convention. - docs: add wall-clock timeout guidance for integration specs (writing-derived-tests.md new section, integration-testing.md and writing-test-specs.md pointers) - virtual-time test frameworks (e.g. kotlinx-coroutines runTest) fire spec timeouts instantly unless waits run on the real clock. - PLAN.md: update the objects_faults.md summary row to the corrected RTO20e scenario.

proxy.md is cross-module infrastructure documentation, not a realtime helper: it is consumed by the realtime (7 proxy specs), rest (rest_fallback.md) and objects (objects_faults.md) modules plus both process docs, unlike the genuinely module-local helper specs (mock_websocket.md, mock_http.md, standard_test_pool.md) which are each used only by their own module. Its old location forced cross-module reach-ins and implied it was realtime-specific. - git mv uts/realtime/integration/helpers/proxy.md -> uts/docs/proxy.md, leaving a one-line stub at the old path so external deep links keep resolving - update all 19 references across 12 files, normalising the pre-existing stale `uts/test/` prefix carried by the realtime proxy specs' references and replacing two mentions of the long-gone in-repo `uts/test/proxy/` directory with a link to the ably/uts-proxy repo - update the directory-tree diagrams in uts/README.md, docs/integration-testing.md and docs/writing-test-specs.md - adjust proxy.md's own cross-reference to sibling style (writing-derived-tests.md) No content changes to the spec itself beyond that one internal reference. Note: ~80 unit/proxy spec files still carry stale `uts/test/...` legacy paths unrelated to proxy.md — left for a separate cleanup PR.

…jects

Align the uts/objects test specs with the canonical objects-features.md on main, and fold in the test-correctness fixes from the objects UTS audit. Reconciliation with main: - Rename the internal CRDT type LiveMap/LiveCounter -> InternalLiveMap/ InternalLiveCounter (271 refs), and rename the four CRDT spec files to internal_live_*.md, repointing all cross-references. Blueprint .create() factories, public PathObject/Instance, and LiveMapUpdate/ LiveMapValue are deliberately left unchanged. - get() precondition RTO23b -> RTO23e: get() now performs ensure-active-channel (RTL33) -- re-attaches on DETACHED, rejects 90001 only on FAILED -- instead of unconditionally throwing 90001. - Rename blueprint value types LiveMapValueType/LiveCounterValueType -> LiveMap/LiveCounter (static .create() factories), matching main. - Consolidate RTO25 (access) / RTO26 (write) preconditions as dedicated sections in realtime_object.md, fixing 8 references to two precondition files that never existed; drop the mislabeled RTO25b-via-get() tests. - public_object_message: replace the ably-js-private _derivedFrom with a spec-neutral derivedFrom (RTLCV4g5/RTLMV4j5 keep this retained Create local-only and unnamed). Test-correctness fixes (objects UTS audit): - Quiescence-barrier pattern for negative subscription assertions. - Tombstone / echo-dedup handling; map-clear (RTLM24) and LWW-reject (RTLM9b) semantics; path depth-coverage (RTO24b2a1) fixes. - ACK-serial helper + SITE_CODE in standard_test_pool; explicit REST per-op cardinality; assorted spec-ID and expected-event corrections. Validated against main objects-features.md and cross-checked with ably-js; spec naming is preferred over ably-js where the two differ.

…specs Implementing the reconciled objects specs against ably-js surfaced a serial-ordering defect: the canonical ack_serial value "ack-{msgSerial}:{i}" and the "s:1"/"s:2" serials in the RTO24a/RTO24c1 tests sort BEFORE the standard pool's "t:0" entry timeserials under the string LWW comparison (RTLM9), so a strictly-derived SDK rejects those operations as stale - every local apply-on-ACK MAP_SET on a pool entry fails, and the RTO24c1 root MAP_SET never fires its subscription. - ack_serial redefined to the sort-safe "t:" + (msgSerial + 1) + ":" + i form, with a NOTE documenting the ordering constraint. - All 8 inline auto-ACK harness copies now call ack_serial(msg.msgSerial, i) instead of duplicating the literal, per the helper's own "never hardcode" convention; the 3 literal "ack-0:0" uses in the echo-dedup and ack-after-echo tests reference ack_serial(0, 0). - RTO24a/RTO24c1 serials "s:1"/"s:2" changed to "t:1"/"t:2"; the adjacent rationale comments rewritten to state both the site-level newness and entry-level LWW ordering facts. - Removed the stale note-only Test IDs RTLC12b/increment-requires-publish-0 and RTLM20d/echo-messages-false-0 (their sections carry no steps or assertions - the preconditions live in realtime_object.md RTO26) so tag sweeps stop reporting them as permanently missing tests. - Added the missing RTLCV3c/no-validation-at-create-0 section to value_types.md (no validation at blueprint creation; rejection happens at evaluation per RTLCV4a). Validated by the derived ably-js suite (ably/ably-js#2257): 309 passing, 0 failing with these values; the original values reproduce 7 failures.

…l helpers Correct three self-inconsistencies in the objects UTS unit specs (verified against the ably-js reference tests and SDK): - SI-1: counter object-states double-counted the created value. Counter sync is additive (RTLC6c+RTLC6d/RTLC16: data = count + createOp.count), so `count:N` + `createOp:N` materialised 2N and contradicted every consumer asserting N. Set the residual `count` to 0 (createOp keeps the initial value) across all 16 affected fixtures in standard_test_pool, objects_pool, parent_references and path_object_subscribe. - SI-2: inbound MAP_SET/MAP_REMOVE on existing pool entries used bare serials ("99"/"100"/"101") that sort before the pool baseline "t:0" under lexicographic LWW (RTLM9e), so they were rejected as stale and silently defeated the tests. - SI-3: RTO20f reused the apply-on-ACK serial, which RTO9a3 echo-dedup discards before the newness check (value stays 110, contradicting its own assertion of 120 and the sibling RTO20 dedup test). Use a serial that is not the ACK serial but sorts below it, so it genuinely exercises RTLC7c. Add a named serial vocabulary to the Canonical Constants so the LWW ordering intent is explicit and used at every synthetic inbound serial: POOL_SERIAL (baseline), remote_serial(i) (remote op that wins vs the baseline) and below_ack_serial(i) (a non-ACK serial that loses to the ACK serial).

…e RTPO19e2 comment standard_test_pool.md: a createOp is a full ObjectOperation whose `action` and `objectId` are mandatory (OOP2) and validated by SDKs before merging — yet every fixture in the module writes the terse form `createOp: { counterCreate: {...} }` and build_object_state copied it verbatim, producing wire-invalid states as literally written. Both SDK suites already compensate (ably-java's builders embed action and inject objectId; ably-js spells both out in every fixture). Codify that: build_object_state now fills in missing `objectId` (from the state's objectId) and `action` (COUNTER_CREATE/MAP_CREATE by payload), keeping all terse fixtures valid without edits. Explicit fields are preserved, so deliberately-mismatched negative fixtures remain expressible. path_object_subscribe.md: rewrite the stale RTPO19e2 comment (it described an older two-message MAP_SET flow) to match the actual fixture: a single OBJECT_SYNC updating counter:score via replaceData (100 -> 200, RTLC6), with root intentionally omitted — per RTO5c2a the root must never be removed from the pool, so the update stays reachable and dispatches with message omitted.

Review comments (PR #499): - Grammar: "a Internal*" -> "an Internal*" at the 4 flagged sites plus a 5th the review missed (internal_live_map.md tombstone prose). - Add objectsGCGracePeriod: 86400000 to the 3 minimal mock CONNECTED connectionDetails (RTO23a, RTO23e-DETACHED, RTO23e-FAILED) — the other 12 blocks already carried it, and 86400000 is the spec's own default (RTO10b3); omitting it invited derived-test failures for fixture reasons unrelated to each test's intent. - Link the LiveObjects REST API docs (built from the OpenAPI spec) in the V2-format REST provisioning section. Naming audit (LiveMap/LiveCounter vs InternalLive*): - RTPO3 heading + requirement row said path resolution walks "LiveMaps"; per RTPO3a1 the walked objects are InternalLiveMaps — the section body was renamed but the heading/table lagged. - build_object_state comment: createOp validation happens on the internal CRDT object — "a LiveCounter rejects" -> "an InternalLiveCounter rejects" (comment was added after the mass rename, so no sweep could catch it). Behavioral audit: - internal_live_map_api.md header over-claimed RTLM24 and RTLM13: the file has no MAP_CLEAR test (RTLM24 is the inbound apply, tested in internal_live_map.md — the main spec defines no public clear()) and no values() test (RTLM13 is untested everywhere, a known gap). Header corrected to RTLM5, RTLM10–RTLM12, RTLM20–RTLM21, RTLMV4, RTLCV4 — verified against the tests actually present. All claims cross-validated against objects-features.md (RTLM5/10-13/ 20/21/24, RTO10b1/b3, RTPO3a1, RTLC6d/RTLM6d).

…K in RTO20 test The ack-after-echo test injected the ACK immediately after increment(), implicitly assuming the publish had already reached the transport. That holds on single-threaded SDKs (ably-js), but on SDKs that dispatch the publish asynchronously (ably-java's sequentialScope) the ACK can arrive while no message is pending on the connection, where it is silently discarded - inc_future then never completes and the test hangs. Add an explicit poll_until on the mock's event log for the outbound OBJECT message before sending the echo/ACK, making the ordering the test intends explicit rather than runtime-dependent.

Previously PathObject#instance returned undefined/null when the path resolved to a primitive (RTPO8d). This is inconsistent with the Instance model itself: RTINS1 defines Instance as a view of "a LiveObject or primitive value", and primitives already surface as Instances via Instance#get (RTINS5c). Replace RTPO8d with RTPO8f: a primitive resolution returns a new Instance wrapping the primitive value. Resolution failure (RTPO8e) still returns undefined/null. Update the corresponding UTS test spec: the RTPO8c primitive-null test becomes RTPO8f/instance-primitive-wrapped-0, asserting the returned Instance has no object id (RTINS3b) and exposes the primitive via Instance#value (RTINS4c).

…ns audit - RTO25b/RTO26b/RTO20e1 unit tests reach DETACHED via client-side channel.detach(): an unsolicited server DETACHED triggers an immediate re-attach (RTL13a) in a compliant SDK, so the channel never observably stays DETACHED and AWAIT_STATE == DETACHED is a race. The shared helper mocks now answer the outbound DETACH with DETACHED. - RTO20e1 covers both reachable states as separate tests: fails-on-channel-detached-0 (client-side detach) and a new fails-on-channel-failed-0 (channel ERROR, matching the proxy tier per #501). SUSPENDED is connection-level and out of unit-mock scope. - RTO15 test asserts observable wire behaviour only: realtime channel.object exposes no public publish (RTO15 is internal), so the result.serials assertion was unrunnable; the serials contract is observed via the neighbouring RTO20 apply-on-ACK tests. The RTO15h clause itself is unchanged. - public_object_message semantics assertions use "lww" - the shipped public type (ObjectsMapSemantics = 'lww') is lowercase; "LWW" could never match. - RTO24c1 subscribe call fixed to listener-first, matching every other with-options subscribe call. - Pseudocode conventions added to uts/README.md (null/undefined mapping, property-vs-call syntax, language-inapplicable inputs) plus language-applicability notes on the RTLC12e1 null row and the RTLMV4b key-type test. - objects_faults.md unit-tier cross-reference updated to the fails-on-channel-failed-0 test it describes.

…preconditions Review feedback: - RTO15 asserts the state entry is the encoded ObjectMessage for the driven mutation (RTO15e3), not just that one entry exists. - Both RTO20e1 tests assert 'inc_future IS NOT complete' before the channel state transition (matching the RTO20e waits-for-synced pattern), pinning the parked-in-wait precondition and closing the false-positive where an SDK that rejects early with 92008 would pass.

Review feedback (ttypic): semantics is the ObjectsMapSemantics enum, wire-encoded as an integer (OMP2) - the lowercase form asserted here previously was ably-js's idiomatic string-literal rendering leaking into the pseudo-code. The assertions revert to the symbolic "LWW", and a new 'Enum values' bullet in the uts/README.md pseudocode conventions defines how SDKs derive their idiomatic public rendering from it (enum member in typed SDKs, string-literal union in ably-js). Also documents the provision_objects_via_rest response contract: the helper returns the created/updated objectIds flattened in request order, so derived tests can target follow-up operations by objectId.

…sync The bulk enterClient test counted only ENTER events on the observing client: if that client's connection drops mid-test, members it missed arrive via the presence re-sync as PRESENT events rather than ENTER, so an ENTER-only count undercounts forever and the poll burns its full timeout - the mechanism behind the intermittent CI failure of the derived test. Members are now counted by clientId from both ENTER and PRESENT events, asserted as a set size. Derived ably-js test updated in lock-step (ably/ably-js@f5be7694).

…from test specs - add reference pseudocode definitions for poll_until and the new poll_until_success to writing-test-specs.md: poll_until_success treats any error raised by the condition as "keep polling" and raises the most recent error on timeout so failures stay diagnosable; document both, plus flush_async(), in the README pseudocode conventions - use poll_until_success for mutable_messages store reads (not-found until a just-written message is visible) and objects_faults recovery polls (value reads raise while the channel is transiently DETACHED by fault injection) - fix value-form poll call sites that returned a boolean while their result was assigned and dereferenced (12 sites across mutable_messages, history, publish, presence and channel_history specs) - replace unit-tier real-time WAITs with deterministic steps: flush_async() (RTC1b, RTL5k), fake timers + ADVANCE_TIME (RSC15f x2), and an explicit second authorize() for RSA10e - RTO10/RTO10b1: stamp GC tombstones with now() so ADVANCE_TIME is what makes them eligible; RTO10b1 previously passed even when objectsGCGracePeriod was ignored - drop redundant default timeouts from objects integration polls

…mer guidance - rename the flush_async() pseudocode convention to process_pending_events(): intent-named like the other conventions (poll_until, AWAIT_STATE) rather than leaking the JavaScript event-loop mechanism; define it cross-language with flushAsync()/setImmediate (ably-js) and runCurrent() (kotlinx) as renderings - move the fake-time/poll-deadline trap out of the integration-timeouts section into "No real timers in unit tests" where readers will look for it; present the two remedies as an ordered list, cite ably-js's RTO10/RTO10b1 backdating as the worked example of the preferred remedy (correctly attributed as the derived rendering of the spec's ADVANCE_TIME), and note when a synchronous stub window is exempt (RTLM19) - state that mainstream fake-timer tools (Jest, sinon) mock Date by default, which is what makes the trap easy to hit - add negative-assertion guidance to writing-test-specs (settle pending events via process_pending_events(), never a fixed WAIT) and bridge the derived-test flushAsync() bullet back to the convention - drop "harness" phrasing in favour of the docs' established helper/suite terminology

# Conflicts: # specifications/objects-features.md

github-actionsBottemporarily deployed to staging/pull/473 May 13, 2026 12:52 Inactive

paddybyers requested a review from ttypic May 13, 2026 13:48

github-actionsBottemporarily deployed to staging/pull/473 May 14, 2026 07:18 Inactive

paddybyers force-pushed the uts-integration-proxy branch from f355624 to 2e12074Compare May 14, 2026 20:17

Base automatically changed from uts-integration-proxy to mainMay 14, 2026 20:18

paddybyers force-pushed the uts-liveobjects branch from f8f1b71 to 08789a6Compare May 14, 2026 20:36

github-actionsBottemporarily deployed to staging/pull/473 May 14, 2026 20:36 Inactive

paddybyers requested a review from mschristensen May 15, 2026 13:58

paddybyers force-pushed the uts-liveobjects branch from 08789a6 to 97d02dbCompare May 27, 2026 21:53

paddybyersand others added 3 commits May 28, 2026 14:08

paddybyers force-pushed the uts-liveobjects branch from 97d02db to 2fba05eCompare May 28, 2026 13:08

github-actionsBottemporarily deployed to staging/pull/473 May 28, 2026 13:09 Inactive

sacOO7 reviewed Jun 1, 2026
View reviewed changes

Comment threaduts/realtime/integration/helpers/proxy.md

sacOO7 reviewed Jun 1, 2026
View reviewed changes

Comment threaduts/docs/integration-testing.md

sacOO7 reviewed Jun 1, 2026
View reviewed changes

Comment threaduts/docs/proxy.md

sacOO7 reviewed Jun 1, 2026
View reviewed changes

Comment threaduts/objects/integration/objects_batch_test.md Outdated

sacOO7 reviewed Jun 1, 2026
View reviewed changes