Data tracks UniFFI - #1034
Data tracks UniFFI#1034
Conversation
Rust has these additional helpers to keep the interface between the rest of the client SDK and the data tracks managers as uniform as possible; instead of caring about specific fields in each response type, you hand over the whole response, and the logic of how the required information is extracted is kept as an implementation detail. A few other advantages:
I implemented |
Data tracks is fully released in OSS and Cloud now so this would be unexpected. Do you have any repro steps for this? |
No repro — false alarm on my side. The join does carry That RAII-on-drop is really the last open question: should FFI consumers mirror Rust's semantics (drop = unpublish) or go "publication-like" (JS keeps the publication in its manager until an explicit unpublish)? We went publication-like on the Swift side so callers aren't forced to retain the handle — this diff shows it: livekit/client-sdk-swift@3e3043e Framing it as "what should the convention be" rather than "should Rust change": the core's RAII is correct/idiomatic for Rust consumers — the divergence is about what the FFI bindings present. JS and (now) Swift both land on publication-like, so there's a de-facto convention worth making explicit. For concreteness, the Swift lifecycle surface:
|
pblazej
left a comment
There was a problem hiding this comment.
I don't see any functional gaps here preventing Swift integration
The rest is mostly consumer-side discussion (like the RAII pattern mentioned above).
Before merging please update both SPM_SIZE_LIMIT_BYTES and ANDROID_SIZE_LIMIT_BYTES with the final sizes.
Approximate sizes @ the latest commit:
SPM_SIZE_LIMIT_BYTES = 1152136ANDROID_SIZE_LIMIT_BYTES = 1250710
|
@hiroshihorie cc what's your take on applying that in flutter (as the infra is ~complete)? |
Add subscribe_with_options on RemoteDataTrack and a DataTrackSubscribeOptions record (buffer_size) so foreign consumers can tune the internal frame buffer, matching the core and JS subscribe APIs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
d0c8d7b to
eda4a11
Compare
163cdd9 to
61bd528
Compare
|
I have temporarily disable Dart tests in CI since they are currently failing. In the interest of getting this merged, we can address in a follow-up PR. |
…1300) - build-android-*: cargo-make's `extend` replaces the parent env map rather than merging it, so `env = { TARGET = ... }` silently dropped ANDROID_RELEASE_FLAG and RUSTFLAGS — `--profile release` produced debug .so files and android-copy-jniLibs found nothing in target/*/release. Derive the flag from CARGO_MAKE_PROFILE in the script instead. - android-bindgen-kotlin: TARGET leaks across cargo-make tasks, so bindgen-kotlin's `build` dependency cross-compiled for Android with the host linker ("cannot find -llog"). Pin TARGET back to the host triple and keep symbols, which library-mode bindgen needs on Linux. - Raise both size gates to measured values: ios-arm64 is 1088 KiB and arm64-v8a 1174 KiB, having grown past the limits set in #1171 when the data-track UniFFI surface landed in #1034. - Check out inputs.tag_name in both reusable workflows; workflow_dispatch was building the dispatch ref (main) rather than the requested tag. ### Before you submit your PR Make sure the following is true before submitting your PR: - [ ] I have read the [contributing guidelines](https://github.com/livekit/rust-sdks/blob/main/CONTRIBUTING.md) and validated that this PR will be accepted. - [ ] I have read and followed the principles regarding breaking changes, testing, and code quality. ### PR description Describe the changes in this PR. Explain what the PR is meant to solve and how to reproduce the issue in the first place. ### Breaking changes If this PR introduces breaking changes, list them here and document the rationale for introducing such a change. ### MSRV If the PR modifies the crate's MSRV (Minimum Supported Rust Version), document it here. ### Testing Ideally, unit test the code you add, but ensure you're not repeating existing test cases. Use as many already written scaffolding, utilities as possible; write your own, when needed. If external services, APIs, tokens are required (e.g., running an LK server instance), provide the necessary information. Make sure your tests perform useful, context-aware assertions and do not simply emulate "happy paths". ### Async We want the project to be runtime-agnostic, so please reuse what's already in [livekit-runtime](https://github.com/livekit/rust-sdks/blob/main/livekit-runtime/) and feel free to add anything missing. It's ok to use Tokio directly, when writing unit tests, if necessary. When testing, do not use artificial delays for the state to "catch up"; instead, respect the event flow and subscribe properly using channels or other mechanisms. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds [data tracks](livekit/rust-sdks#1034) to the Swift SDK, implemented on top of the Rust data-track engine via UniFFI ([livekit-uniffi-xcframework 0.1.8](https://github.com/livekit/livekit-uniffi-xcframework/releases/tag/0.1.8)). ## API ```swift // Publish + push let track = try await room.localParticipant.publishDataTrack(name: "telemetry") try track.tryPush(frame: DataTrackFrame(payload: data)) // non-blocking try await track.send(contentsOf: frames) // AsyncSequence, drop-on-full by default // Optionally declare what the frames are, and make the schema resolvable let schema = DataTrackSchemaId(name: "telemetry.v1", encoding: .protobuf) try await room.localParticipant.defineSchema(schema, definition: protoSource) let typed = try await room.localParticipant.publishDataTrack( name: "typed", options: DataTrackPublishOptions(schema: schema, frameEncoding: .protobuf) ) // Subscribe + receive let stream = try await remoteTrack.subscribe(bufferSize: 16) for await frame in stream { ... } // DataTrackStream is an AsyncSequence // A subscriber can resolve the schema its publisher declared if let declared = remoteTrack.info.schema { let definition = try await room.localParticipant.getSchema(declared, publishedBy: remoteTrack.publisherIdentity) } ``` Remote publish/unpublish surface on `RoomDelegate` and `ParticipantDelegate`; subscribed tracks are on `RemoteParticipant.dataTracks`, keyed by track name. Everything is Objective-C compatible (`DataTrackObjCTests` covers the surface). ## Design decisions **Rust owns the protocol, Swift owns the session.** The UniFFI managers (`LocalDataTrackManager`/`RemoteDataTrackManager`) implement the publish/subscribe state machine, packetization, and E2EE. The SDK contributes what only it has: the signal connection, the WebRTC data channels, participant identity, and delegate fan-out. Concretely, a session-scoped `DataTracks` coordinator (one per `Room`, `Room+DataTrack.swift`) feeds SFU signal messages into the managers, forwards their outbound requests through `SignalClient` (serialized, so publish/unpublish ordering is preserved), and pumps packets between the managers and the DTP data channel. **Thin public wrappers over the bindings.** The generated bindings are `internal import`ed and every public type (`LocalDataTrack`, `RemoteDataTrack`, `DataTrackFrame`, `DataTrackStream`, `DataTrackInfo`, schema/encoding types, error enums) is a small SDK-owned wrapper (~900 lines total). Tradeoff considered: exposing the generated types directly would save the layer, but pins the public API to regenerated code we don't control, and UniFFI emits Swift-only structs — `NSObject` wrappers are what makes the API reach Objective-C. The wrappers also carry the SDK idioms (lowerCamel error enums, `Track.Sid`-style identifiers, delegate naming, value semantics). **Publication lifetime follows the handle (RAII).** The returned `LocalDataTrack` *is* the publication: releasing the last reference unpublishes it, as does calling `unpublish()`. This matches rust-sdks, where the handle drops the publication. Consequently there are no local publish/unpublish delegate events — Rust dispatches room events only for *remote* data tracks, and the handle (`isPublished`, `waitForUnpublish()`) is the local observer. `withDataTrack(name:body:)` scopes a publication to a block for the common case. **Retain-cycle handling at the FFI boundary.** UniFFI callback interfaces hold their Swift delegate strongly (no weak references over FFI). A dedicated `ManagerDelegate` object references the `Room` and coordinator weakly, making it the designated weak link so `manager → delegate → room → manager` cycles can't leak (`Room+DataTrack.swift` documents the shape). **Reconnect semantics.** Quick reconnect preserves publications via `SyncState.publishDataTracks` (the managers replay their publish responses); full reconnect republishes and re-asserts subscriptions. The `DataTracks` subsystem survives reconnects (channels are swapped in) and is torn down only on real disconnect, which also unpublishes remote tracks and notifies delegates. The publisher-channel readiness gate is re-armed — not failed — on teardown, so a `publishDataTrack` issued mid-reconnect waits for the new channel instead of racing a dead transport. A local full reconnect recreates participants but *not* their data tracks: those are detached silently and re-attached, so no spurious unpublish is reported. When a *remote publisher* full-reconnects, the Rust manager treats the republication as a SID reassignment: the subscriber's existing `RemoteDataTrack` survives with its SID rewritten in place and active subscriptions transparently re-requested — no unpublish/republish events fire. This is why `RemoteParticipant.dataTracks` is keyed by **name**, not SID (Rust documents SIDs as unstable across reconnects; client-sdk-js keys by name for the same reason). **Backpressure.** `tryPush` is non-blocking and throws `queueFull`, handing the rejected frame back so it can be retried (parity with Rust's `PushFrameError::into_frame`). `send(contentsOf:)` defaults to dropping frames when the queue is full (DTP is lossy by design — unbounded buffering would trade a dropped frame for unbounded latency). On the channel, `DataTrackFrameSender` meters packets out on buffered-amount events against an 8 KiB low-water mark, keeping send latency bounded while letting a frame of any size stream out: at most one frame waits while another drains, a newer frame evicts the waiting one (drop-oldest), and frames are handled whole so a partial frame is never left on the wire. This mirrors `DataChannelSender` in rust-sdks; client-sdk-js instead blocks the producer, which isn't available here (the producer is a fire-and-forget FFI callback with no backpressure channel). `DataTrackFrameSenderTests` pins these semantics, including the cross-SDK divergence. Subscribe-side buffer is caller-configurable (`subscribe(bufferSize:)`). **E2EE.** When the room has E2EE configured, the same key provider drives data-track frame encryption via UniFFI `EncryptionProvider`/`DecryptionProvider` shims; `DataTrackInfo.usesE2ee` reflects it. Unlike data-channel payloads (a per-message property), data-track encryption is a track-level protocol property subscribers key their decryption on, so it can't be consulted per frame: the publishing manager is built on the *first publish* and captures the toggle then, which is late enough for a `setE2EEEnabled(true)` issued after connecting to apply. Toggling between publishes does not, and `Room.setE2EEEnabled` documents that. The cryptor resolves `Room.e2eeManager` on each call rather than capturing it at connect, so a manager assigned late is picked up for both encryption and decryption, and a missing one throws instead of silently sending plaintext. ## Protocol update The `protocol` submodule moves v1.45.8 → v1.50.4 (regenerated with `make proto`). Two reasons: - `DataTrackInfo` gained `frame_encoding` and `schema` after v1.45.8. Because `handleParticipantUpdate` re-serializes the parsed `ParticipantInfo` to hand the managers raw bytes, the older protocol silently stripped both fields before Rust saw them — declared metadata never reached subscribers. (The SFU was echoing it correctly; this was ours.) - It brings `DataBlob`/`StoreDataBlobRequest`/`GetDataBlobRequest`, which `defineSchema`/`getSchema` are built on. The general hazard remains and is worth a follow-up: any `ParticipantInfo` field newer than the pinned protocol is lost on the way to the managers. Threading the received wire bytes instead of re-serializing removes both the loss and a redundant encode. ## Size impact App Size job measured **16.28 MB** for the SDK's uncompressed `.app` delta before the protocol update; the budget moves 16 → 16.5 MB to leave room for it. That figure includes the nanopb protocol migration from `main`, which cut ~1.8 MB; the data-track share is the 0.0.6 → 0.1.8 UniFFI bump plus this integration (~+0.4 MB: compiled bindings at the dead-strip floor plus the Rust dylib growth). Isolating data tracks behind a Swift package trait was evaluated and rejected for now: the core SDK already depends on LiveKitUniFFI (tokens, log forwarding), both UniFFI components live in one dylib, and the bindings are one module — a trait could only gate the wrapper layer that dead-strip already removes. Revisit if the Rust side ships a feature-split artifact. ## Known upstream issue `LocalDataTrack.tryPush` cannot surface the FFI's own error: `PushFrameErrorReason` is declared in the `livekit_datatrack` UniFFI component while `try_push` lives in `livekit_uniffi`, and the cross-component error lift fails, throwing `UniffiInternalError.bufferOverflow` instead. Left unhandled this leaked an FFI-internal type through the public API and broke `send(contentsOf:)`'s drop-on-full policy, which inspects the reason. The wrapper recovers the applicable reason from `isPublished`; the typed catch remains first and takes over once the bindings are fixed. Worth filing against livekit-uniffi. ## Not in this PR - `tryPush` hands the rejected frame back, but the FFI still drops Rust's own `PushFrameError` payload. - Remote pipeline options (`max_partial_frames`) aren't exported by UniFFI yet. - Follow-ups filed from review: sharing the buffering state machine with `DataChannelPair`, and threading wire bytes through the signal path instead of re-serializing parsed join/participant updates. ## Testing Two unit suites run without a server: `DataTrackFrameSenderTests` covers the outbound drain, and `DataTrackSendTests` covers `send(contentsOf:)`'s queue-full policy against a sink that rejects on demand — deliberately not by saturating a live pipeline, which made the outcome depend on how fast the SFU drained. End-to-end tests against `livekit-server --dev` across six suites: publish/subscribe/roundtrip (small + multi-packet frames), publish options and schema/encoding metadata round-tripping through the SFU, schema definition storage and resolution, error cases (duplicate name, unauthorized, disconnected, push after unpublish, saturated queue), E2EE on, off, and toggled after connect, delegate events, unpublish paths, RAII handle release, `withDataTrack` scoping including cancellation, subscribe buffer clamping and drop-oldest eviction, concurrent multi-track push and 64-track publish, plus lifecycle: pre-join publications, publish during a full reconnect, publisher full-reconnect SID reassignment, quick-reconnect sync state, remote tracks surviving the local client's full reconnect, and room moves. `DataTrackObjCTests` covers the Objective-C surface including NSError bridging and value semantics. `DataTrackStream` only ends when its track is unpublished, so every read goes through bounded `next(within:)`/`collect(_:)` helpers — a lost frame on an unreliable channel should fail a test, not hang the job. The repeated two-room publish/subscribe preamble lives in a `withPublishedDataTrack` fixture; tests that control *when* the publish happens keep driving `withRooms` directly, since that timing is what they exercise. Schema definitions are stored as participant data blobs, which the server keeps behind `enable_participant_data_blob` — enabled for CI, and documented in `AGENTS.md` for local runs. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
### Before you submit your PR Make sure the following is true before submitting your PR: - [x] I have read the [contributing guidelines](https://github.com/livekit/rust-sdks/blob/main/CONTRIBUTING.md) and validated that this PR will be accepted. - [x] I have read and followed the principles regarding breaking changes, testing, and code quality. ### PR description Automates the Dart/Flutter half of the livekit-uniffi release (CLT-2872), the release-side prerequisite for livekit/client-sdk-flutter#1160. - **Make the generated package publishable.** Release builds omit the dev dylib and `publish_to`, gain LICENSE/README/CHANGELOG and real version constraints, and fail closed for unknown profiles. A `dart-clean` task keeps stale host dylibs out of release packages, and a shipped `analysis_options.yaml` silences lints that only fire on generated code. - **Re-enable the Dart tests** disabled since #1034. The uniffi-dart pin on main already carries the multi-crate codegen fixes from upstream, so this PR no longer changes the pin. - **Add `uniffi-dart-publish.yml`**, a tag-push workflow (pub.dev only accepts OIDC tokens from tag-push runs). It resolves and validates the tag against the crate version, builds every `build-<triple>.zip` through the reusable `uniffi-cdylib.yml` and attaches them to the release, then builds the Dart package from a copy outside the work tree (`packages/` is gitignored and pub archives from git's file listing), validates it, and publishes. The publish job depends on the asset job, so there is nothing to poll and a package can never publish without its assets. - **Harden `uniffi-cdylib.yml`.** It checks out the release tag rather than the dispatch ref, builds the Linux libraries inside manylinux_2_28 containers (glibc 2.28 floor, where ubuntu-latest gives 2.39 and breaks Debian 12 / Ubuntu 22.04 / AL2023), builds aarch64 Linux on a native arm64 runner, and installs cargo-ndk as a prebuilt binary. - **Pin the Dart SDK** to 3.13.1 in both the test and publish workflows so PR CI validates the SDK the publish runs with. **Publishing stays disabled** (`PUBLISH_ENABLED=false`, every run stops at `--dry-run`) until the first manual publish creates the package on pub.dev and automated publishing is configured. Runbook in the workflow header. ### Breaking changes None. Dev-profile `cargo make dart-package` behaves as before. ### MSRV No changes. ### Testing - Dev build: all 5 Dart FFI tests pass, same flow now re-enabled in CI. - Release build: `dart pub publish --dry-run` exits 0 with zero warnings and no dylib in the package. - Full release matrix dry run from this branch against the v0.1.9 tag: all 12 cdylib targets plus the Swift and Android jobs passed with `dry_run=true` (https://github.com/livekit/rust-sdks/actions/runs/34203702564). The CI-built Linux libraries were downloaded and verified: bare `.so` at the zip root, sha256 sidecars match, highest required symbol version GLIBC_2.28 on both architectures. - Consumer: livekit/client-sdk-flutter#1160's uniffi tests pass against the generated package. - Not exercisable before merge: the publish workflow's own job wiring, because GitHub only registers dispatchable workflows from the default branch. Plan is to dispatch it from main with `dry_run=true` before the next livekit-uniffi release. Whether knope's API-created tag fires the push trigger is confirmed only by the first real release. Both cost nothing while publishing is disabled. ### Async No async code added; the crate surface is untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Summary of changes:
livekit-uniffiResolves CLT-2472