Skip to content

XP-2460: merge upstream v2.10.0 into ello/main - #16

Merged
tamirh merged 44 commits into
ello/mainfrom
tamir/xp-2460-upstream-v2.10.0
Aug 12, 2026
Merged

tamirh merged 44 commits into
ello/mainfrom
tamir/xp-2460-upstream-v2.10.0

Conversation

@tamirh

@tamirh tamirh commented Aug 8, 2026

Copy link
Copy Markdown

Brings ello/main up to upstream v2.10.0 (from a v2.7.0 base, 30 upstream commits). The fork gets meaningfully smaller: upstream has absorbed several deltas we were carrying.

Part of XP-2460.

Blocked on ElloTechnology/flutter-webrtc#2. LiveKitPlugin.kt now calls AudioSwitchManager.setAudioSessionManagementEnabled, which only exists from flutter_webrtc 1.6.0 — this will not compile until that PR merges. The internal pin currently points at that PR's branch SHA (19d804ac) and must be re-pinned to the merged ello/main SHA before this merges.

Deltas dropped — upstream now covers them

Delta Why it can go
Hand-edited livekit_models.pbenum/pbjson.dart upstream's protocol regen provides AGENT_ERROR = 16 natively. Removes a hand-edited base64 descriptor blob
meta: ^1.16.0 loosening consuming app resolves meta 1.18.0; upstream's ^1.17.0 is satisfiable
Kotlin 2.1.0 bump upstream declares the identical version
RPC completer guards / _pendingAcks bookkeeping RPC V2 (livekit#1087) restructures this into RpcClientManager with a single resolution path — a strict superset
mixWithOthers on the playAndRecord presets upstream deleted the presets; Ello is adopting upstream's automatic audio policy
pubspec.lock a downgrade artifact from resolving on an older Flutter, not an intentional pin

git diff v2.10.0 HEAD over lib/src/proto/ and android/build.gradle is empty — those deltas are genuinely gone, not merged around.

Deltas kept — verified present after the merge

rePublishAllTracks muted-skip · widened DisconnectReason · non-throwing protocol→public conversion · redacted RpcError.toString · Track.statsMonitorEnabled · containError listener containment · sampleIntervalMs · warmupFFT · makeBackgroundTaskQueue · the nine Room/Engine transport-health getters.

Two worth calling out, because upstream still has the bug:

  • rePublishAllTracks — upstream's loop still republishes a muted publication whose native track was disposed, which is fatal on reconnect. This lives in local.dart, the same file RPC V2 guts, so the two conflict hunks were resolved individually rather than taking the file wholesale. --theirs here would have silently reintroduced the crash.
  • DisconnectReason — upstream still maps 8 protocol values through a force-unwrapped lookup while its own protocol now emits AGENT_ERROR, so stock v2.10.0 throws on an agent error. Our conversion covers every value with an unknown fallback. Worth upstreaming.

New: Room.pendingRpcCount

RPC V2 moved ack/response bookkeeping off LocalParticipant into a privately-held RpcClientManager, so the two count getters we carried lost their home and transport monitoring lost the signal. This exposes the one total the new structure supports.

This is a semantic change, not a rename. RpcClientManager merges both waits into a single map, so ack-pending and response-pending genuinely cannot be reported apart. The consuming app reads both separately today and will need updating — a compile error, deliberately, rather than a silently wrong number.

Conflicts

Six, all as predicted: local.dart (above), LiveKitPlugin.kt (both sides kept — our background task queue and warmupFFT alongside upstream's audio handoff and seven new handlers, all verified present), native_audio.dart (upstream's deletion), pubspec.yaml, pubspec.lock, rpc_test.dart (upstream's; rpc_pending_acks_leak_test.dart removed as it tested deleted code).

The fork's flutter_webrtc URL also moves from SSH to HTTPS, which should let the consuming app drop its dependency_overrides shim.

Review focus

lib/src/participant/local.dart — the one place a plausible resolution reintroduces a fatal bug. Everything else is either mechanical or covered by the assertions above.

Not verified here

No CI on this fork. Builds, flutter test, and the audio-ownership behaviour change land in the consuming repo. The audio switch is default-on with no opt-out and alters iOS session category/mode, so it needs on-device verification there.

(Written by AI)

1egoman and others added 30 commits May 16, 2026 09:27
I noticed the example's color scheme was out of date. So, I pulled in a
bunch of colors from
https://github.com/livekit-examples/agent-starter-react, made some small
design updates to the button + text field, and swapped in the more
modern LiveKit logo.

Definitely a lot more that could be done here, I was aiming for "better"
not "perfect".

## Screenshots

<img width="596" height="830" alt="Screenshot 2026-05-15 at 12 46 11 PM"
src="https://github.com/user-attachments/assets/c9436e79-023e-41f5-b053-a6d96109440c"
/>

<img width="513" height="713" alt="Screenshot 2026-05-15 at 12 46 39 PM"
src="https://github.com/user-attachments/assets/974bf3e4-ea85-4784-a0b5-b0873eb636e0"
/>
Bump PB for PacketTrailer related features
…ls in example application (livekit#1057)

This PR fixes an issue in the Android example application where the
device screen turns off automatically during an active video call due to
screen timeout.

### **Problem**
During an ongoing LiveKit call on Android, the screen is not kept awake,
which leads to:

- Poor user experience during video calls
- Interrupted visibility when no user interaction occurs

### **Solution**
This PR adds native Android window flag handling in MainActivity to
ensure the screen remains active while a call is in progress:

- Uses native flags to prevent screen timeout
- Uses native flags as well to handle lock screen visibility

### **The flags are:**
- **Enabled** when the room is active
- **Cleared** when the room is disposed

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
## Summary

Align Flutter's generated default lower screen-share simulcast layer
with the common SDK behavior.

This updates the default generated lower layer to:
- use half resolution
- keep the top layer's frame rate
- use `max(150kbps, topLayerBitrate / 4)`
- carry through explicit bitrate/network priorities

Also aligns the `screenShareH720FPS5` and `screenShareH1080FPS30` preset
bitrates with the common SDK values.

## Notes

This does not add the JS-only screen-share presets such as `h360fps15`,
`h720fps30`, or `original`; that broader preset parity work should be
handled separately.

## Tests

- `dart format --set-exit-if-changed lib/src/types/video_parameters.dart
lib/src/utils.dart test/utils_test.dart`
- `flutter test test/utils_test.dart`
- `flutter analyze --no-pub lib/src/types/video_parameters.dart
lib/src/utils.dart test/utils_test.dart`
## Summary
- Suppress the C++/WinRT experimental coroutine deprecation error for
`permission_handler_windows_plugin`.
- Keeps the example Windows build working on the `windows-2025-vs2026`
GitHub Actions image.
- Shorten Build workflow job names so checks render as `Build /
Android`, `Build / Dart Analyze`, etc.

## Testing
- `git diff --check`
- Parsed `.github/workflows/build.yaml` with Ruby YAML.
## Summary
- Port Swift SDK simulcast lower-layer clamping behavior to Flutter.
- Clamp lower-layer max framerate so it does not exceed the configured
top layer.
- Clamp lower-layer bitrate only when the layer does not actually
downscale resolution.
- Use the larger output dimension when computing simulcast scale-down
values.
- Add coverage for clamp behavior, ladder length, unchanged presets,
priority preservation, and computed encodings.

## Testing
- `dart format --output=none --set-exit-if-changed
lib/src/types/video_parameters.dart lib/src/utils.dart
test/utils_test.dart`
- `flutter test test/utils_test.dart`
- `flutter analyze --no-pub lib/src/types/video_parameters.dart
lib/src/utils.dart test/utils_test.dart`
)

## Summary

Previously, `setVideoQuality` / `setVideoDimensions` on a remote track
were **ignored** (with a warning) whenever `adaptiveStream` was enabled.
This PR adopts the JS SDK's approach instead: the manual preference is
**merged** client-side with the dimensions derived from the visible
views, and the **smaller (more conservative)** of the two is sent to the
server.

Along the way it fixes a few related correctness issues in the
adaptive-stream path. The logic mirrors `client-sdk-js`'s
`RemoteTrackPublication` (`isEnabled`, `emitTrackUpdate`,
`areDimensionsSmaller`, `layerDimensionsFor`).

## Changes

**Manual quality + adaptive stream merge**
- `setVideoQuality` / `setVideoDimensions` / `setVideoFPS` are no longer
rejected when adaptive stream is on; the request is merged with the
visibility-derived dimensions, smaller area wins (matching JS
`areDimensionsSmaller`, strict `<`).
- When only a quality is requested, it's compared against that quality's
simulcast layer dimensions before deciding which to send.

**Explicit enable/disable overrides visibility**
- Enable/disable state is modeled as an internal tri-state
(`TrackEnabledPreference`: unset / enabled / disabled), mirroring JS's
`requestedDisabled`.
- An explicit `enable()` / `disable()` now always wins over
adaptive-stream visibility. Previously the visibility timer computed
`disabled` independently and silently ignored the user's request.

**Debounce correctness**
- A manual update cancels any pending debounced visibility update, and
the debounced send rebuilds settings from current state at fire time —
so a stale snapshot can't clobber a newer manual update.

**Misc**
- `fps` is now preserved across adaptive-stream visibility updates
(previously dropped).
- Adaptive-stream visibility state is reset when the track changes, so
stale dimensions can't leak into a later update.
- Merge/disable/build logic extracted into pure functions
(`resolveVideoSettings`, `resolveDisabled`, `buildUpdateTrackSettings`)
in `track_settings.dart`.

## Tests
- Unit tests for the pure resolution/build logic: merge precedence,
equal-area tie-break, tri-state disable, and proto building.
- Publication-level test asserting the `UpdateTrackSettings` actually
sent to the signal client for `enable()` / `disable()`.
The adaptive-stream visibility observer fed each view's logical (DIP)
size straight to the server, so on retina/HiDPI displays it
under-requested by the device pixel ratio and the server returned an
upscaled/soft layer. Server layers are sized in physical pixels.

- Add `AdaptiveStreamPixelDensity` (`auto` | `fixed(double)`) mirroring
the JS SDK's `pixelDensity` option, plus
`VideoTrackRenderer.adaptiveStreamPixelDensity` (default: `auto`), set
per view rather than room-wide. Fractional densities are supported and
the resolved value is capped at 3x to bound bandwidth.
- Track each rendered view with a `VideoTrackViewRegistration` that owns
both the `GlobalKey` and pixel density, so density updates stay tied to
the renderer lifecycle.
- Resolve auto mode from each view's `devicePixelRatio` via
`MediaQuery.maybeDevicePixelRatioOf`; fixed modes use a constant
multiplier.
- `_computeVideoViewVisibility` scales each view's logical size by the
resolved density, materializes the visible sizes once per tick, and
requests the largest scaled dimensions across all views.
* Added: Session API support for simpler E2EE setup
* Changed: Manual video quality selection can be used with adaptive
stream enabled
* Changed: Generated protocol definitions for LiveKit protocol v1.45.8
* Fixed: waitForBufferStatusLow busy-waiting after engine close
* Fixed: Simulcast lower layers exceeding the top layer
* Fixed: forceRelay log message interpolation
* Fixed: sendSyncState error handling so sync-state preparation failures
are not swallowed
* Fixed: Screen share simulcast default low layer alignment
* Fixed: Region failover null-provider dereference
* Fixed: Android builds with dependencies that require compileSdk 36
* Fixed: Deferred track listener leaks across reconnects
* Fixed: Adaptive stream dimensions on high-density displays
* Fixed: Session.start() reentrancy during concurrent calls
* Fixed: Connected server address resolving from the wrong peer
connection
* Fixed: Reconnect counter null assertion on the first reconnect attempt
* Fixed: Premature publication disposal during unpublish
…ivekit#1106)

## Summary
- Align Flutter default local-video degradation behavior with the Swift
SDK.
- Default unset `VideoPublishOptions.degradationPreference` to
`maintainResolution` for camera and screen-share publishing.
- Keep explicit degradation preferences overrideable by apps.

## Context
Related to livekit#1097, which explores preserving video quality through a
live-streaming option. This PR takes the smaller SDK-default approach
instead: use maintain-resolution by default, matching Swift, without
adding a separate app-facing toggle for this behavior.

## Testing
- `dart analyze`
- `flutter test test/core/room_e2e_test.dart`
## Summary

Migrates the Android plugin to AGP 9's built-in Kotlin while keeping
older toolchains building (same pattern as
flutter-webrtc/flutter-webrtc#2075):

- Apply the **Kotlin Gradle Plugin only when built-in Kotlin is
inactive** — AGP < 9, or AGP 9 with `android.builtInKotlin=false` (the
configuration Flutter currently ships by default while the ecosystem
migrates). When AGP 9's built-in Kotlin is active it registers the
`kotlin` extension itself and rejects KGP, so applying it is skipped.
- Set the JVM target through the `kotlin { compilerOptions {} }` DSL
**when the extension supports it** (KGP 1.9+ / AGP 9 built-in Kotlin),
falling back to the legacy `kotlinOptions` DSL for apps still on KGP
1.8.x.
- Bump the standalone buildscript fallback KGP to 2.1.0 so it is
self-consistent.
- Add a changeset entry for the generated release notes.

## Context

AGP 9 uses built-in Kotlin support and rejects Android plugins that
still apply KGP directly. This follows the Flutter compatibility
migration path instead of raising the minimum supported toolchain.

## Verification

- Example app builds (`flutter build apk --debug`) on the current stable
toolchain (AGP 8.x + modern KGP path).
- The AGP 9 built-in path mirrors the reviewed and merged flutter-webrtc
implementation.
…ead-back (livekit#1107)

## What

Runtime control of audio processing (AEC / NS / AGC / HPF) for local
audio tracks, plus an engine-wide diagnostic read-back, built on the
WebRTC-SDK audio processing options API (webrtc-sdk/webrtc#247 +
webrtc-sdk/webrtc#254).

## API

**Set** — `AudioProcessingOptions` with per-component enabled flags and
modes (`automatic` / `platform` / `software`), applied either at capture
time via `AudioCaptureOptions` or at runtime:

```dart
final result = await localAudioTrack.setAudioProcessingOptions(options);
```

Caller bugs (invalid combination, remote track) throw
`AudioProcessingException`; legitimate outcomes return a typed
`AudioProcessingApplyResult` (`applied` / `stored` / rejections).

**Read** — the audio processing module is owned by the native peer
connection factory and shared engine-wide, so the snapshot lives on
`AudioManager`:

```dart
final state = await AudioManager.instance.getAudioProcessingState();
```

Per component: `requested` (nullable — null means nothing was ever
applied), `isSoftwareResolved` / `isSoftwareActive`,
`isPlatformAvailable` / `isPlatformResolved` / `isPlatformActive`, and
`effective` as the merged verdict. Same requested → resolved → active →
effective vocabulary as the native SDKs.

## Commits

Bottom-up, each builds standalone:

1. `chore(deps)`: WebRTC-SDK pin bump
2. Dart `AudioProcessingOptions` for `LocalAudioTrack`
3. Routing through the LiveKit native plugin (iOS + Android handlers)
4. Typed apply results
5. Engine-wide v2 state read-back on `AudioManager`

## Dependencies / not yet done

- **Lib pin is still `144.7559.08`** — will bump to `144.7559.09` (which
carries the state v2 API from webrtc-sdk/webrtc#254) once published. The
state read-back native code requires `.09` to compile.
- **Android requires `FlutterWebRTCPlugin.getPeerConnectionFactory()`**
— flutter-webrtc/flutter-webrtc#2077.
- Device smoke test on iOS + Android pending the `.09` artifacts.
## Summary
- Add `deployment` field to `RoomAgentDispatch` for targeting specific
agent deployments
- Add `agentDeployment` to `TokenRequestOptions` to pass deployment
through token requests
- Update generated JSON serialization code

The `deployment` field allows targeting a specific agent deployment
(e.g., "staging"). Leave empty to target the production deployment.

Related PRs:
- node-sdks: livekit/node-sdks#675
- python-sdks: livekit/python-sdks#722
- rust-sdks: livekit/rust-sdks#1176
- client-sdk-swift:
livekit/client-sdk-swift#1043
- client-sdk-js: livekit/client-sdk-js#1971

## Usage
```dart
final options = TokenRequestOptions(
  roomName: 'my-room',
  agentName: 'my-agent',
  agentDeployment: 'staging',  // Optional: target specific deployment
);
```

Or directly via `RoomAgentDispatch`:
```dart
final dispatch = RoomAgentDispatch(
  agentName: 'my-agent',
  metadata: 'my-metadata',
  deployment: 'staging',
);
```

## Test plan

### Unit Tests
```bash
flutter test
flutter test test/token/token_source_test.dart -v
```

### Manual Verification

**1. Verify JSON serialization includes deployment:**
```dart
final dispatch = RoomAgentDispatch(
  agentName: 'my-agent',
  deployment: 'staging',
);
final json = dispatch.toJson();
print(json);  // Should include 'deployment': 'staging'
```

**2. Verify TokenRequestOptions converts to request correctly:**
```dart
final options = TokenRequestOptions(
  roomName: 'test-room',
  agentName: 'my-agent',
  agentDeployment: 'staging',
);
final request = options.toRequest();
print(request.roomConfiguration?.agents?.first?.deployment);  // Should print 'staging'
```

**3. Verify JSON round-trip:**
```dart
final original = RoomAgentDispatch(
  agentName: 'my-agent',
  deployment: 'staging',
);
final json = original.toJson();
final restored = RoomAgentDispatch.fromJson(json);
assert(restored.deployment == 'staging');
```

### End-to-End Verification
1. Use TokenSource to get credentials with agentDeployment set
2. Connect to room - agent with matching deployment should join
3. Verify only staging agent receives the dispatch

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
## What

Adds first-class, process-wide audio session and routing control through
`AudioManager` on iOS and Android. LiveKit owns the platform audio
session by default, while apps that need exact platform behavior can
switch to manual mode and apply typed session configs.

## API and behavior

- **Automatic by default**: calls need no setup. LiveKit applies a
managed communication policy.
- **iOS automatic mode**: the native WebRTC audio-engine delegate drives
`AVAudioSession` from engine lifecycle events. Listen-only playout uses
`playback`; recording uses `playAndRecord`.
- **Android automatic mode**: LiveKit uses a communication session
through the new AudioSwitch-backed `LKAudioSwitchManager`.
- **Manual mode**: `setAudioSessionOptions(...)` and
`deactivateAudioSession()` switch `AudioManager` to manual mode.
`setAudioSessionManagementMode(AudioSessionManagementMode.automatic)`
hands lifecycle control back to LiveKit.
- **Typed options**: `AudioSessionOptions.communication()` and
`AudioSessionOptions.media()` pre-fill Apple and Android configs, with
per-platform overrides applied verbatim in manual mode.
- **Speaker routing**:
`AudioManager.instance.setSpeakerOutputPreferred(...)` owns speaker
preference and forced speaker routing. Wired and Bluetooth devices still
win unless `force: true`.

## Compatibility

- Existing calls keep working without audio-session setup.
- `Hardware` audio members and `Room.setSpeakerOn(...)` are deprecated
forwarders to `AudioManager`.
- `flutter_webrtc` native audio-session management is disabled so
LiveKit has one owner for the session.
- `bypassVoiceProcessing` now only controls WebRTC voice processing; it
no longer changes the session intent.

## Docs and tests

- Adds `docs/audio.md` and updates the README audio sections.
- Adds coverage for session options, automatic/manual mode transitions,
Apple/Android policy resolution, routing serialization, and engine-state
observation.
- Verified locally with `dart analyze`, `flutter test
test/audio/audio_session_test.dart`, and `flutter test --reporter
compact`.
## Summary
- move create-time audio processing setup into the local capture start
path so publish/preconnect prepares platform audio processing before
WebRTC opens the microphone, matching the Swift SDK flow
- keep `LocalAudioTrack.setAudioProcessingOptions` as a command-style
runtime API that returns on success and throws
`AudioProcessingException` on failure
- expose structured failure reasons for invalid combinations,
unavailable platform support, native apply failures, and unknown
failures
- clean up local audio tracks when capture/publish startup fails and
keep cleanup failures from masking the original error
- bump `flutter_webrtc` to `1.5.2` for Android audio device module
access

## Behavior
- `LocalAudioTrack.create(...)` stores requested audio processing
options; it no longer attempts to apply them immediately.
- `track.start()` starts local capture and applies stored processing
options before the microphone opens on supported platforms.
- Publish and preconnect paths fail during capture start if the exposed
native platform API reports audio processing setup failure.
- Runtime `setAudioProcessingOptions(...)` still applies immediately for
active local audio tracks and throws on failed native apply/store.
- Android uses `JavaAudioDeviceModule.prewarmRecording(options)`. That
WebRTC API returns `void`, so this PR can surface thrown failures, but
clean internal `false` returns from `initRecordingIfNeeded()` /
`prewarmRecordingIfNeeded()` are not observable until
WebRTC/flutter_webrtc exposes a resultful API.

## Testing
- `flutter pub get`
- `flutter pub get` in `example`
- `flutter analyze --no-pub`
- `flutter test --no-pub`
- `flutter build apk --debug --no-pub` in `example`
- Android emulator smoke test: example joined a room, published local
microphone audio, muted/unpublished, and showed no
ADM/platform-unavailable errors
## Summary

- Add `LiveKitClient.initialize(initialAudioSessionOptions: ...)` for
Android WebRTC audio-device initialization.
- Seed Android automatic audio-session policy from the same initial
options so playback-first apps do not need to immediately call
`AudioManager.setAudioSessionOptions(...)` with the same preset.
- Rename the playback preset to `AudioSessionOptions.mediaPlayback()`
for clearer intent.
- Avoid sticky Android speaker routing when updating speaker preference.
- Update audio session docs and change entry.

## Runtime behavior

Android uses `AudioSessionOptions` in two places with different timing:

| API | Timing | What changes |
| --- | --- | --- |
| `LiveKitClient.initialize(initialAudioSessionOptions: ...)` | Before
WebRTC initializes. | Passes Android audio configuration into
flutter_webrtc/WebRTC ADM creation. This is where WebRTC playout
`AudioAttributes` such as `usageType` and `contentType` are set. It also
seeds LiveKit's automatic Android session policy. |
| `AudioManager.instance.setAudioSessionOptions(...)` | Runtime. |
Replaces LiveKit's stored session policy, switches to manual management,
and applies LiveKit's platform session behavior: Android audio mode,
audio focus mode, stream type, focus ownership, routing handler policy,
and iOS category/options/mode. |

The important Android limitation is that WebRTC playout
`AudioAttributes` are still initialization-time configuration. Calling
`AudioManager.setAudioSessionOptions(...)` at runtime applies LiveKit's
platform session policy, but it does not mutate the `AudioAttributes` on
an already-created WebRTC audio device module. For playback-first
Android apps that need media volume/routing from WebRTC playout, pass
`AudioSessionOptions.mediaPlayback()` to `LiveKitClient.initialize(...)`
before connecting.

## Validation

- `flutter test --no-pub test/audio/audio_session_test.dart`
- `flutter analyze --no-pub

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…for all video codecs (livekit#1120)

## Summary
- Apply `x-google-start-bitrate` SDP hint to all video codecs (VP8, VP9,
AV1, H264, H265), not just SVC codecs
- Use 90% of target bitrate as start bitrate to prevent initial
blurriness
- Degradation preference already defaults to `maintainResolution`


## Problem
Video starts blurry for 5-15 seconds before improving. This is caused by
WebRTC's bandwidth estimator starting at ~300kbps and slowly ramping up
to the target bitrate.

## Solution
**x-google-start-bitrate**: Tell WebRTC to start at 90% of target
bitrate instead of ramping up from ~300kbps. Applied consistently to all
video codecs.

Note, The PR only addresses the x-google-start-bitrate fix (changing
from SVC-only to all video codecs, and using 0.9 multiplier)
- The maintainResolution default was already in place in Flutter SDK, so
no change was needed for that part
  

## Test plan
- [ ] Verify video quality is sharp from the start when publishing
- [ ] Test with VP8, VP9, H264, AV1 codecs
- [ ] Verify bandwidth estimator adapts properly if network can't handle
the start bitrate

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
See this pull request for more info about what this does and how rpc v2
works: livekit/client-sdk-js#1832

Also, for an in depth description of behavior in edge cases, see
[RPC_SPEC.md
here](https://github.com/livekit/client-sdk-js/blob/00e93de0c3d85f846fa6707749fa46457a653a29/RPC_SPEC.md).

> [!WARNING]
> This pull request was LLM generated and has only been lightly reviewed
by the author, who is not a flutter expert. I have tested this and can
confirm it works in the happy path, but no other validation has been
done.
> 
> A more thorough review of this needs to occur before it could be
merged.

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
Fixes livekit#1061

Example usage:
```dart
VideoTrackRenderer(
  ...,
  placeholderBuilder: (context) {
    return const CircularProgressIndicator.adaptive();
  },
)
```
## Summary

Adds native certificate pinning support for SDK-owned TLS traffic in the
Flutter SDK.

- Applies to SDK-owned HTTPS requests and WSS signaling on native
platforms
- Does not apply to WebRTC media, TURN, or application-owned token
endpoints
- Not supported on Flutter web: configuring pinning there fails fast
with `UnsupportedError` at `Room.connect` instead of silently connecting
without pinning
- Validates the peer certificate during TLS connection setup before HTTP
or WSS request bytes are sent, and closes the socket on validation
failure
- Supports SPKI SHA-256 pins, exact leaf certificates, and custom
trusted certificates via `CertificateBytes`
- Adds `CertificatePinningException` so pinning failures can fail fast
and surface clearly
- Pinning failures are treated as non-recoverable during reconnect: the
SDK surfaces `DisconnectReason.signalingConnectionFailure` instead of
retrying TLS handshakes against an untrusted endpoint
- Pinning failures skip region failover during full reconnects, and
reconnect disconnect events are owned by the engine to avoid duplicate
or dropped events
- Also fixes a pre-existing ordering issue where
`EngineDisconnectedEvent` was emitted before engine cleanup in the
non-recoverable reconnect branch, which caused the room to drop the
event during full reconnects

## Rule Behavior

Rules are selected by host. Exact hosts, single-label wildcards like
`*.livekit.cloud`, multi-label wildcards like `**.livekit.cloud`, and
`*` are supported. `*.livekit.cloud` matches `project.livekit.cloud` but
not `a.b.livekit.cloud`; `**.livekit.cloud` matches both. Empty `hosts`
applies a rule to every SDK-owned TLS connection.

Hosts that match no rule are connected with platform trust only and the
SDK logs a warning. LiveKit Cloud region failover connects to
server-provided regional hostnames like
`project.region.production.livekit.cloud`, which carry more labels than
the project URL — use `**.livekit.cloud` so pinning also covers those
connections.

All rules that match a host are applied. Within one check type, any
configured value may match. Across check types, each configured type
must pass.

That means:

- Multiple matching SPKI rules form one accepted pin set
- `primaryPins` and `backupPins` are both accepted
- SPKI pins plus exact leaf certificates require both checks to pass
- `trustedCertificates` replaces platform trusted roots for that
matching host

SPKI pins are matched against the leaf certificate's public key only.
Dart does not expose the rest of the chain, so pinning an intermediate
or root CA key never matches.

## Example: SPKI Pins

Use this when possible. It is friendlier to certificate renewal because
the pin follows the public key rather than the full leaf certificate.

```dart
RoomOptions(
  networkOptions: NetworkOptions(
    certificatePinning: CertificatePinningOptions(
      rules: [
        CertificatePinningRule(
          hosts: ['**.livekit.cloud'],
          primaryPins: ['sha256/current-public-key-pin'],
          backupPins: [
            'sha256/next-public-key-pin-1',
            'sha256/next-public-key-pin-2',
          ],
        ),
      ],
    ),
  ),
)
```

## Example: Exact Leaf Certificate

Use this when the app should trust one exact leaf certificate loaded
from assets. Renewing or changing the leaf certificate requires shipping
updated certificate bytes unless another configured check also allows
the new certificate. `pinnedLeafCertificates` is an identity check after
TLS trust validation, so private or self-signed certificates also need
`trustedCertificates`.

```dart
final certificate = await CertificateBytes.fromAsset(
  'assets/livekit_leaf_cert.pem',
);

CertificatePinningRule(
  hosts: ['my-project.livekit.cloud'],
  pinnedLeafCertificates: [certificate],
  trustedCertificates: [certificate],
)
```

## Example: Custom Trust Store

Use this for the asset-based Flutter pattern where the app ships a leaf,
intermediate, or root certificate as trust material. When a pinning rule
matches a host, the SDK owns TLS setup for that connection and
`HttpOverrides.global` / `badCertificateCallback` are not consulted —
use `trustedCertificates` for self-signed or private PKI setups.

```dart
final certificate = await CertificateBytes.fromAsset(
  'assets/livekit_intermediate_ca.pem',
);

CertificatePinningRule(
  hosts: ['**.livekit.cloud'],
  trustedCertificates: [certificate],
)
```

## Example: Combine Modes

Checks can be combined. This requires the TLS chain to validate against
the custom trust store and the peer certificate SPKI to match one
configured pin.

```dart
CertificatePinningRule(
  hosts: ['**.livekit.cloud'],
  primaryPins: ['sha256/current-public-key-pin'],
  backupPins: [
    'sha256/next-public-key-pin-1',
    'sha256/next-public-key-pin-2',
  ],
  trustedCertificates: [certificate],
)
```

## Testing

- Added unit coverage for SPKI parsing, real X.509 parsing, single-label
and multi-label wildcard matching, and merged rule semantics
- Added native IO tests for HTTPS and WSS failure timing before request
bytes are sent
- Added native IO tests for exact leaf certificates, trusted leaf
certificates, trusted CA certificates, combined modes, and proxy
handling
- Added signal client tests for fail-fast on initial connect and no
event emission during reconnect attempts

Local verification:

```bash
flutter analyze --no-pub
flutter test --no-pub --reporter compact
```
## Summary

Follow-up to livekit#1065. Ensures the SDK emits exactly one
`RoomDisconnectedEvent` per failed connection attempt.

Previously a failed initial connect produced two
`EngineDisconnectedEvent`s, and therefore two `RoomDisconnectedEvent`s
and two room cleanups:

1. The signal client emits
`SignalDisconnectedEvent(signalingConnectionFailure)` while `connect()`
is failing (the validate path has done this since livekit#406, and the
certificate pinning path in livekit#1065 follows the same pattern), which the
engine relayed as `EngineDisconnectedEvent`.
2. `Engine.connect`'s catch then emits
`EngineDisconnectedEvent(joinFailure)` for the same failure.

## Changes

- `Engine.connect`'s catch is now the single emitter for initial connect
failures. It picks the reason by error type:
`signalingConnectionFailure` for `CertificatePinningException`,
`joinFailure` otherwise.
- The `signalingConnectionFailure` relay in the engine's signal listener
is removed (with a comment explaining why). During reconnects the
engine's reconnect handling already owns disconnect emission, so the
relay only ever produced duplicates. This also removes the
`_isReconnecting`/`_attemptingReconnect` guard that livekit#1065 added to
suppress the relay during reconnects, since there is no longer anything
to suppress.
- `SignalDisconnectedEvent` at the signal level is unchanged, only the
engine-level relay is removed.

## Behavior change

Apps listening for `RoomDisconnectedEvent` now receive one event per
failed connect instead of two. The reasons are unchanged for the common
case (`joinFailure`); certificate pinning failures surface as
`signalingConnectionFailure`. Apps that specifically depended on
receiving both events for a single failure would see one.

## Testing

Added room-level tests through the mock e2e container asserting exactly
one `RoomDisconnectedEvent` per failed initial connect, for both a
certificate pinning failure (`signalingConnectionFailure`) and a generic
websocket failure (`joinFailure`). Both tests fail against the previous
code (two events observed) and pass with this change.
…ivekit#1128)

## Summary

`AudioFrameCapture.start()` currently fails on desktop Linux and Windows
with:

```
MissingPluginException(No implementation found for method startAudioRenderer on channel livekit_client)
```

The Dart layer already routes desktop platforms to the native
implementation (`audio_frame_capture_native.dart`), but native handlers
for `startAudioRenderer` / `stopAudioRenderer` exist only on iOS, macOS,
and Android. The Linux and Windows plugins handle only `startVisualizer`
/ `stopVisualizer` and fall through to `NotImplemented()`.

This PR adds the missing handlers, bringing Linux and Windows to parity
with iOS/macOS/Android for `AudioFrameCapture`.

## Implementation

The low-level `AudioTrackSink` infrastructure was already in use by the
audio visualizer on both platforms, so the change is additive and
self-contained — no changes to `flutter_webrtc` or the Dart layer are
needed.

- **`shared_cpp/audio_renderer.{h,cpp}`** (new): platform-agnostic
conversion pipeline, a direct port of the Android `AudioRenderer` /
`AudioResampler`:
- validate (WebRTC sinks always deliver int16 PCM; anything else is
dropped)
- resample: passthrough / linear interpolation (upsample) / box filter
(downsample, anti-aliasing)
  - channel reduction: keep the first N channels (matching Android)
  - encode as little-endian `int16` or `float32` bytes
- **`linux/livekit_plugin.cpp`**, **`windows/livekit_plugin.cpp`**: new
`AudioRendererSink` class, structurally cloned from the existing
`VisualizerSink` (EventChannel + `StreamHandlerFunctions`,
`AddSink`/`RemoveSink`, task-runner hop to the platform thread). Streams
converted frames to Dart over
`io.livekit.audio.renderer/channel-<rendererId>`. Unlike the visualizer,
frames arriving before a subscriber attaches are dropped rather than
queued, matching the Android renderer semantics for live audio.
- **Method handlers** mirror the Android `handleStartAudioRenderer` /
`handleStopAudioRenderer` contract: same argument validation and error
codes (`INVALID_ARGUMENT`, `No such track`), idempotent start for an
existing `rendererId`, and success on stop for unknown ids.
- **CMakeLists**: `shared_cpp/audio_renderer.cpp` added to both
platforms' sources.

## Testing

- The shared conversion pipeline was exercised standalone (passthrough,
upsample, downsample, float32 encoding, channel reduction, invalid-input
drops, format defaults).
- The event payload matches what `audio_frame_capture_native.dart`
consumes (`commonFormat`, `sampleRate`, `channels`, `data` as
`Uint8List`, plus `frameLength` for parity with Android).

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
## Summary

- enable `VideoRenderMode.platformView` on macOS in `VideoTrackRenderer`
- avoid texture-only resize assumptions by reading width/height from the
shared video renderer interface
- add a patch changeset for macOS platform video rendering support

## Validation

- `flutter analyze --no-pub`
## Summary

Switching `renderMode` on a live `VideoTrackRenderer` (texture to
platformView or back) previously misbehaved on iOS, and would have on
macOS after livekit#1116:

- platformView to texture crashed during build. `_renderer` still held
the `RTCVideoPlatformViewController`, which then failed the `as
rtc.RTCVideoRenderer` cast.
- texture to platformView leaked the old texture renderer. It stayed
attached to `srcObject`, kept consuming decoded frames and held its GPU
texture.
- initState adopted a texture-type `cachedRenderer` even in platform
view mode.

## Fix

`didUpdateWidget` now resets the renderer when the effective render mode
changes, with ownership respected:

- Platform view controllers are only detached, never disposed here. They
are created and disposed by `RTCVideoPlatFormView`, so disposing them
from this widget raced its own dispose (the controller's `_disposed`
guard is set only after two awaits, so both dispose calls could pass
it).
- A caller-provided `cachedRenderer` is never disposed on a mode switch,
so it can be re-adopted when switching back.
- Self-created texture renderers are disposed according to
`autoDisposeRenderer`, matching unmount behavior.

## Testing

- `flutter analyze` and `dart format` clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
`Helper.selectAudioOutput` already routes audio on Android via device ID
strings (speaker, earpiece, bluetooth, wired-headset). The desktop-only
gate was silently swallowing the call and leaving
`room.setAudioOutputDevice()` as a no-op on Android.

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
…sion mode (livekit#1127)

## Description

Adds the two switches needed for CallKit coordination, mirroring the
Swift SDK pattern (`AudioManager.setEngineAvailability` and
`AudioSessionEngineObserver.isAutomaticConfigurationEnabled`).

### Engine availability

```dart
// at startup, before connecting
await AudioManager.instance.setEngineAvailability(AudioEngineAvailability.none);

// in CallKit provider(didActivate:), for apps whose CallKit events arrive in Dart
// (e.g. flutter_callkit_incoming). Apps with a native CXProvider delegate should
// call the native static LiveKitPlugin.setEngineAvailability there instead.
await AudioManager.instance.setEngineAvailability(AudioEngineAvailability.defaultAvailability);
```

This is the highest priority gate over anything that may start the audio
engine. Requests made while unavailable are honored once availability
allows, so apps can call `room.connect` and `setMicrophoneEnabled(true)`
inside CallKit action handlers and let the engine start when CallKit
activates the session. No-op on non-Apple platforms, always safe from
cross-platform code.

### External call system session mode

```dart
await AudioManager.instance.setAudioSessionManagementMode(
  AudioSessionManagementMode.externalCallSystem,
);
```

LiveKit keeps configuring the session category/mode from engine
lifecycle like automatic mode, but never calls `setActive` in either
direction. CallKit owns activation timing. This improves on the Swift
example flow, where the app must set the category itself inside
`didActivate`.

The mode is named platform-neutrally on purpose. On Android it currently
behaves like manual mode, and the same mode is reserved to stand down
LiveKit's audio-focus and routing management when Telecom
(`androidx.core.telecom`) integration lands, since the Telecom framework
owns routing for registered calls.

### Native early gating


`LiveKitPlugin.setEngineAvailability(isInputAvailable:isOutputAvailable:)`
is a public static so an AppDelegate can gate audio before the Flutter
engine exists (CallKit killed-state wake). The value is stored and
applied at plugin registration, after the engine observer is installed
and before any audio operation can start the engine.

## Implementation notes

- Implemented entirely on LiveKit's own plugin and method channel. The
ADM API (`RTCAudioDeviceModule.setEngineAvailability`) is public in the
WebRTC-SDK pod, so no flutter_webrtc change or release is needed.
- The availability setter dispatches off the platform thread, since an
availability flip can rebuild the audio engine.
- One new observer flag (`isSessionActivationEnabled`) threads through
the three activation sites: managed configure, cached configure, and the
engine-disable deactivation path.
- `deactivateAudioSession()` is a guarded no-op under
`externalCallSystem`.

## Testing

- `dart analyze lib test` clean, all 344 tests pass (4 new tests for
channel args and the external-mode deactivation guard).
- iOS example builds.
- A CallKit example app exercising the full flow (outgoing, incoming via
PushKit, mute, end) is the next step in livekit-examples.

## Follow-ups

- CallKit example app update in livekit-examples/flutter-callkit.
- Android Telecom (`androidx.core.telecom`) integration behind the same
`externalCallSystem` mode.
- docs/audio.md CallKit section.
## Description

Adds `AudioManager.setMicrophoneMuteMode` / `getMicrophoneMuteMode` with
a LiveKit-owned `MicrophoneMuteMode` enum (`voiceProcessing`, `restart`,
`inputMixer`), mirroring the Swift SDK's
`AudioManager.microphoneMuteMode`.

## Background

On iOS/macOS the AVAudioEngine-based ADM defaults to `voiceProcessing`
muting, which plays the platform's mute/unmute sound effect (see
flutter-webrtc/flutter-webrtc#2098). Apps can select `inputMixer` or
`restart` to mute silently.

```dart
await AudioManager.instance.setMicrophoneMuteMode(MicrophoneMuteMode.inputMixer);
```

The mode applies to LiveKit's own mute path: disabling a published local
audio track drives the engine-level microphone mute in webrtc
(`WebRtcVoiceSendChannel::MuteStream` -> `adm->SetMicrophoneMute`). For
the fastest silent mute toggling, combine `inputMixer` with
`AudioCaptureOptions(stopAudioCaptureOnMute: false)` (documented on the
API).

## Implementation notes

- Self-contained: implemented on LiveKit's own plugin and method
channel. `LiveKitPlugin.swift` calls the ADM's `muteMode`/`setMuteMode:`
directly (public API in the WebRTC-SDK pod already linked), so this does
not depend on any flutter_webrtc change or release. The companion
flutter-webrtc PR (flutter-webrtc/flutter-webrtc#2105) exposes the same
control to plain flutter-webrtc users independently.
- No flutter_webrtc types on the public surface: the enum is
LiveKit-owned and named to match the Swift SDK (`restart`, not
`restartEngine`).
- No-op on non-Apple platforms, including for `unknown`, so `set(await
get())` round-trips safely in cross-platform code.
- The setter runs off the platform thread natively, since a mode change
while muted can rebuild the audio engine. Setter errors propagate to the
caller (mirrors the Swift SDK's throwing API), the getter falls back to
`unknown`.
- Engine-wide state, recommended to set once before connecting.

## Testing

- `dart analyze lib test` clean, all 315 tests pass.
- iOS example builds against the released flutter_webrtc 1.5.2 pin, no
dependency override needed.
…#1134)

## Summary

This PR ports the `ConnectionCheck` connection-diagnostics utility from
[client-sdk-js](https://github.com/livekit/client-sdk-js/tree/main/src/connectionHelper)
to Dart, the same checks that power
[livekit.io/connection-test](https://livekit.io/connection-test).
Flutter apps can now run structured preflight diagnostics proactively
(at app start) or reactively (when `room.connect()` fails) to determine
*why* a connection cannot be established (firewall, VPN, blocked TURN,
etc.), instead of asking users to reproduce on web.

Ref: [Diagnosing Connection Errors with Connection Test
Utility](https://kb.livekit.io/articles/3972989092-diagnosing-connection-errors-with-connectionchecker)

## Demo
<img width="3840" height="2160" alt="rtc_connection_check"
src="https://github.com/user-attachments/assets/6359a952-4ac1-4a0b-89a5-33045024f1e2"
/>

## API

```dart
final connectionCheck = ConnectionCheck(url, token);
final listener = connectionCheck.createListener();
listener.on<ConnectionCheckUpdateEvent>((event) {
  print('${event.info.name}: ${event.info.status.name}');
});

// recommended minimum set
await connectionCheck.checkWebsocket();
await connectionCheck.checkWebRTC();
await connectionCheck.checkTURN();

// additional checks
await connectionCheck.checkReconnect();
await connectionCheck.checkPublishAudio();
await connectionCheck.checkPublishVideo();
await connectionCheck.checkConnectionProtocol();
await connectionCheck.checkCloudRegion();

print('all checks passed: ${connectionCheck.isSuccess}');
await listener.dispose();
await connectionCheck.dispose();
```

## Testing

- `flutter test`: 364 passing (20 new tests covering the `Checker` base
lifecycle, `ConnectionCheck` orchestration/eventing, ICE candidate
parsing, and `WebSocketCheck` against the mock WebSocket connector).
- `flutter analyze`, `dart format`, `import_sorter`, `check_version`
passing
- Manually tested web, iOS, Android

## Example

`example/lib/pages/connection_check.dart` adds a "Connection Check" page
(launched from the connect screen) that runs all checks and displays
per-check status + logs.
)

## Summary

Marks the audio APIs that are new since 2.8.0 as `@experimental` before
they ship in a stable release, so their shape can still evolve without a
breaking-change dance.

**Marked experimental**

- Audio session surface: `AudioSessionOptions`,
`AppleAudioSessionConfiguration`, `AndroidAudioSessionConfiguration`,
the platform enums, `AudioSessionManagementMode`, and `ValueOrAbsent`
- `AudioManager` session methods: `setAudioSessionOptions`,
`setAudioSessionManagementMode`, `deactivateAudioSession`
- Engine state observability: `AudioEngineState`, `audioEngineState`,
`audioEngineStateStream`
- Audio processing surface: `AudioProcessingOptions`,
`AudioProcessingMode`, `AudioProcessingFailureReason`,
`AudioProcessingException`, `LocalAudioTrack.setAudioProcessingOptions`,
and the `AudioProcessingState` diagnostic types

**Deliberately left stable**

- `MicrophoneMuteMode` APIs, which mirror the Swift SDK enum that has
been stable since 2.10.0
- Speaker output preference methods, which the deprecated `Hardware`
APIs reference as migration targets
- `AudioCaptureOptions`, which predates 2.9.0. Only the new
`AudioProcessingOptions` interface it implements is marked
- Engine availability and `externalCallSystem` were already marked in
livekit#1127

Annotations only, no behavior change. No changeset since this does not
alter released behavior, but happy to add one if we want it called out
in the changelog.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
…ation (livekit#1138)

## Summary

Apple's automated App Review flags the literal `buttonPressed:` selector
string in application binaries (ITMS-90338 / Guideline 2.5.1) and blocks
App Store submissions. Since the plugin compiles `ios/Classes/**` into
every app, any iOS app using `livekit_client` is exposed — even if it
never uses screen sharing or ReplayKit.

`BroadcastManager.showPicker(for:)` constructed and performed the
private selector to programmatically activate the system broadcast
picker (reached via `setScreenShare` → `broadcastRequestActivation`
method channel). It now activates the picker using only public API:
locate the picker's `UIButton` subview and fire it via `sendActions(for:
.touchUpInside)`. If the button is ever not found (e.g. a future OS
changes the view hierarchy), a warning is logged instead of silently
no-opping.

No API changes; runtime behavior is identical.

## Verification

- `rg buttonPressed ios/ macos/` returns no matches, so the selector
string no longer reaches downstream binaries
- `swiftc -typecheck` against the iOS simulator SDK passes for the
changed file and its dependencies
pblazej and others added 12 commits July 28, 2026 17:39
Added AGENTS.md (agent/contributor guidelines distilled from repo
history) with CLAUDE.md pointing to it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Changes the default `renderMode` from `texture` to `auto`. For now
`auto` resolves to texture rendering on all platforms, so behavior is
unchanged. This lets the SDK change how `auto` resolves in a future
release without an API break.

Platform view support and related fixes from the original diff already
landed via livekit#1116 and livekit#1135.
## Summary

Release 2.9.0 — consumes all 20 pending changesets (8 minor / 12 patch →
minor bump), matching the published `v2.9.0-dev.0` pre-release line.

### Highlights
- **Added**: certificate pinning, RPC v2, audio engine availability +
`externalCallSystem` mode for CallKit, AudioManager session/routing
APIs, ConnectionCheck utility, microphone mute mode, runtime audio
processing controls, desktop audio frame capture, VideoTrackRenderer
placeholder
- **Fixed**: removal of the non-public `buttonPressed:` selector (App
Store rejection risk), Android media routing / `selectAudioOutput`,
audio processing lifecycle fixes, macOS platform video rendering, single
disconnected event, video render mode switching

### Release bookkeeping / repo organization
- Version bumped to 2.9.0 in `.version`, `pubspec.yaml`, both podspecs,
`lib/src/livekit.dart`, and README
- **Backfills the missing `2.8.1` CHANGELOG section** — 2.8.1 was
released from its branch without merging bookkeeping back, leaving main
at 2.8.0 with two already-released changesets still pending; this PR
reconciles that (the 2.9.0 section excludes the two entries that shipped
in 2.8.1)
- Cherry-picks the `docs/` → `doc/` pub layout chore that was stranded
on `release/2.9.0-dev.0`
- Declares the certificate-pinning test fixture key in `false_secrets` —
without this, pub.dev **rejects the publish** (validated via dry-run)

## Verification

- `dart pub publish --dry-run` passes (only the intentional
`flutter_webrtc` exact-pin warning remains, same as prior releases)
- `flutter analyze` clean on the flutter-callkit example against this
branch (only expected `@experimental` warnings)
- No stale `2.8.0` version references remain

## After merge

1. Tag `v2.9.0` on main → publish workflow pushes to pub.dev
2. Delete `release/2.9.0-dev.0` (superseded by the stable release)

Note: the changeset-check workflow will warn "No changeset found" —
expected for a release PR that consumes `.changes/`.
Adds a new button on the bottom control bar for the "rpc tester":

<img width="100" height="103" alt="Screenshot 2026-05-15 at 11 09 19 AM"
src="https://github.com/user-attachments/assets/5d7db331-f817-4321-9a22-4834d0966612"
/>

When clicked, it opens up a new panel which has two sections:

<img width="508" height="855" alt="Screenshot 2026-05-15 at 11 09 04 AM"
src="https://github.com/user-attachments/assets/0ea49592-18d3-49b9-a603-959a8d8de175"
/>

The top section allows sending RPC messages - select a destination
participant, topic, and enter a payload, and click "Send". Note that
there are two payload presets - `"hello world"` and `"X" * 20000` to
provide some easy to generate data to test some edge cases.

The bottom section lets you configure rpc handlers. Enter a rpc handler
topic and a static response, and submit. Then when this participant
receives a RPC request, the request will be responded to with the static
response. This static response also has the same payload presets as the
request.

> [!WARNING]
> This pull request was LLM generated and has only been lightly reviewed
by a human. The author has tested this and confirms it works in the
happy path, but no other validation has been done.
> 
> A more thorough review of this needs to occur before it could be
merged.

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
Sending zero instead of null confuses other sdks (at least rust), e.g.
[here](https://github.com/livekit/rust-sdks/blob/e37adc1dc4cb90322733fcb0fe1155262f309ff1/livekit-data-stream/src/incoming.rs#L325)

---------

Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
…#1146)

## What

Dart 3.13 (shipping with Flutter 3.47, currently in beta) no longer
keeps null promotion of captured variables across `await`. The video
`negotiate()` closure in `LocalParticipant` dereferences the captured
nullable `publishOptions` after an `await`, which now fails to compile:

```
lib/src/participant/local.dart:384: Error: Property 'videoCodec' cannot be accessed on 'VideoPublishOptions?' because it is potentially null.
Info: Variable 'publishOptions' could not be promoted due to an 'await' or 'yield'.
```

This breaks every app that depends on `livekit_client` once users update
to Flutter 3.47, including the agent starter example.

## Fix

Pass the publish options into the `negotiate()` closure as a
non-nullable parameter instead of capturing the nullable variable. At
both call sites `publishOptions` is already promoted non-null in the
enclosing function body, so this needs no null assertions at all. The
audio closure gets the same treatment for consistency, which removes the
pre-existing `publishOptions!` there too.

## Verification

- `flutter analyze` on 3.47 beta goes from 5 errors to 0, and stays
clean on stable 3.44
- All unit tests pass
- The agent starter example builds on 3.47 beta with this patch applied
to the resolved package
## What

Adds Swift Package Manager support for the iOS and macOS plugins,
following the Flutter SPM plugin convention:

- `ios/livekit_client/Package.swift` and
`macos/livekit_client/Package.swift`, with `Sources/livekit_client/`
symlink trees into `shared_swift/` (and the iOS broadcast files in
`ios/Classes/`), so the existing sources stay the single source of
truth.
- The packages depend on flutter_webrtc's Swift package
(`../flutter_webrtc`, resolved by the Flutter tool) and consume its
`flutter-webrtc` and `WebRTC` products. Reusing flutter_webrtc's WebRTC
binary target keeps a single copy of the xcframework in the package
graph, mirroring how the podspecs share the WebRTC-SDK pod today.
- The example app's Xcode projects get the SPM integration committed
(package reference, product dependency, prepare pre-action in the Runner
scheme), matching what `flutter` would generate.

CocoaPods is unchanged: podspecs and `Classes/` are untouched, and apps
with SPM disabled keep building through pods.

## Depends on

- flutter-webrtc/flutter-webrtc#2062, including the `WebRTC` library
product export. **This PR can only land after a flutter_webrtc release
that ships SPM support**, at which point the pinned flutter_webrtc
version needs a bump.

## Testing

With Flutter 3.44.1 (SPM on by default) and a local
`dependency_overrides` pointing flutter_webrtc at the PR 2062 branch:

- `flutter build macos --debug` and `flutter build ios --debug
--simulator` of the example both succeed.
- Both plugins compile as Swift packages (verified `LiveKitPlugin` /
`FlutterWebRTCPlugin` symbols in the app binaries) with exactly one
embedded `WebRTC.framework`.
- `permission_handler_apple` has no SPM support and keeps building
through CocoaPods in hybrid mode.

Note for local testing: the Flutter tool copies plugins that declare
SwiftPM dependencies on other plugins into the app's build directory
with rsync. With the example app living inside this repo that copy
recurses into itself (flutter_tools bug, to be filed upstream).
Workaround is to make `example/build` a symlink to a directory outside
the repo before building.

---

## Update 2026-08-03

flutter_webrtc 1.6.0 shipped with Swift Package Manager support, so this
PR is unblocked:

- Bumped the pinned flutter_webrtc to 1.6.0 and removed the need for any
local overrides. Verified the example builds via SPM against the
released package from pub.dev, including the versioned pub cache path
rewrite.
- Added a CI step to the iOS and macOS jobs that symlinks example/build
outside the repository, working around the Flutter tool's rsync
recursion when a plugin with SwiftPM plugin dependencies hosts its own
example app (upstream flutter_tools bug, to be filed).
- Merged main (RPC tester, data stream fix, Dart 3.13 fixes).
## What

Two small additive APIs, both marked experimental:

- `ScreenSelectDialog.show(context)` shows the existing desktop picker
dialog and returns the selected source id as a `String?`, null when the
user cancels. The id goes straight into
`ScreenShareCaptureOptions.sourceId`.
- `Hardware.instance.requestCapturePermission({fullScreenOnly})` wraps
the capture permission request on Android and macOS. On platforms that
need no upfront request it returns true, so callers can use it
unconditionally.

## Why

Starting a screen share currently forces downstream code to import
`flutter_webrtc` directly for `DesktopCapturerSource` (the picker result
type, which this SDK does not re-export) and
`Helper.requestCapturePermission`. That is why `livekit_components`
declares its own `flutter_webrtc` dependency with an exact version pin,
and the pin drifting out of sync with ours is what currently downgrades
users to livekit_client 2.6.1 (livekit/components-flutter#49, reported
again via the agent starter example).

With these two helpers, components and apps can drive the whole screen
share flow through `livekit_client` alone. The follow-up components
release drops its `flutter_webrtc` dependency entirely, so this class of
resolution conflict cannot recur.

## Notes

- No behavior change to the dialog itself, the static helper wraps the
existing `showDialog` pattern
- The dialog customization params are threaded through `show()`
- Web safe, the example app compiles for web with these changes
…it#1140)

Every publish workflow run in this repo's history has failed with exit
code 65. The dart-lang reusable publish workflow runs a non-interactive
`dart pub publish`, which treats any validation warning as fatal. Our
exact `flutter_webrtc` pin is intentional (the native WebRTC-SDK pods
must match between the two packages) but always produces the 'should
allow more than one version' warning, so the workflow can never succeed
and every release so far has been published manually.

This replaces the reusable workflow with an inline job that publishes
with `--force`, which downgrades warnings to informational.
`dart-lang/setup-dart` is kept solely for pub.dev auth: with `id-token:
write` it mints a temporary pub.dev token from the GitHub OIDC token and
registers it in the shared pub config, which the Flutter-side `dart pub
publish` then picks up. The pub.dev trust configuration is based on
repository and tag claims, not the workflow definition, so no changes
are needed on the pub.dev side.
## What

Folds the pending changesets into the 2.10.0 release. Version propagated
by `scripts/create_version.dart`, all version checks pass, `.changes/`
emptied.

## Changelog

* Added: Swift Package Manager support for iOS and macOS. CocoaPods
remains fully supported. (livekit#1142)
* Added: `ScreenSelectDialog.show` and
`Hardware.requestCapturePermission` so apps can start screen share
without importing flutter_webrtc (livekit#1147)
* Fixed: Compile errors on Dart 3.13 where nullable publish options are
no longer promoted across await (livekit#1146)
* Fixed: Omit data stream totalLength when size is unknown (livekit#1141)
* Fixed: Harden the example RPC tester (livekit#1085)

Minor bump driven by the SPM and screen share additions.

## Why now

Dart 3.13 (Flutter 3.47 beta, soon stable) fails to compile every
published livekit_client, so livekit#1146 needs to ship before 3.47 lands. The
screen share helpers unblock livekit/components-flutter#52, which
removes the flutter_webrtc pin conflict that downgrades users to
livekit_client 2.6.1 (livekit/components-flutter#49).

## After merge

Tag `v2.10.0` to publish. First run of the reworked publish workflow
from livekit#1140 (tag guard, pin-warning-only validation gate, forced
publish).

Downstream: components-flutter livekit#52 switches to `livekit_client:
^2.10.0`, then a components release and an agent-starter-flutter bump.
# Conflicts:
#	android/src/main/kotlin/io/livekit/plugin/LiveKitPlugin.kt
#	lib/src/participant/local.dart
#	lib/src/support/native_audio.dart
#	pubspec.lock
#	pubspec.yaml
#	test/core/rpc_test.dart
RPC V2 moved the ack and response bookkeeping off LocalParticipant into
RpcClientManager, which the Room holds privately, so the pendingRpcAckCount
and pendingRpcResponseCount getters this fork carried no longer have a
home. Consumers watching for RPC backlog had no replacement.

RpcClientManager merges both waits into one map, so the ack and response
counts can no longer be reported separately; this exposes the single total
the new structure supports rather than inventing a split.
@tamirh tamirh self-assigned this Aug 8, 2026
tamirh added 2 commits August 8, 2026 19:14
PerformRpcParams.ackTimeout survived the rebase as a declaration that nothing
reads, so every call fell back to a fixed 7s round-trip allowance. On a device
whose event loop stalls for longer than that, a delivered RPC fails with
connectionTimeout while the recipient is still working, and the recipient is
handed a work budget computed from the wrong subtraction.
Both sides appended a `when` branch after "stopAudioRenderer": ours
"warmupFFT", upstream's "setAudioProcessingOptions". The merge kept both
bodies but dropped the brace between them, so "setAudioProcessingOptions"
parsed as a statement inside the "warmupFFT" branch and the whole class
stopped compiling — onDetachedFromEngine fell out of the class body and
Kotlin reported LiveKitPlugin as abstract.
@tamirh
tamirh merged commit 4982edd into ello/main Aug 12, 2026
@tamirh
tamirh deleted the tamir/xp-2460-upstream-v2.10.0 branch August 12, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.